HTTP Range header

http://stackoverflow.com/questions/3303029/http-range-header

***************************

 58 down vote favorite

25

I was reading http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35 and trying to figure out how to continue a file download.

For example, suppose a file is of length 100 bytes and I have all the 100 bytes. However, I don't know what the expected file size should be, so I ask for the file and specify a Range header that looks like this:

Range: bytes=100-

Is this a valid Range request?

http http-headers header

shareimprove this question

edited Apr 16 '15 at 18:41 mins 2,27152239

asked Jul 21 '10 at 19:28 dhruvbird 2,59622432

5

Erm, the example under it cites 'bytes=9500-' as valid, so.... – Wrikken Jul 21 '10 at 19:36

1

The most current ref is RFC7233 -- httpwg.github.io/specs/rfc7233.html – Mark Nottingham Apr 16 '15 at 21:24

add a comment

3 Answers

active oldest votes

up vote 41 down vote accepted

It's a syntactically valid request, but not a satisfiable request. If you look further in that section you see:

If a syntactically valid byte-range-set includes at least one byte- range-spec whose first-byte-pos is less than the current length of the entity-body, or at least one suffix-byte-range-spec with a non- zero suffix-length, then the byte-range-set is satisfiable. Otherwise, the byte-range-set is unsatisfiable. If the byte-range-set is unsatisfiable, the server SHOULD return a response with a status of 416 (Requested range not satisfiable). Otherwise, the server SHOULD return a response with a status of 206 (Partial Content) containing the satisfiable ranges of the entity-body.

So I think in your example, the server should return a 416 since it's not a valid byte range for that file.

shareimprove this answer

answered Jul 21 '10 at 20:10 Marc Novakowski 31.4k75059

So is there any way a client can resume a download without making a HEAD call to first figure out the content length and then do the math and fetch the actual content? I mean some sort of open addressing like "give me all bytes after such and such byte..." – dhruvbird Jul 21 '10 at 20:18

5

The client will already know if it has all of the data from the original request - it should have either received a Content-Length header in the original response, or if it was chunked encoding it will have received a zero-length chunk to indicate the response was complete. If you haven't saved off this state and just have a chunk of bytes on disk, then yes you'll have to either do a HEAD request or use the Range header to ask for a byte range, and if you get back a 416 response you know you have all the bytes. – Marc Novakowski Jul 21 '10 at 20:41

I think Expect-Continue lets you do streaming chunks more or less like desired? – MJB Jun 10 '11 at 3:37

@MarcNovakowski Actually, consider the case of wget and using the -c flag. Since wget doesn't maintain any metadata about the file being complete, suppose the size of the file on disk is 99 bytes. wget will request the byte range "100-", and I feel that the server should respond with a 0 length response since the request is just 1 past the end of the file. – dhruvbird Jan 17 '14 at 3:06

add a comment

Did you find this question interesting? Try our newsletter

Sign up for our newsletter and get our top new questions delivered to your inbox (see an example).

 

up vote 92 down vote

As Wrikken suggested, it's a valid request. It's also quite common when the client is requesting media or resuming a download.

A client will often test to see if the server handles ranged requests other than just looking for an Accept-Ranges response. Chrome always sends a Range: bytes=0- with its first GET request for a video, so it's something you can't dismiss.

Whenever a client includes Range: in its request, even if it's malformed, it's expecting a partial content (206) response. When you seek forward during HTML5 video playback, the browser only requests the starting point. For example:

Range: bytes=3744-

So, in order for the client to play video properly, your server must be able to handle these incomplete range requests.

You can handle the type of 'range' you specified in your question in two ways:

First, You could reply with the requested starting point given in the response, then the total length of the file minus one (the requested byte range is zero-indexed). For example:

Request:

GET /BigBuckBunny_320x180.mp4 
Range: bytes=100-

Response:

206 Partial Content
Content-Type: video/mp4
Content-Length: 64656927
Accept-Ranges: bytes
Content-Range: bytes 100-64656926/64656927

Second, you could reply with the starting point given in the request and an open-ended file length (size). This is for webcasts or other media where the total length is unknown. For example:

Request:

GET /BigBuckBunny_320x180.mp4
Range: bytes=100-

Response:

206 Partial Content
Content-Type: video/mp4
Content-Length: 64656927
Accept-Ranges: bytes
Content-Range: bytes 100-64656926/*

Tips:

You must always respond with the content length included with the range. If the range is complete, with start to end, then the content length is simply the difference:

Request: Range: bytes=500-1000

Response: Content-Range: bytes 500-1000/123456

Remember that the range is zero-indexed, so Range: bytes=0-999 is actually requesting 1000 bytes, not 999, so respond with something like:

Content-Length: 1000
Content-Range: bytes 0-999/123456

Or:

Content-Length: 1000
Content-Range: bytes 0-999/*

But, avoid the latter method if possible because some media players try to figure out the duration from the file size. If your request is for media content, which is my hunch, then you should include its duration in the response. This is done with the following format:

X-Content-Duration: 63.23 

This must be a floating point. Unlike Content-Length, this value doesn't have to be accurate. It's used to help the player seek around the video. If you are streaming a webcast and only have a general idea of how long it will be, it's better to include your estimated duration rather than ignore it altogether. So, for a two-hour webcast, you could include something like:

X-Content-Duration: 7200.00 

With some media types, such as webm, you must also include the content-type, such as:

Content-Type: video/webm 

All of these are necessary for the media to play properly, especially in HTML5. If you don't give a duration, the player may try to figure out the duration (to allow for seeking) from its file size, but this won't be accurate. This is fine, and necessary for webcasts or live streaming, but not ideal for playback of video files. You can extract the duration using software like FFMPEG and save it in a database or even the filename.

X-Content-Duration is being phased out in favor of Content-Duration, so I'd include that too. A basic, response to a "0-" request would include at least the following:

HTTP/1.1 206 Partial Content
Date: Sun, 08 May 2013 06:37:54 GMT
Server: Apache/2.0.52 (Red Hat)
Accept-Ranges: bytes
Content-Length: 3980
Content-Range: bytes 0-3979/3980
Content-Type: video/webm
X-Content-Duration: 2054.53
Content-Duration: 2054.53

One more point: Chrome always starts its first video request with the following:

Range: bytes=0-

Some servers will send a regular 200 response as a reply, which it accepts (but with limited playback options), but try to send a 206 instead to show than your server handles ranges. RFC 2616 says it's acceptable to ignore range headers.

shareimprove this answer

edited Apr 16 '15 at 17:28

answered Sep 11 '13 at 15:21 Victor Stoddard 1,52811115

1

Who is working on the definition of Content-Duration? Link? – Julian Reschke Sep 11 '13 at 15:33

What do you do if the content is a live video stream that does not have a fixed duration? – Joel Barsotti Feb 16 '14 at 21:12

@Joel, you need to reply with a duration even if you don't know it. In that case, just try 0.0. For the client, the duration doesn't matter anyway since you can't usually scan a live stream. If the 0.0 doesn't work, just try something really high like 1000000.00. – Victor Stoddard Apr 2 '14 at 1:55

@VictorStoddard can chunked streaming be applied to regular file download where no Range header is present in client request? How should server respond in that case? – gkiko Sep 23 '15 at 10:32

@gkiko There isn't much difference other than using the Transfer-Encoding header instead of Content-Length in Chunked Transfer Encoding. Chunks can come from a single file and the server can set the chunk size. The client should buffer and piece together the chunks as they're received. Alternatively, HTTP Streaming uses prerecorded segments of a media file, where they are saved on the server as individual parts (ts files). These segments are served using regular HTTP file GET requests obtained from an index file. I've found segmenting to be tricky but that was years ago. – Victor Stoddard Sep 24 '15 at 2:20

add a comment

up vote 7 down vote

Contrary to Mark Novakowski answer, which for some reason has been upvoted by many, yes, it is a valid and satisfiable request.

In fact the standard, as Wrikken pointed out, makes just such an example. In practice, Firefox responds to such requests as expected (with a 206 code), and this is exactly what I use to implement progressive download, that is, only get the tail of a long log file which grows in real time with polling.

shareimprove this answer

answered Jan 16 '14 at 14:14 user3198011 7911

1

Please read Marc Novakowki's answer again. "satisfiable" has a particular meaning in the RFC, which he quoted. This request is not satisfiable because the requested bytes are beyond the length of the file. – Scott Lamb Nov 9 '15 at 4:49

add a comment