Re: Partial contenet,206

On 31/01/2013 2:11 a.m., Eliezer Croitoru wrote:
> On 1/30/2013 8:48 AM, Amos Jeffries wrote:
>> Does not matter. For starters we want to be *able* to store any
>> cacheable response.
> Since I dont know the exact progress of collapse forwarding.
> The request contains the desire to fetch partial data so now the only
> thing is to somehow store and re-serve it.
> If I remember right the vary headers are being used to create the
> store-id of the request.
>
> What is actually being done when forcing full response rather then
> partial?
>
> What I was pointing about the store-id being decided before getting
> any data is for what i'm about to write.
>
> What happens in a case when a user requests partial content and the
> server denies it and returns full response? squid cache that or not?

Yes, the same caveats about merging with existing cached Ranges apply
except that this should *replace* the whole cached Range entry with the
new object instead of merging. Merge should only happen when two Ranges
are supposedly the same object. Maybe using overlapping bytes to help
verify they are identical.

>
> A nice example for partial content that is distributed on multiple
> objects is YOUTUBE and other videos.
> One video can be composed from partial parts of one video.
> There are situations which some client fastforward to specific place
> in the video but still the player works with the chunks intelligently
> enough that only some of the chunks are out of the fixed size of a chunk.
> So eventually it's ok.

Have you actually sighted use of the Range: HTTP headers in Youtube
traffic? and have you verified that (2) beow is no longer happening (as
of 2 year ago it was)?

In my experience ...
1) they do not use HTTP Range: headers on most of the videos (at least 2
years ago it was rare).
2) they prefix each snippet of video with FLV meta headers
3) several of the HD video encodings used are VBR encodings based on
IP-layer speed detection.

Together all these details mean that no two YouTube "range" replies can
be merged together. They best one can hope for is that two clients will
make identical requests. But given that they are generated by the user
clicking along the video timeline bar the likelyhood of seeing that
happen in one proxy is too small to bother coding for.

Also, serving a proper HTTP Range reply by slicing a previously stored
video would loose that nasty FLV meta header. Effectively corrupting the
players view of teh response.

>
> So we can might try to store partial responses in a cache object
> specifically for partial response.
> This is where my question was about size since most of the clients
> will request more then just 8k chunk and we can even do some
> statistics from existing live server logs to make sure of it.
>
> This is the most practical way in the current situation.
> This indeed will open a small weakness in cases which there are two
> clients requesting the same object one with partial content request
> and the other a full response.
> A solution for that can be for squid to learn if it dosn't know how to
> serve partial content from a complete file.

Squid already has this capability. For the range_offset_limit and HIT
Range: serving capabilities.
This is mainly why I was suggesting wait on Collapsed Forwarding,
because you might find it easier to collapse the two requests down and
service the Range: request out of the full requests response. Or service
the full request using two parallel 'filler' requests around the
underway Range request - together the three background requests
consisting of a full object fetch.

Amos
Received on Thu Jan 31 2013 - 02:21:38 MST