Re: Partial contenet,206

On 30/01/2013 2:07 p.m., Eliezer Croitoru wrote:
> This topic was discused couple times and I want to get back to it with some highlights.
> Since we want to maximize cache efficiency we must apply something that will allow partial content caching.
> The situation now is kind of all or nothing.
> One of the reasons for that alone is that squid stores objects in a way that dosn't allow cached files/objects manipulation.
> The cache objects are in kind of RO mode but this might not be an issue for all solutions that can be offered to solve partial content caching.
> Things we should be considering how much do we want to gain from partial contenet caching? a 8kb cache gain 512kb and more?

Does not matter. For starters we want to be *able* to store any
cacheable response.

> do we want a perfeft object in cache?
We do. But definition of 'perfect' when it comes to Ranges... we want
the bytes to be prefectly matching the servers copy, even if we are
storing a sub-range of the whole.
Extreme care taken to validate that two ranges are from the exact same
variant before merging, etc. To store them separately if there is any doubt.

> or partial data can be inserted to a object?

We want that too of course.

The best proposal I've heard put forward years ago was to store the
headers meta data and the object entity in separate cache files and when
storing ranges to open the file size and fill only the stored part.
Possibly with background fetches as low-priority traffic to fetch the
other ranges and make it perfect for future requests.
  * we already have range_offset_limit and quick_abort settings to
control these. Will probably require some adjustment in what they do
and/or new directives.
  * the background fetch part of this is stuck behind a re-design of the
async fetch code in 3.x (also blocking Collapsed Forwarding [see Alex]).

NP: this whole topic is probably best looked at after the Collapsed
Forwarding feature is built into 3.x. The background async requests for
Ranges will need to be collapsable in case any client wants to do its
own range filling (ie client Squids).

> I dont know how cache like varnish or orhers do this.
> any ideas? I was thinking of asking the varnish developers about their approach to the subject.
> Anorher issue is that the cache object id (store-id)is decided before any response is being recived from the server.
> This can also be solved in a way but first before any other step please feel free to throw anything you have.(i know there was another thread in the past)

Amos
Received on Wed Jan 30 2013 - 06:49:02 MST