On Thu, 2 Nov 2000, Andres Kroonmaa wrote:
> I wonder how can you eliminate StoreEntry? IMHO it contains crucial
> information that allows squid to skip disk accesses. Moving parts
> of this data into squidfs doesn't seem to change much in ram usage.
> Moving this crucial information onto disks implies enormous performance
> penalty, doesn't it?
I believe there are at least two ways. One is what reiserfs folks are
doing -- moving StoreEntry information to the "smart" filesystem
structures (or, at least, that's the impression I got).
The second way is quite different. It is based on one
rule-of-thumb and one assumption:
- optimize for the most common case, if any
- most proxied HTTP transactions require disk access at
one stage on another
If the above assumption is true, then there is no significant
advantage to keep StoreEntry in RAM because response time is dominated
by disk operation anyway. Thus, one can put StoreEntry, together with
object data, on disk (the exact data-placement scheme can differ and is
not the point here).
The only big exception to the assumption above is miss versus hit
detection. Those decisions, however, do not require StoreEntry
contents in most cases! The "crucial information" you are talking
about is just the fact the StoreEntry is present (or not). Thus a
small digest that gives yes/no answers with a good probability is
sufficient.
With the scheme above, you optimize for the majority of accesses and
loose for false hits (which should be rare).
The RAM saved can be used for hot object cache (when it is fixed)
and/or for the filesystem cache to further improve the performance.
The ideas above are not new, of course. They have been around for at
least two years (probably more) waiting for somebody with enough spare
time to implement them.
Alex.
Received on Thu Nov 02 2000 - 12:09:09 MST
This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:12:54 MST