Re: Performance tweaks for large caches

At Fri, 14 Jun 1996 01:43:11 +1000, Mark Treacy writes:
>> A summary of the changes follows
>> Our ~9G cache holds 500,000 objects. With only 100 directories, this
>> means that each directory holds 5,000 objects. After a week or 2 the
>> directories develop holes and we had directories 100k in size. This
>> makes pathname lookups rather expensive, particularly if the activity
>> hits many such sized directories (flushing various kernel caches).
>> I have changed the swap area to a 2 level directory heirarchy. The changes
>> here construct 16 first level directories and 256 second level directories
>> for a total of 4096 cache directories. This reduces the size of all the
>> directories to be less than the typical logic block size (8k).

That's a fairly productive change, although we haven't really noticed
it, partly because our cache systems are configured across more
filesystems on more spindles to maximise seek bandwidth. That doesn't
currently help Squid (they were specced for CERN :), but hopefully it
will shortly once I update my asynchronous IO patch for beta17 and
send it in. (Although Squid tries to use non-blocking IO, we haven't
found a Unix that actually implements that for disk IO, with the
result that all disk IO is sequential and causes a bottleneck with
large numbers of clients --- these patches allow Squid to actually
kick off multiple IO requests in parallel on a suitable system (eg
Irix)).

>> Background Restore takes too long if there are no events occuring
>> (50 objects read once a second), added a flag to cause squid to block
>> until the restore is complete.

We did notice that with a big cache restores take too long, but
haven't hada chance to look into it. Our goal is to minimise
downtime, so blocking until restore is complete is unacceptable. I'll
have a look at the code later and see if we can come up with a reload
algorithm that basically soaks up all the spare capacity on the
machine to reload as fast as possible whilst trying not to impact
active connections.

Pete.
Received on Thu Jun 13 1996 - 09:53:17 MDT