Re: SquidNG page (Was: Direct mapped like caching)

On Tue, Nov 07, 2000 at 02:08:49AM -0600, Joe Cooper wrote:
> BTW-I've posted tarballs at my website of the reiser_raw/butterfly
> squid, if folks want to take a look. I've also written to the
> SourceForge site folks to open up the CVS there, so that it will be
> available via anonymous CVS. But I think merging it into the
> squid.sourceforge.net cvs will come soon as well. (The website is here:
> http://www.swelltech.com/pengies/joe/squidng.html )

Without the pages like this the software would never get used, only
talked about ;-) Thank you Joe.

Here is a little supplement to your page, can you add it?

Thanks,

-- sizif

PS Current reiserfs_raw patch (for 2.4.0-test10) is available at
ftp.botik.ru:/pub/local/sizif/squid/reiserfs_raw_20001108.dif.bz2.
Note that KAIO syscall number has changed again, so you have to
recompile your squid.

---------8<-------------------------

Butterfly cache dirs should be configured in squid.conf like this:

cache_dir butterfly -1 /cache1 threads=N
cache_dir butterfly -1 /cache2 threads=M
...

The optional threads= paremeter specifies how many KAIO threads should
be created to serve I/O for this directory. (Actually there is a
common pool of threads for all directories, the total number of
threads being N+M+...). The default is threads=8 (eight threads per
butterfly cache dir).

There is also the refuse_swapout=N parameter. If the number of KAIO
operations in flight grows beyond N (which means the disk subsystem is
overloaded), squid will defer swap-out operations. This allows squid
better cope with load bursts (but damages hit ratio if bursts are
longer than can fit in cache_mem). The default is refuse_swapout=1500
(probably too high).

Using multiple butterfly cache-dirs is OK and good for perfomance if
the cache dirs reside on different physical disks. Squid will
distribute the objects among the cache_dirs uniformly using a hash
function. However, the butterfly cache dir selection algorithm is
currently primitive and assumes all butterfly cache dirs are of equal
size. If you make them non-equal, squid will still work, but the
objects that fell into smaller cache dir will have shorened lifetime
-- that is, you will get suboptimal cache replacement and hence worse
hit ratio.

Also, adding, removing, or re-ordering cache_dirs is harmful, as these
operations change the cache dir selection. Again, squid will work,
but fail to find many of the previously cached objects (the selection
function is changed, so they instantly appear in a wrong cache_dir).
That is, cache dir addition/removal/reordering is nearly equivalent to
removing your cache contents -- it causes immediate drop in hit ratio,
with subsequent slow climb back to normal.

---------8<-------------------------
Received on Wed Nov 08 2000 - 09:06:29 MST