Re: [squid-users] Questions regarding COSS setup

Markus Meyer wrote:
> Hi all,
>
> I want to use COSS but I'm not sure if I understand all the options
> right. So this is what I want:
>
> Although our average object size is ca. 27 kB most of the files are much
> smaller. This is the distribution of about 11 M files from one of our
> Squids:
> <1kB: 19585 0,1%
> <2kB: 583402 5,2%
> <4kB: 4897854 44,3%
> <8kB: 1049084 9,4%
> <10kB: 145059 1,3%
> <20kB: 351615 3,1%
> <30kB: 182077 1,6%
> <40kB: 325084 2,9%
> <50kB: 597911 5,4%
> <60kB: 807344 7,3%
> <70kB: 742066 6,7%
> <80kB: 527370 4,7%
> <90kB: 333543 3%
> <100kB: 206064 1,8%
> >100kB: 280574 2,5%
>
> I want to let Squid do as less IO as possible. So I thought I set
> "maximum_object_size_in_memory" to 4 kB and "max-size" for COSS to 3x
> 8kB = 24 kB. The rest goes into AUFS.

Looking at that 8KB will catch 20% more than 4KB would.

>
> So my configuration lines would look something like this:
>
> cache_dir coss /web/cache/1/coss/ 20480 max-size=24576
> cache_dir coss /web/cache/2/coss/ 20480 max-size=24576
> [...]
> cache_dir aufs /web/cache/1/aufs/ 81920 290 256
> cache_dir aufs /web/cache/2/aufs/ 81920 290 256

add min-size=24576 to the AUFS to prevent them grabbing the small files
intended for COSS.

> [...]
>
> Help is needed with the following things:
>
> - "--with-coss-membuf-size" compile-time option is set to 1 MB per
> default. Does it make sense to change this value?

AFAIK, no, but you may want to test that.

>
> - How big should I make the COSS files? I thought about 20 GB on four
> disks for COSS and 60 GB on the same disks for AUFS.

I'm not sure about this.

More total size means more slices being swapped in and out to load rarer
things. Larger slice size reduces that, but increases loading time.

COSS are limited by the maximum 2^24 objects per cache_dir, like all
stores so far. So that times your average file size should give you
something like 64 GB per dir as a theoretical absolute max store size.

>
> - How do I understand "block-size"? What values should I use? I can't
> get my head around the docs in the Squid-Wiki.

Block is equivalent to inodes as I understand it. Each file is stored in
1-N blocks. A block of 512 bytes storing a 12 byte file will waste 500
bytes. As will two blocks storing a 524 byte object.

To reach your 20GB directory size you will need block-size=2048.

Going by your object distribution I'd say thats probably workable.
Though 1KB (dir size 16GB) would have less wastage.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21
   Current Beta Squid 3.1.0.15