Re: [squid-users] Squid HD Limitation

Mr. Issa(*) wrote:
> Ok mate the scenario is as follows:
>
>
>
> Clients -->> Mikrotik Router -->Squid BOX
>
> Traffic of port 80 is Natted on the MK router to the Squid Box.
>
> Squid is installed on Debian lenny. Debian is running on an intel
> SSR212MMC2 system with the following specs:
>
> 32GB of Rams
> 8Terra Hard Disk on a HW RAID controller each 4hds are RAID10 and
> mounted as 2 cache directories

My opinion of RAID behind Squid is very poor. Avoid if at all possible.

HW RAID is claimed to be workable though, particularly as the price
range and quality goes up.

The RAID0 operations of striping are duplication of the object spread
Squid itself performs between its cache_dir. So all you really gain
there is a larger total disk (Squid don't care about that) and risk of
loosing the entire lot if any stripe platters die.

The RAID1 can provide backup against disk failure (Squid like that).
Thats about the only benefit there. Con is just whatever overheads the
RAID controller has for a LOT of writes.

>
> cache1 & cache2
>
> /dev/sdc1 1.8T 87G 1.7T 5% /cache1
> /dev/sdd1 1.8T 87G 1.7T 5% /cache2
> pr0xySRV:~# free -m
> total used free shared buffers cached
> Mem: 32242 32065 176 0 1361 7570
> -/+ buffers/cache: 23133 9109
> Swap: 1898 0 1897
>
> squid conf:
> cache_dir aufs /cache1 1335043 16 256
> cache_dir aufs /cache2 1335043 16 256
>
> cache_swap_low 90
> cache_swap_high 95
>
> cache_replacement_policy heap LRU
> memory_replacement_policy heap LRU
>
> The problem is as you can check the previously attached Graph .. the
> Small tick at an hourly interval.... and the Problem started to appear
> When we had 100GB of cache as a sum of both cache_dir

Everythign you say just above confirms for me that the 5% swap high/low
difference will be the problem.
  * The default garbage collection is ... done hourly. It won't
completey halt requests going through, but will take up a relatively
large chunk of CPU walking the index scheduling stuff to be removed.
  * the removals only get done deeply when cache_swap_low % of the cache
is full.
  * when cache_swap_high % of the cache is reached things get extra
strict and use even more CPU doing removals.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE24
   Current Beta Squid 3.1.0.17