Re: [squid-users] memory utilization

On 21/11/2011 6:06 p.m., Benjamin wrote:
> Hi Amos,
>
> Really your support is grateful to us.You always help us to
> understanding of squid and its work pattern.Thanks for your kind support.
>
>> On 21/11/2011 1:33 a.m., benjamin fernandis wrote:
>>> Hi All,
>>>
>>> I configured squid box to get good cache performance and for that i
>>> set cache_mem and object size in cache.
>>>
>>> cat /etc/squid/squid.conf | grep cache_mem
>>> cache_mem 6144 MB
>>>
>>> cat /etc/squid/squid.conf | grep -i maximum_object_size_in_memory
>>> maximum_object_size_in_memory 1 MB
>>>
>>> And whenever i check memory usage at OS level while squid is serving
>>> to traffic. it shows me
>>>
>>> free -mto
>>> total used free shared buffers
>>> cached
>>> Mem: 7995 345 7650 0
>>> 30 78
>>> Swap: 8999 0 8999
>>> Total: 16995 345 16650
>>>
>>>
>>> So as per my squid setup i set 6GB , So why free -mto showing me that
>>> only 345 mb is used and 7650 mb is free.
>>>
>>>
>>> As per my understanding , if i assign 6GB RAM to squid then 6GB will
>>> be deducted from my actual memory and then OS has that remaining
>>> amount of memory.
>>>
>>> My perception is right ?
>>
>> Yes your understanding is generally correct.
>>
>> There is one circumstance I can think of when the OS might show lower
>> than cache_mem usage. That is when memory pooling control has been
>> disabled when building Squid. In that case Squid will not
>> pre-allocate any memory for use.
>>
>
> If suppose memory pooling control has been disabled in that case, does
> squid will not have full control on memory utilization or suppose i
> assign 6 GB ram to squid, so does squid use 6gb ram for caching ?
>
> Memory pooling control is mandatory to enable in squid while we are
> talking for memory utilization or work flow of squid ?

No. Squid will continue to operate as normal and will allocate the 6GB
cache_mem eventually, it just means the OS is aware of how much
cache_mem area has (or not) been used at any given time. And squid will
continuously be allocating/deallocating memory blocks. You can see the
churn in the cachemgr memory allocation report.

The benefits of pooling is speed of avoiding allocating new blocks of
RAM. We have not had any actual speed measurements in many years to get
actual numbers. But when I tested it casually about Mar this year the
default malloc library most OS supply was still slower than pools. For
higher speed you will be needing some modern library like talloc which
seemed as good or better than pools (though no numbers means that may be
an illusion). The other allocator libraries competing with talloc are
also very close.

>
>>>
>>>
>>> What is the purpose of --enable-async-io option in squid?
>>
>> To enable Asynchrnous I/O (AIO) for threaded read/write to disks.
>> This is required for AUFS.
>>
> Yes, i am using AUFS.
>>> i read on internet that it enable more performance while using more
>>> thread with that option
>>
>> Yes. Up to the amount of AIO threads your disk controller can work
>> with efficiently. There is a point at which the threads become too
>> many for the controller and things get slow again. I'm not personally
>> aware what that upper limit is.
>>
> i installed squid rpm which i got from my os distro.i used fedora 15
> 64 bit.But i can't find that option in squid -v command.so i guess
> that is not included while installing squid from rpm.
>
> So --enable-async-io is mandatory while we r using AUFS or while we
> are talking for performance how many thread is idle for squid. On
> internet i see 128 thread configuration at many blogs.
>
> Suppose --enable-async-io is not enabled with squid so in that case
> what is default thread for AUFS?
>

Well, its equivalent to "--with-aufs-threads=N_THREADS --with-pthreads
--enable-storeio=ufs,aufs", so they may have set those individually
instead. Or just using the default of 16 per cache_dir.

Amos
Received on Mon Nov 21 2011 - 05:51:15 MST