Re: [squid-users] "Quadruple" memory usage with squid

Robert Collins wrote:
> On Mon, 2009-11-23 at 21:40 -0500, Linda Messerschmidt wrote:
>>> Maybe. We would like to diagnose this problem and fix it properly,
>> but if
>>> its too much hassle you can go that way.
>> It would definitely be my preference to diagnose and fix the problem
>> and I can live with a fair amount of hassle to get there. (Unless you
>> are saying "you are using redirectors" is the problem, in which case
>> memfs it is. ;-) )
>
> redirectors should be fine.
>
> If the page tables of squid are swapped out or phenominally huge, fork
> could be slow.
>
> You could test using vfork instead of fork (be sure to see the man page
> and you may need to change the child execution code a little.
>
> http://www.netbsd.org/docs/kernel/vfork.html has some interesting notes
> from the BSD world about this.
>
> -Rob

I would agree with Rob but find it strange that there is no copy-on-write...

vfork will probably help but has side effects and I don't know
if the Squid code is suitable for vfork.

My guess is that the delay of fork() is probably caused by the fact that the
process' TLB table is huge. 64-bit machines support page sizes larger than 4K
which reduces the TLB usage and the total TLB size a lot.
The detailed release notes of FreeBSD state:
[amd64, i386] The FreeBSD virtual memory subsystem now supports fully transparent use of
superpages for application memory; application memory pages are dynamically promoted to or
demoted from superpages without any modification to application code. This change offers the
benefit of large page sizes such as improved virtual memory efficiency and reduced TLB
(translation lookaside buffer) misses without downsides like application changes and virtual
memory inflexibility. This is disabled by default and can be enabled by setting a loader
tunable vm.pmap.pg_ps_enabled to 1.

I would give it a try and set vm.pmap.pg_ps_enabled to 1

-Marcus
Received on Tue Nov 24 2009 - 13:09:46 MST