Re: Malloc packages

Andres Kroonmaa wrote:

> - stupid OS memory allocator. Solaris is notorious for that. It
> allocs memory, but never frees. Process at any time is as large
> as its peak memory usage has ever been.
> Fix: --enable-dlmalloc
> I'm not sure how is default Linux malloc tuned, but I'd not be
> surprised if it has similar behaviour.

The standard Linux malloc implementation is based on Doug Lea Malloc.
Have not seen any signs of stupidity there in my tests, exept possibly
the fact that large allocations is not page aligned but page+4 (not that
malloc guarantees page alignment or even should...)

> - idle pools limit. Default is "none", which means 2G. mempools do
> not return memory to OS. Heavy fragmentation occurs over time.
> Fix: memory_pools_limit=8M. Going much up doesn't really help.

Fully agree, and I always have this set to about 8MB, less where memory
is a scarse resource.

> - increased memory footprint of Squid. I've seen this when I compiled
> in heap-replacement. Not sure how its now, but when I tried it, it
> consumed about 2-4 pointers per storeentry in addition. Together
> with malloc roundups this was quite notable. One of my 2.3 squids
> is experiencing this. heap memory overhead is not accounted for
> as it doesn't use mempools.

True. The heap in the heap policies are not accounted. It is allocated
as one large blob and can be seen very easily when analyzing memory
allocations with a malloc debugger/tracer.

> I can't say that there isn't any leaks. It just doesn't show up
> for me. If we know where leaks are, then we can address them.
> As I understand, noone really knows if and where they are.

There quite likely are some leaks at different places, but not in normal
requests paths. I would however not be too surpriced if there is some
leaks in the ntlm code as it is a fairly large new piece of code which
has not been much audited or traced.

Regards
Henrik
Received on Wed Oct 03 2001 - 02:40:28 MDT