Re: Squid-2.4 memory leaks

On 11 Oct 2001, at 23:45, Henrik Nordstrom <hno@squid-cache.org> wrote:

> Which by 100% confirms that you are seeing a servere problem of memory
> fragmentation.
>
> Why 100% certain: The free memory is "constantly" growing, not
> displaying any signs of a rough sawtooth pattern. But then there is some
> alarming sharp edges there where the amount of free memory suddenly
> grows so there may be more to it after all.
>
> I'd recommend start looking for another malloc() implementation. You say
> you have tried dlmalloc but to no avail.. odd. historically dlmalloc has
> been able to cope with Squid loads quite well on other platforms where
> the systems malloc() does not.

 dlmalloc is ok, there is nothing any heap malloc can actually do if
 something in squid is using up huge parts of memory, then releasing it
 but leaving some small bits in use in high mem.
 But I can't imagine anything in squid that wouldn't fit into 900MB of
 free space because of fragmentation...
 Really sounds like he is using broken malloc implementation, what we
 can't say about dlmalloc...
 Notice, that process size has never shrunk, which it definitely should
 have at least once during a day.

 What I can think of, is hypotetically this usage pattern:
 - request buffer grows very big, over mempool size, it is realloc()ed,
 - StoreMemBuffer is taken from heap, sticking into ram beyond reqbuf,
 - after reqbuffer is freed, hole is created in freemem.
 - this hole is being filled, partially fragmenting it.
 - new realloc can't fit into fragmented freemem, so it allocs new space.
 ... etc.

> Florin Andrei wrote:
> >
> > On Thu, 2001-10-04 at 11:58, Henrik Nordstrom wrote:
> > > Florin Andrei wrote:
> > >
> > > > Hmmm... I still see aweful memory leaks with 2.4STABLE on Irix, on a
> > > > machine that gets like 90 req/second. The process starts at approx 200
> > > > MB, then it grows like hell, until it crashes 1...3 days later, when
> > > > it's almost 2 GB in size. :-/ Tried dlmalloc, it doesn't help.
> > >
> > > Any clue on where this memory goes? I.e. any sign in the cachemgr pages?
> >
> > Ok, here are the most significant MRTG graphs for my proxy:
> > - the memory usage (1200.0 k actually means 1200MB = 1.2GB)

> > As you can see, i have lots and lots of free memory, and with no
> > explanation as of why. :-)

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Online
 Tel: 6501 731, Fax: 6501 725
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Fri Oct 12 2001 - 03:46:05 MDT