Re: MemPools rewrite

On Tue, 31 Oct 2000, Andres Kroonmaa wrote:

> the only benefit is to avoid a pointer overhead per each pooled item.
 
> that is 8MB of overhead per 1M items (64-bit systems). Given that we
> have 3 huge pools (StoreEntry, MD5, heap_node), we currently suffer 24MB
> overhead per 1M objects. Given 4M objects, we suffer ~100M overhead
> just for keeping mempool pointers. Without chunking, malloc itself is
> adding another 100M overhead. We can have 200M wasted for 4M objects.
> Its about 20% of total memory used. I think this is excessive. disks
> are so much cheeper than ram and systems that can handle multi-GB ram.

> we can get xxxGB of disks in a cheap box and wish to handle yyM objects
> with ram limit of 1-2G, that is why I'm in the mode of reducing ram usage
> so much.

Andres,

        I understand your motivation, and will stop bothering you
about potential design/implementation problems. Again, what you
propose is doable and probably will save Squid some RAM.

        IMHO, you are trying to solve the right problem with the wrong
tools: If you are lucky, implementing complex and tricky MemPools will
save some 32-48 bytes per object while eliminating memory-resident
StoreEntry and related objects (so that *all* per-object info except,
maybe, disk "address" is stored on disk until needed) will address the
root of the problem. Thus, IMO, your solution is a complex temporary
workaround until the "right thing" can be done.

-oo-

Alex.
Received on Wed Nov 01 2000 - 14:57:51 MST