Re: MemPools rewrite

On 1 Nov 2000, at 14:57, Alex Rousskov <rousskov@measurement-factory.com> wrote:

> > we can get xxxGB of disks in a cheap box and wish to handle yyM objects
> > with ram limit of 1-2G, that is why I'm in the mode of reducing ram usage
> > so much.
>
> I understand your motivation, and will stop bothering you
> about potential design/implementation problems. Again, what you
> propose is doable and probably will save Squid some RAM.

 On contrary, I very much appreciate your warnings and comments!
 My coding experience is very limited, and thats why I turn to you
 guys for guidance in the first place. I think this have saved me
 from doing something stupid and wasting time on discovering it
 the hard way. If I object something, then only to better understand
 the reasoning and implications, and to find ways to solve for both
 goals, reduce memory usage and not brake anything. So don't stop
 bothering me ;)

> IMHO, you are trying to solve the right problem with the wrong
> tools: If you are lucky, implementing complex and tricky MemPools will
> save some 32-48 bytes per object while eliminating memory-resident
> StoreEntry and related objects (so that *all* per-object info except,
> maybe, disk "address" is stored on disk until needed) will address the
> root of the problem. Thus, IMO, your solution is a complex temporary
> workaround until the "right thing" can be done.

 You are very right here. I'm trying to make a temp solution. I've had
 to move mempools into lib to allow more stuff get mempooled, and as I
 have found lots of very small allocs that need to be pooled, it has
 naturally occured to me that we have very high overhead.

 Reducing StoreEntry overhead is definitely the next target, but this
 cannot be easily done without drastic changes. I've started all the
 stuff from basic need to Pool everything we can in squid, and came
 to realise that we want to reduce malloc overhead itself. I hope to
 get ready before 2.4 is released, or shortly after. I any case, I
 hope this work can be included into 2.4 already, and further work
 on StoreEntry size left for next devel.

 In fact, I wanted to spark off another thread on how to reduce squid
 memory footprint in general. And I think we'll have to discuss that.
 But also, if we reduce alot the inmemory database item sizes, we'll
 see proportionally higher overhead from malloc itself. In this sense,
 optimising mempools should have positive effects in the future too.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 Delfi Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Thu Nov 02 2000 - 00:37:19 MST