Re: Chunked mempools, a first verdict

On 3 May 2001, at 14:37, Adrian Chadd <adrian@creative.net.au> wrote:

> > All else is bunch of hacks and needed/useful helpers.
> > Eg. someone have put stats for memPool allocs/frees on the Info page.
> > For that he have added two globals to the zoo:
> >
> > extern unsigned int mem_pool_alloc_calls;
> > extern unsigned int mem_pool_free_calls;
>
> Ok, I hate external variables, because I'm an evil "I don't want
> it overwritten" person. Thats a personal thing. If it were
> me I'd have a memPoolGetStats(mempool_stats_t *stats);

 Thats done already. Only 1 pool at a time.
 Probably should extend it to include the above two. Or create a
 separate one ala memPoolGetTotalStats(mempool_stats_t *stats);

> > memPoolInUseCount(pool) - to get number of items in use for a pool
> > memPoolCleanModule() - destroy all idle pools, at shutdown (and reconfig?)
> > memPoolsTotalAllocated() - get total allocated memory, for Info page
>
> I'd combine memPoolsTotalAllocated() with the memPoolGetCallStats().

 can do that.

> > Now specific to chunked mempools:
> > memPoolCleanIdlePools - need periodic cleanup function, with access to event.c
> > to reschedule itself. Needs access to worker funcs under lib/ or must be
> > implemented in full under lib/
>
> .. or you can implemented it as a standalone function,specify in the
> API that it needs to be called periodically, and then somewhere
> in src/ have a wrapper to give it access to event.c .

 thats how it is done. memPoolClean(MemPool * NULL) does full cleanup.
 memPoolCleanIdlePools is a mere wrapper under src/. still 2 symbols.
 What confuses me is that both are counted as excessive.
 I'd understand requirement that lib should be with as small API as
 possible, and one could have unlimited number of wrappers for all
 good cases.

> stuff in lib/ really shouldn't know anything about src/ .

 ..and can't easily.

> > memPoolTune - useful, to tune chunk sizes
> > memPoolFlushMeters - need to get accounting meters uptodate before reports
>
> > memPoolReport() - cachemgr reporter
>
> Right. This dumps out the per-mempool info?
> This one could be split a little - you could extend the stats
> struct above to include an list of mempool info, and then
> have a function in src/ which converts it to cachemgr.

 yup. memPoolGetStats(MemPoolStats *stats, int index) allows to walk
 all pools, 0..x, for each pool returning all relevant data for cachemgr.
 I used index to remove the need for shared Arrays Pools struct.

> > memMeterSyncHWater() - if we want high water mark accounting.
> >
> > Now, either funcs or data structs to get statistics reported:
> > TheMeter - accounting totals for all pools
> > Array Pools - to traverse perpool accounting stats for reporting
>
> If you implement my suggestions above I don't think you need these to be
> external.

 I'll try.

> > What of the above can we discard in principle??
> >
> > Stuff with duality due to lib/ vs. src/ split:
>
> If its going in lib/ I'd say to myself "if I take the code and
> any other code it requires in lib/ and use it in my program
> that has nothing to do with squid's src/, would it still run?"

 hmm. it would. only dirty hack to reference squid_curtime needs to
 be reworked.

 Well, I've undone my control func, removed all but Report func from
 MemPoolStats, most of which went into lib and few bits into mem.c as
 Henrik suggested. I'll look into how to consolidate stats data and
 make lib totally independant from rest of Squid.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Delfi Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Thu May 03 2001 - 03:24:06 MDT