RE: Chunked mempools, a second verdict

This way the cleanup interval can be made adaptive:
while doing memPoolClean(), gather stats about the catual
cleanup done, and adjust mem_stat_counter raising itor
loweriung it using some heuristics on the cleanup stats.
something like: if the actual cleaned-up pools are
less than x%, raise mem_stat_counter by ((target-x)/pct)% (and obviously
the opposite applies). By tuning target you can balance between
CPU spent on cleanup and memory wasted on unused pools. By
varying pct you define how responsive the algorithm is
(better having it somewhat slow, i.e somewhere like 5 or 10)

-- 
	/kinkie 
> -----Original Message-----
> From: Henrik Nordstrom [mailto:hno@hem.passagen.se]
> Sent: Friday, May 04, 2001 8:59 PM
> To: Andres Kroonmaa
> Cc: squid-dev@squid-cache.org
> Subject: Re: Chunked mempools, a second verdict
> 
> 
> Andres Kroonmaa wrote:
> 
> >  like what functions? Alloc/Free out of question, they are tuned for
> >  performance, and any overhead is unwanted there. GetStats? Iterate?
> >  We don't have much funcs left in fact. If we manage to make pools
> >  selfcleaning, we'd loose currently best place: memPoolClean.
> 
> memPoolClean is probably a good place, and it will still be there when
> pools are selfcleaning even if not visible in the external API.
> 
> For the purpose in MemPools, simply sampling the time every N
> memPoolAlloc call (wher N could be 500 or any other arbitrary number)
> should be more than sufficient. What you are interested in is 
> long term
> averages, not instant exact values.
> 
> I think making MemPools self adjusting by piggy back on
> memPoolAlloc/Free will only have a negliable impact on performance:
> 
>    if (!mem_stat_counter--)
> 	memPoolClean()
> 
> and in memPoolClean() reset the counter, read the current time and so
> on.. (a better name for memPoolClean then is probably maintainPools or
> some other internal name)
> 
> Note: A higher precision time can be emulated from the 
> sampled time and
> mem_stat_counter.
> 
> Also, I think the maxage argument of memPoolClean() should be made a
> tunable rather than a argument to the clean function, which in turns
> calls for an extensible memPoolGlobalTune() call replacing
> memPoolSetIdleLimit and the maxage memPoolClean() argument.
> 
> #define MEMPOOL_TUNE_IDLELIMIT	1
> #define MEMPOOL_TUNE_MAXAGE 2
>   memPoolGlobalTune(int variable, int value)
> 
> --
> Henrik
>