Re: tcp proxy hackery

On Mon, Mar 17, 2008, Robert Collins wrote:

> > It maxes out on my kit at the above speed; but at 32k replies it hits 3100 req/sec
> > and close to a gigabit. I'll whack a recent linux distro+kernel on the test boxes
> > in a few days and see how it compares.
>
> Sounds like its going well. I'd love to see a similar benchmark for
> memory allocations - something that exercises the slab and buffer
> allocator in squid, so we can tune that in -3.

Benchmarking allocators is easy. Benchmarking allocators in real-world situations
is less easy. The whole point of this work is to provide the minimum code needed
to provide efficient(!) HTTP proxy services which can then be benchmarked in a
real world situation.

I'm not sure how that'll feed into Squid-2 and Squid-3 in a useful fashion except
to say "this is whats possible". The way Squid-2 and Squid-3 actually use
memory is IMHO pretty horrible.

It'll also answer the question of "is it worth having a slab allocator in 2008".
My gut feeling says yes, but only for very small allocations. A lot of work has
gone into memory allocators since the original slab allocator design in 1999/2000
and I think trying to build a local allocator that scales past lots of CPUs is
probably a waste of effort. I'll wait and see. I just wish the current malloc's
let me take a "hint" for the allocation size and provide it again on free() to
bypass whatever pointer -> memory region mapping lookup it has to do.

Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -