Re: tcp proxy hackery

On Mon, Mar 17, 2008, Adrian Chadd wrote:

> Benchmarking allocators is easy. Benchmarking allocators in real-world situations
> is less easy. The whole point of this work is to provide the minimum code needed
> to provide efficient(!) HTTP proxy services which can then be benchmarked in a
> real world situation.
>
> I'm not sure how that'll feed into Squid-2 and Squid-3 in a useful fashion except
> to say "this is whats possible". The way Squid-2 and Squid-3 actually use
> memory is IMHO pretty horrible.
>
> It'll also answer the question of "is it worth having a slab allocator in 2008".
> My gut feeling says yes, but only for very small allocations. A lot of work has
> gone into memory allocators since the original slab allocator design in 1999/2000
> and I think trying to build a local allocator that scales past lots of CPUs is
> probably a waste of effort. I'll wait and see. I just wish the current malloc's
> let me take a "hint" for the allocation size and provide it again on free() to
> bypass whatever pointer -> memory region mapping lookup it has to do.

And to add - I'm already doing three malloc()s per proxied request:

* one for the proxy object - which handles both the client/server FDs for now;
* two fixed-size RX buffers (the TX side just writes data from the RX rather than
  copying it into an outgoing buffer. That'd be plain silly!)

So I'm already giving the memory allocator in FreeBSD-7 a good thrashing, especially
when running >1 thread. Heck, I even triggered locking contention in the VM with
NTHREADS > 1; I'll have to look at that when I get a quad core machine. :)

It'll be much more interesting to see how the allocator holds up when I'm doing
HTTP proxying rather than just naive TCP proxying. Just note that at my current
workload I'm seeing < 10% user and > 90% system time usage - I can't even get
adequate user-space profiling with such a small amount of time (and high syscall
volume, which I think is throwing off freebsd's performance measurement code.)

Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -