Re: squid-2 dataflow and copying changes

On Sat, Apr 05, 2008, Henrik Nordstrom wrote:

> It would also be good to look into a temporary fix for 2.7 as the above
> is probably not 2.7 material.

Well, its sort of related - the one or two places where I'm currently passing
in a buffer via storeClientCopy() can be turned into a storeClientRef() or
a storeClientCopy(sc, NULL (buffer)..) ; so the store client is signaled to
return a large buffer as required.

Then I can implement the fix as "allocate buffer for reading in the first
chunk of data and grow the buffer (or over-allocate it in the first place,
letting the VM do the talking), and then either pass the buffer back to the
caller OR copy the first 4k out."

> > This'll involve bringing over various parts of my Squid-2 work in s26_adri
> > into the main tree. I'll initially bring over some of the code reshuffling
> > and the string related changes. Once those are stable and functioning well,
> > I'll look at bringing over some other parts of the work - referenced strings,
> > writev() for request/reply information.
>
> Interesting. Do you have any performance metrics comparing this with
> 2.7?

Only what I pulled from a few months ago - the improvements aren't all that great
until I start avoiding string buffer copying and reduplication. CPU-wise, I was
saving roughly 15% CPU between 2.7 and s26_adri; quite a bit less than expected.

But then, the whole point of this particular exercise is to lay the foundation for
further code and dataflow and improvements to improve performance and scalability.
Its hard to fix anything right now (in either tree!) because the codebase needs
massive tidying up and noone here knows exactly what to target in the medium to long
term, and as I stated all along, this is precisely what I was trying to attack
in the first place.

Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -