Re: "manipulating" outbound data

On Sat, 2003-08-16 at 02:41, Brian Akins wrote:
> Just an idea:
>
> Can someone with greater squid knowledge than me tell me how hard they
> think it would be to add "hooks" to manipulate outgoing data? I have a
> project where this may come in handy.

It depends :}.

I'm going to assume you meant data being sent to the client, not data
being send to the webserver.

For squid-2.5, it's relatively easy - with a couple of gotchas. You need
to do what Henrik has done with the iCAP branch (in fact, building an
iCAP server may be the best way) and feed the data through an extra
store entry - doing your transformation between the first store entries
client interface and the server interface of the second. This approach
should allow range requests to work reasonably well, provided that you
follow all the relevant ETAG and content-length rules. (I.e. Etags are
unique per entity content at one URL ...). That said, the 2.5 range
logic is uhmm, byzantine, and likely to throw spanners in the works
regardless :}.

The second gotcha is what you need to do - in-squid logic has to be
none-blocking, and stream based - many of the requests here about
content transformation in the last year or two have been 'how do I get
the whole object to do pattern matching :}'.

For squid-3.0, it's a whole lot easier. The hooks are already there for
general content transformation of data being sent to the client. The
clientStreams interface, written to allow the ESI code to hook in
cleanly, is a fairly minimalist streams interface, and all you need to
do is add in your stream interface to the stream, and then read data,
and supply data as requested by the stream clients.

It's about, oh, 100 lines of code to do that.

That interface doesn't allow handshake-related changes though. By which
I mean:
* TE
* Range request fiddling

It doesn't allow those, because TE comes after range logic - which all
resides in the ClientSocketContext. In the future, I'd like to tease
range requests into a client stream module - but for now, thats in the
'future' bin.

The internal path that TE needs though, is aided by the client streams -
just not the actual encoding.

Rob

-- 
GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.