Re: squid adding gzip support

On 14 Aug 2003, Gonzalo Arana wrote:

> 1) When to compress:
> a) Client must have sent Accept-Encoding: and gzip has to be listed
> in supported encodings ('compress' could be implemented with not
> too much effort).
> b) Server response should not be Transfer-Encoding: chunked. If it
> is, it should be de-chunked, right?

You do not need to care about transfer-encoding. Squid is HTTP/1.0 and
transfer-encoding is only allowed in HTTP/1.1, further transfer-encoding
is hop-by-hop.

> c) Content-type of answer is text/html or text/plain (this could be
> moved to gzip_access, I think).
> d) Answer from server is not already compressed :-).
> d) Server response does not contain Content-Range header (in case of
> a miss). I simply won't compress partial data. This is due to
> the fact that Ranges header usually is used on 'resume download'
> operation, and I want to speed up interactive navigation.

And this is where you need to be extremely careful with ETag, to make sure
uncompressed and compressed content is never mixed as the same entity
(ETag value).

> e) gzip_access (new directive) is ok
> I will add this in order to enable compression only on dial-up
> clients (slow clients).
> Is this ok?

Yes.

> 2) Where to compress:
> I will compress on comm_write.

Not very good place for Content-Encoding. You need to do the compression
in client side, before calling comm_write.

The compression modified not only the body of the response, but also the
headers.

For Squid-2.5 it should be possible to do someting like what I am doing
for ICAP, pipe the transformation via a StoreEntry.

> 3) Which squid to patch:
> Unless squid 3 is going to be STABLE next week, I have to work on
> squid 2.5, since I have a deadline of 1 month to accomplish this :-(

A bit short time..

> Of course, Once this is working, I will start working on a patch
> for squid3.

Unfortunately the internals of Squid-2.5 and Squid-3 is quite different.
Squid-3 is supposed to be a lot easier to accomplish this kind of thing.

> 4) Answer modification:
> a) Should I alter Vary header (by adding Accept-Encoding)?

Yes.

> b) ETag: May I build it from md5sum of compressed content?

Or simply by extending the existing ETag with some more data of your own,
making it different from the original. Doing md5sum has the drawback that
you need access to the whole content to calculate the sum which is not
really feasible for large objects..

Regards
Henrik
Received on Thu Aug 14 2003 - 15:09:13 MDT