Re: [squid-users] ecap adapter munging cached body

> Vary in Squid is currently treated as an exact-match text key. So when asked
> for a "gzip,deflate" variant Squid does not have enough smarts to serve the
> "deflate" variant. So it MISSes and gets a fresh one, which may or may not
> be gzipped, but is served gzipped to the client anyway.

Right on, that makes sense. I was really trying to test gzip module
vs no zipping, for clients that don't support any zipping.

> When passing the second request through squid twice in a row does the reply
> change from a MISS to a HIT? or stay a MISS?

The second request stays a MISS.

> Squid caches only one of N variants so the expected behviour is that each
> new variant is a MISS but becomes a HIT on repeated duplicate requests until
> a new variant pushes it out of cache.

Ah, well that would sort of explain it, except I don't get a
subsequent cache HIT when requesting a zipped version.

So, to test all this out, I have the webserver returning either:
a) a full HTML page (57580 bytes) when no Accept-Encoding header is present
b) some alternate content (the Accept-Encoding header echoed back 5000
times) when Accept-Encoding is present, such that the response is a
different size and dependent on the Accept-Encoding header.

Then, I issue the same request headers, just modifying the
Accept-Encoding header value (or excluding that header altogether).

I'm using the values of "asdf" for a bogus Accept-Encoding value that
shouldn't trigger gzipping, and "gzip" for when I actually want to
invoke the module. To be clear, the webserver isn't zipping at all.

The request headers before optionally adding an Accept-Encoding header are:

GET /squid-test HTTP/1.1
User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7
OpenSSL/0.9.8l zlib/1.2.3
Accept: */*
Host: www.example.com

Here are the response headers, all requests issued serially in the
order listed here:

1. Empty Accept-Encoding header - two requests in a row, expect MISS
then HIT, full content, not zipped.

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:19 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:19 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Length: 57580
X-Cache: MISS from www.example.com
X-Cache-Lookup: MISS from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: keep-alive

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:19 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:19 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Length: 57580
Age: 2
X-Cache: HIT from www.example.com
X-Cache-Lookup: HIT from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: keep-alive

2. Accept-Encoding: asdf header - two requests in a row, expect MISS
then HIT, alternate content, not zipped.

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:35 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:35 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Length: 20006
X-Cache: MISS from www.example.com
X-Cache-Lookup: MISS from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: keep-alive

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:35 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:35 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Length: 20006
Age: 2
X-Cache: HIT from www.example.com
X-Cache-Lookup: HIT from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: keep-alive

3. Try no Accept-Encoding again - get a HIT, same full content from (1).

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:19 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:19 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Length: 57580
Age: 22
X-Cache: HIT from www.example.com
X-Cache-Lookup: HIT from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: keep-alive

4. Now try Accept-Encoding: gzip. Two requests in a row, expect MISS
then HIT, get MISSes every time.
(I included the squid access log rows here to see the small zipped
content length - 660 bytes.)

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:49 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:49 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
X-Cache: MISS from www.example.com
X-Cache-Lookup: MISS from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: close

[23/Jan/2011:15:50:49 -0500] "GET http://www.example.com/squid-test
HTTP/1.1" 200 660 "345 ms" "-" "curl/7.19.7
(universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3"
TCP_MISS:ROUNDROBIN_PARENT

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:50 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:50 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
X-Cache: MISS from www.example.com
X-Cache-Lookup: MISS from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: close

[23/Jan/2011:15:50:51 -0500] "GET http://www.example.com/squid-test
HTTP/1.1" 200 660 "345 ms" "-" "curl/7.19.7
(universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3"
TCP_MISS:ROUNDROBIN_PARENT

5. Try Accept-Encoding: asdf again, get a HIT with the cached
alternate content, not zipped.

HTTP/1.0 200 OK
Cache-Control: max-age=600
Expires: Sun, 23 Jan 2011 21:00:35 GMT
Vary: Accept-Encoding
Mime-Version: 1.0
Date: Sun, 23 Jan 2011 20:50:35 GMT
Server: AOLserver/4.5.1
Content-Type: text/html; charset=utf-8
Content-Length: 20006
Age: 20
X-Cache: HIT from www.example.com
X-Cache-Lookup: HIT from www.example.com:80
Via: 1.0 www.example.com (squid/3.1.10)
Connection: keep-alive

So, it looks to me like both the variants for the different
Accept-Encoding values are being cached (not just one variant), except
when the gzip module is invoked.

I can change the behavior of the webserver to not include Vary:
Accept-Encoding for content meant to be cached by squid, but that
results in responses of the cached (unzipped) version even for clients
who accept zipped versions, once the cache is populated by a client
not requesting a zipped version, and that defeats the point of the
gzip module for me because I want to gzip cached content for clients
that support it.

Regards,
-Jon
Received on Sun Jan 23 2011 - 22:14:58 MST