Re: [squid-users] Problem with HTTP 1.1 replies

On 11/02/11 15:19, Packet Racer wrote:
> Hopefully this is the right list for this questions:
>
> Currently running squid 2.6.STABLE21-3 (the RedHat distributed one),
> and having problems with a specific site that makes of use of HTTP
> 1.1. The issue can be boiled down to this:
>
> The site loads a page and asks the browser NOT to cache it. Then it
> asks the browser to reload it a 2nd and 3rd time. The 2nd and 3rd
> load depend on the browser pulling fresh data from the server. When
> it works, it looks like this:
>
> *** Client request #1:
> GET http://www.[...snipped...].com/boost-gzip-cookie-test.html HTTP/1.0
> Accept: */*
> Referer: http://www.[...snipped...].com/
> Accept-Language: en-us
> UA-CPU: x86
> Connection: Keep-Alive
> User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; [...snipped...])
> Cookie: has_js=1; cmTPSet=Y
> Host: www.[...snipped...].com
>
> *** Server reply #1 (headers only shown):
> HTTP/1.1 200 OK
> Date: Tue, 08 Feb 2011 03:48:57 GMT
> Server: Apache/2.2.3 (Red Hat)
> Last-Modified: Thu, 03 Feb 2011 21:32:11 GMT
> ETag: "17c801c-15a-49b677f912cc0"
> Accept-Ranges: bytes
> Content-Length: 460
> Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
> Expires: Sun, 19 Nov 1978 05:00:00 GMT
> X-Header: Boost Citrus 1.8
> Connection: close
> Content-Type: text/html; charset=utf-8
>
> *** Client request #2:
> GET http://www.[...snipped...].com/boost-gzip-cookie-test.html HTTP/1.0
> Accept: image/gif, image/x-xbitmap, [...snipped...], application/xaml+xml, */*
> Accept-Language: en-us
> UA-CPU: x86
> Connection: Keep-Alive
> User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; [...snipped...])
> Host: www.[...snipped...].com
> Cookie: has_js=1; cmTPSet=Y
>
> *** Server reply #2:
> HTTP/1.1 200 OK
> Date: Tue, 08 Feb 2011 03:48:57 GMT
> Server: Apache/2.2.3 (Red Hat)
> Last-Modified: Thu, 03 Feb 2011 21:32:11 GMT
> ETag: "17c801c-15a-49b677f912cc0"
> Accept-Ranges: bytes
> Content-Length: 460
> Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
> Expires: Sun, 19 Nov 1978 05:00:00 GMT
> X-Header: Boost Citrus 1.8
> Connection: close
> Content-Type: text/html; charset=utf-8
>
> Request and Reply #3 are exactly the same as #2. As you can see, the
> site depends on the browser honoring the Cache-Control header, which
> is an HTTP 1.1 construct.
>
> When traffic goes through Squid, however, what you get is this:
>
> *** Client request #1:
> GET http://www.[...snipped...].com/boost-gzip-cookie-test.html HTTP/1.0
> Accept: */*
> Referer: http://www.[...snipped...].com/
> Accept-Language: en-us
> UA-CPU: x86
> Proxy-Connection: Keep-Alive
> User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; [...snipped...])
> Cookie: has_js=1; cmTPSet=Y
> Proxy-Authorization: [...snipped...]
> Host: www.[...snipped...].com
>
> *** Reply #1:
> HTTP/1.0 200 OK
> Date: Tue, 08 Feb 2011 03:08:16 GMT
> Server: Apache/2.2.3 (Red Hat)
> Last-Modified: Thu, 03 Feb 2011 21:32:11 GMT
> ETag: "17c801c-15a-49b677f912cc0"
> Accept-Ranges: bytes
> Content-Length: 346
> Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
> Expires: Sun, 19 Nov 1978 05:00:00 GMT
> X-Header: Boost Citrus 1.8
> Content-Type: text/html; charset=utf-8
> Content-Encoding: gzip
> X-Cache: MISS from [...snipped...]
> X-Cache-Lookup: MISS from [...snipped...]:3128
> Via: 1.0 [...sniped...]:3128 (squid/2.6.STABLE21)
> Proxy-Connection: keep-alive
>
> *** Client request #2:
> GET http://www.[...snipped...].com/boost-gzip-cookie-test.html HTTP/1.0
> Accept: image/gif, image/x-xbitmap, image/jpeg, [...snipped...],
> application/xaml+xml, */*
> Accept-Language: en-us
> UA-CPU: x86
> Proxy-Connection: Keep-Alive
> If-Modified-Since: Thu, 03 Feb 2011 21:32:11 GMT; length=346
> User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; [...snipped...])
> Cookie: has_js=1; cmTPSet=Y
> Proxy-Authorization: [...snipped...]
> Host: www.[...snipped...].com
>
> *** Reply #2:
> HTTP/1.0 304 Not Modified
> Date: Tue, 08 Feb 2011 03:08:16 GMT
> Server: Apache/2.2.3 (Red Hat)
> ETag: "17c801c-15a-49b677f912cc0"
> Expires: Sun, 19 Nov 1978 05:00:00 GMT
> Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
> X-Cache: MISS from [...snipped...]
> X-Cache-Lookup: MISS from [...snipped...]:3128
> Via: 1.0 [...snipped...]:3128 (squid/2.6.STABLE21)
> Proxy-Connection: keep-alive
>
> Well, now the browser is sending "If-Modified-Since:" and Squid says
> "Not Modified." That kind of breaks the subsequent pages that load

No. The browser is sending "If-Modified-Since", Squid is passing this to
the server. The server sends back "Not Modified".

Squid is obeying the "no-store" (do not storing the response) and the
"no-cache" (do not generate or alter the reply based on local proxy
cache storage).

> from that site. It seems to me that IE7 and IE8 (the two I tested
> with) do not honor the Cache-Control header if they see an HTTP/1.0
> response.

see above. The website test is relying on a specific inefficient and
somewhat broken mode of HTTP being used. Its own server is configured to
use HTTP properly and generate more efficient replies.

>
> So, the question is: What are the possible solutions that I can implement?

Complain to the website author. Demonstrate with any one of
If-Modified-Since, If-Unmodified-Since, If-Match or If-None-Match
request. The proxy is not relevant, just the use of HTTP/1.1 conditional
headers.

>
> Things like changing the browser or asking the site to stop doing the
> boost-gzip-cookie-test are not viable solutions. I'm thinking about

The boost-gzip-cookie-test is fatally flawed.

They can avoid the bug somewhat by always sending out the full reply
body, even if an efficient conditional reply is requested.

There is no guarantee that this trick will always work though. Recent
proxies are working towards better traffic efficiency so may one day
crop the duplicated body away.

> an upgrade to 2.7 or 3.1, but that will take some time to plan and
> test. Plus, I'm not sure that an upgrade will fix the problem,
> anyway. Anyone know?
>
> Ideally I'm hoping that there's some way to tell Squid not to modify
> the server responses when the request asks fora
> boost-gzip-cookie-test.html. Is there? Or maybe to insert a "Pragma:
> No-cache" into the reply?

Squid is not modifying these responses.

You can make it strip away the conditional headers on requests destined
to this site using the header_access directive.

That will leave the site broken for all other web visitors who don't use
your proxy though. Reporting the problem to them is really the best
thing to do.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.11
   Beta testers wanted for 3.2.0.4