Re: [squid-users] TCP_REFRESH_UNMODIFIED instead of TCP_IMS_HIT? "Revalidation failed"

On 30/11/2011 9:49 p.m., David Wojak wrote:
> On 11/30/2011 01:32 AM, Amos Jeffries wrote:
>> On Tue, 29 Nov 2011 15:20:04 +0100, David Wojak wrote:
>>> Hi all!
>>>
>>> First, I am pretty new to proxies, so please bare with me if I'm
>>> doing something completely wrong. What I want to do:
>>>
>>> I have a VPN tunnel between two sides and a server serving java
>>> webstart jars on one side and I want to cache these on the other side.
>>> Because the jars should always be the same as on the original server,
>>> I want them to be stale all the time, so that squid checks the last
>>> modified date and returns the jar from cache only if the dates are
>>> equal. So far so good.
>>
>> No, this is not good use of HTTP. try sending the must-revalidate
>> cache control option instead of old date. Possibly also a max-stale=
>> value to determine how stale the reply is allowed to be on failure
>> cases.
> Thanks for the reply, Amos! The thing is - a Java Webstartclient will
> fetch the JARs. So the header is given, actually with "no-cache" from
> Webstartclient. That's why I try using reload-into-ims and
> refresh-ims. The "no-cache" header from Webstartclient should be
> transformed into a If-Modified-Since - and refrehs-ims should cause
> squid to contact the origin server. In short: a no-cache should be
> transformed into a must-revalidate.

We are talking at cross-wires here about two different things. The
"reload-into-ims" is fine. Although it assumes there is no alterations
to the client software. There is no guarantee the assumed headers will
arrive. To require correct operation you should send the
"must-revalidate" from the server.

>>
>>>
>>> What I have done:
>>>
>>> I used the standard squid (3.1.16) config on my gentoo system and put
>>> a refresh_pattern instead of the default refresh_patterns (I only want
>>> to cache jars):
>>>
>>> refresh_pattern -i \.(jar)\??(.*?)$ 0 0% 99999 reload-into-ims
>>> refresh-ims
>>>
>>> and set the following options:
>>>
>>> coredump_dir /squid_cache
>>> cache_replacement_policy heap LFUDA
>>> cache_dir ufs /squid_cache 2000 16 256
>>>
>>> maximum_object_size 256 MB
>>>
>>> Testing with a browser: The first time I request a jar I get X-Cache:
>>> MISS and X-Cache-Lookup: MISS which is perfectly fine but after that
>>> it gets interesting:
>>>
>>> HTTP/1.0 200 OK
>>> Date: Tue, 29 Nov 2011 13:37:59 GMT
>>> Content-Type: application/java-archive
>>> Content-Length: 66245
>>> Last-Modified: Tue, 29 Nov 2011 12:21:04 GMT
>>> Server: Jetty(6.1.16)
>>> Warning: 110 squid/3.1.16 "Response is stale", 111 squid/3.1.16
>>> "Revalidation failed"
>>> X-Cache: HIT from proxytest
>>> X-Cache-Lookup: HIT from proxytest:3128
>>> Via: 1.0 proxytest (squid/3.1.16)
>>> Connection: keep-alive
>>>
>>> Why do I get a "Revalidation failed"? I took a look at the headers
>>> between proxy and original server -> the original server sends a HTTP
>>> 1.1 304. So the revalidation should succeed, right? If I take a look
>>> at the squid log, it says:
>>>
>>> 1322573879.858 8 192.168.100.215 TCP_REFRESH_UNMODIFIED/200
>>> 66658 GET
>>> http://bla.bla.bla:8080/afdc3604/lib/commons-logging-1.1.1.jar -
>>> DIRECT/192.168.100.199 application/java-archive
>>>
>>> So I get the jar file not from cache, but DIRECT from the original
>>> server. Why? The file is unmodified. Shouldn't I get a TCP_IMS_HIT?
>>>
>>> If I change the refresh_pattern so that the file is not stale at any
>>> time I get it from cache just fine, so cache is working basically. But
>>> the revalidation just doesn't work and I have no idea why?
>>>
>>> Any hints?
>>
>> We had a bug a while back about the Warning being added in the wrong
>> place. I'm not recalling clearly if that got fixed or not before 3.1.6.
>>
>> Either way, if your 304 is emitted indicating an already stale object
>> the warnings are both correct. The object is stale of course. But
>> also the revalidation is required to produce a new fresh object with
>> 2xx status, an indication that object is currently fresh with 304
>> status, or a 4xx error. Returning a 3xx with invalid freshness
>> information is a failure. Your Squid is constantly running error
>> recovery operations on these 304, thus the Warning and need to use
>> refresh_pattern '99999' to "fix" the cache timing errors.
>>
>> Amos
> With "Returning a >3xx< with invalid freshness information is a
> failure" you also mean 304?

Yes

> What would an invalid freshness information be? The Last-Modified date
> actually is the same, but I'm not sure if there shouldn't be more info
> in the 304 response from jetty:

Ah sorry I read
"I want them to be stale all the time, so that squid checks the last
modified date and returns the jar from cache only if the dates are
equal"

.. and jumped to the conclusion you were sending a future Last-Modified
timestamp to make Squid think it was stale already.

>
> Client to Server (via proxy):
>
> GET http://bla.bla.bla:8080/afdc3604/lib/commons-logging-1.1.1.jar
> HTTP/1.1
> Host: tlptest.m2n.at:8080
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20100101
> Firefox/8.0
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-us,en;q=0.5
> Accept-Encoding: gzip, deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Proxy-Connection: keep-alive
> Cache-Control: no-cache
>
>
> Proxy to origin server:
>
> GET /afdc3604/lib/commons-logging-1.1.1.jar HTTP/1.1
> If-Modified-Since: Tue, 29 Nov 2011 12:21:04 GMT
> Host: bla.bla.bla:8080
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20100101
> Firefox/8.0
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-us,en;q=0.5
> Accept-Encoding: gzip, deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Via: 1.1 proxytest (squid/3.1.16)
> Cache-Control: max-age=5999940
> Connection: keep-alive
>
>
> Origin Server to Proxy:
>
> HTTP/1.1 304 Not Modified
> Server: Jetty(6.1.16)

There is no Date header here. Which may be a problem as it makes the 304
invalid, but Squid assumes "now" and sends that to the client...

>
>
> Proxy to client:
>
> HTTP/1.0 200 OK
> Date: Wed, 30 Nov 2011 08:27:16 GMT
> Content-Type: application/java-archive
> Content-Length: 66245
> Last-Modified: Tue, 29 Nov 2011 12:21:04 GMT
> Server: Jetty(6.1.16)
> Warning: 110 squid/3.1.16 "Response is stale", 111 squid/3.1.16
> "Revalidation failed"
> X-Cache: HIT from proxytest
> X-Cache-Lookup: HIT from proxytest:3128
> Via: 1.0 proxytest (squid/3.1.16)
> Connection: keep-alive
>

I've been checking up. It appears the Warning bug is not fixed yet. It
is wrong and can be ignored.

If you can, fix that Date up though.

Amos
Received on Wed Nov 30 2011 - 12:24:08 MST