Re: [squid-users] How to make squid ignore max-age?

On Tue, Apr 19, 2011 at 8:03 PM, Yang Zhang <yanghatespam_at_gmail.com> wrote:
> On Mon, Apr 18, 2011 at 9:18 PM, Yang Zhang <yanghatespam_at_gmail.com> wrote:
>> On Mon, Apr 18, 2011 at 9:08 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>>> On Mon, 18 Apr 2011 20:07:33 -0700, Yang Zhang wrote:
>>>>
>>>> We're using squid (3.0.STABLE19-1ubuntu0.1) as an application cache
>>>> and we're trying to cache everything, for a long time, but max-age=0
>>>> is throwing off squid.
>>>
>>> max-age=0 is called "reload" in HTTP terminology.
>>>
>>>>
>>>> I have this refresh_pattern in squid.conf:
>>>>
>>>> $ grep ^refresh_pattern /etc/squid3/squid.conf
>>>> refresh_pattern ^ftp:           1440    20%     10080
>>>> refresh_pattern ^gopher:        1440    0%      1440
>>>> refresh_pattern . 525600 100% 525600 override-expire ignore-private
>>>> ignore-no-cache ignore-no-store
>>>>
>>>
>>> To quote Frankenstein
>>>  "Why? whyyyy?"
>>>
>>>
>>>> I have the following response prepared to be sent from a dummy web server:
>>>>
>>>> $ cat response
>>>> HTTP/1.1 200 OK
>>>> Server: Apache-Coyote/1.1
>>>> Expires: 0
>>>> Pragma: no-cache
>>>> Cache-Control: no-cache, must-revalidate, max-age=0
>>>> Content-Type: text/html;charset=UTF-8
>>>> Date: Mon, 18 Apr 2011 10:06:26 GMT
>>>> Connection: close
>>>
>>> Those headers say two things:
>>>  * that cached content MUST NOT be supplied to the client. EVER.
>>>  * that any existing cached content also MUST be checked for updates on the
>>> next request.
>>>
>>> Ignoring this requirement *from the server*. Again... why?
>>>
>>>
>>> I can only imagine that you have fallen for the myth that "application
>>> caches" must always provide HITs. This is false and following it will break
>>> things for you in potentially nasty ways.
>>>
>>> A cachable application will not send max-age=0 or "Expires: 0", both of
>>> which mean the object was obsolete the second it was produced.
>>>
>>> Squid operating as a reverse-proxy will cache the replies and keep updating
>>> fresh copies as long as the server is available to present new ones. If the
>>> server dies an 5xx start coming back, the stale versions will start to be
>>> used. How long for and whether "must-revalidate" produces 4xx/5xx to the
>>> client depends on particular squid versions.
>>>  I recommend you grab the newest release you can for the best HTTP/1.1
>>> caching behaviour. there are still things we are fixing but work is ongoing
>>> so the newer the release the better things work. I maintain a PPA for Ubuntu
>>> (https://launchpad.net/~yadi/+archive/ppa) with newer versions.
>>>
>>> <snip>
>>>>
>>>> This response prevents anything from being cached; all requests for
>>>> http://localhost:8765 result in TCP_MISS.  I believe it's because of
>>>> the max-age=0 - once I remove that, this request is cached fine.  Yet
>>>> I had set override-expire in the refresh_pattern.
>>>> (Note that I don't think it's a must-revalidate/ignore-revalidate (new
>>>> in v3.1) problem - I can leave that in fine.)
>>>
>>> However "We're using squid (3.0.STABLE19-1ubuntu0.1)" ... not a v3.1
>>> release.
>>>
>>> "ignore-revalidate" blocks the *client* from requesting a revalidation
>>> (max-age=0). But still obeys the server controls.
>>>
>>>>
>>>> Any hints on how to make squid ignore the max-age=0?  I've pored over
>>>> the refresh_pattern documentation a bunch of times but still can't see
>>>> what's wrong.  Thanks in advance for any help.
>>>
>>> If you must;
>>>  * "ignore-reload" to ignore the max-age=0 from either client or server
>>> IIRC.
>>
>> Just tried this out - it doesn't work.  The comments in the squid.conf
>> also say that it's for max-age=0 from the client (no mention of
>> server).
>>
>>>  * "reload-into-ims" to convert a client max-age=0 ("force a new copy or
>>> die") into a soft IMS request ("new content or cached copy").
>>
>> The client is not issuing any max-age=0.  Tried adding this in anyway
>> - didn't work.
>>
>>>
>>> 'must-revalidate' will still have affect until you are using a 3.1 release
>>> with the "ignore-must-revalidate". The result of revalidate should only be
>>> 304 (TCP_IMS_*) or 200 (TCP_REFRESH_UPDATE), not a "TCP_MISS" since the
>>> content being validated *is* in the cache.
>>
>> My point earlier is that the must-revalidate from the server doesn't
>> seem be what's preventing the response from getting cached....
>
> Also, FWIW, these -max-age=0 responses are making it *into* the cache
> (verified by grepping /var/spool/squid3), for whatever reason (not
> sure why Squid is deciding to keep them if only to later not use
> them).
>
> --
> Yang Zhang
> http://yz.mit.edu/
>

Update: using Squid 3.1.12.1 and ignore-must-revalidate fixed this.
Weird since the original problem only arose from the presence of
max-age=0; the revalidate itself wasn't preventing caching.

-- 
Yang Zhang
http://yz.mit.edu/