Re: [squid-users] Diagnosing Objects That Are Not Cached (squid/3.0.STABLE8)

Chris Robertson wrote:
> Norbert Hoeller wrote:
>> Although my squid server is only supporting three users, I am
>> pleasantly surprised at the percentage of traffic handled by the
>> cache. However, I think I can do better. Scanning the logs, I
>> noticed a number of cases where the same files were resulting in a
>> TCP_MISS/200 condition. A lot are due to '?'-strings in the URL, even
>> though the content is not always dynamic (I am researching options).
>
> Making sure you don't have...
>
> acl QUERY urlpath_regex cgi-bin \?
> cache deny QUERY
>
> ...in your squid.conf, in conjunction with the refresh_patterns you DO
> have are about as far as you can go without starting to break standards.
>
>> However, I see a fair number of cases where I would expect the
>> object to be cached. Is there a process for diagnosing why squid
>> thinks the object is not cache-able?
>>
>
> Up the debug_options for 22 and maybe 65. See
> http://wiki.squid-cache.org/KnowledgeBase/DebugSections for a list.
>
>> For example, from the access.log and store.log files:
>>
>> 1265776256.982 3934 10.1.2.123 TCP_MISS/200 861 GET
>> http://www.facebook.com/images/loaders/indicator_blue_small.gif -
>> DIRECT/66.220.146.18 image/gif
>> 1265776281.912 370 10.1.2.123 TCP_MISS/200 861 GET
>> http://www.facebook.com/images/loaders/indicator_blue_small.gif -
>> DIRECT/66.220.146.18 image/gif
>> 1265776294.352 407 10.1.2.123 TCP_MISS/200 861 GET
>> http://www.facebook.com/images/loaders/indicator_blue_small.gif -
>> DIRECT/66.220.146.18 image/gif
>>
>> 1265776256.982 RELEASE -1 FFFFFFFF 54F74EF7200473A70B1E94F452E355C0
>> 200 -1 -1 1265776256 image/gif 522/522 GET
>> http://www.facebook.com/images/loaders/indicator_blue_small.gif
>> 1265776281.912 RELEASE -1 FFFFFFFF D50A66A7F3C5E849631BF132637A99A3
>> 200 -1 -1 1265776281 image/gif 522/522 GET
>> http://www.facebook.com/images/loaders/indicator_blue_small.gif
>> 1265776294.352 RELEASE -1 FFFFFFFF 25AA53B40429083AF5B68BAF4A663EF0
>> 200 -1 -1 1265776294 image/gif 522/522 GET
>> http://www.facebook.com/images/loaders/indicator_blue_small.gif
>>
>> redbot.org returns:
>>
>> HTTP/1.1 200 OK
>> Accept-Ranges: bytes
>> Cache-Control: max-age=2592000
>> Content-Type: image/gif
>> Expires: Sat, 13 Mar 10 22:37:08 GMT
>> X-Cnection: close
>> Date: Thu, 11 Feb 2010 22:37:08 GMT
>> Content-Length: 522
>>
>> General: The Expires header's value isn't a valid date.

That above is probably the killer. As we get closer to HTTP/1.1
compliance we get more things discarded for non-compliance.

Invalid date maps to "-1" and non-cacheable.

Contact facebook and let them know. Its likely costing them a great deal
of money.

<snip>
>> I am using a fairly vanilla squid.conf file, with the exception of
>> some ad-blocking:
>>
>> #Suggested default:
>> refresh_pattern ^ftp: 1440 20% 10080
>> refresh_pattern ^gopher: 1440 0% 1440
>> refresh_pattern (cgi-bin|\?) 0 0% 0

The pattern should be -i (/cgi-bin/|\?) for best effect.

Any other refresh_patterns in use? Those URL look suspiciously like
ones which would be caught by trying to handle mime in refresh_pattern.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE23
   Current Beta Squid 3.1.0.16