Re: [squid-users] Squid 3 HTTP accelerator not caching content

Tom Williams wrote:
> Amos Jeffries wrote:
>>> So, I setup my first Squid 3.0STABLE9 proxy in HTTP accelerator mode
>>> over the weekend. Squid 3 is running on the same machine as the web
>>> server and here are my HTTP acceleration related config options:
>>>
>>> http_port 80 accel vhost
>>> cache_peer 192.168.1.19 parent 8085 0 no-query originserver login=PASS
>>>
>>>
>>> Here are the cache related options:
>>>
>>> cache_mem 64 MB
>>> maximum_object_size_in_memory 50 KB
>>> cache_replacement_policy heap LFUDA
>>> cache_dir aufs /mnt/drive3/squid-cache 500 32 256
>>>
>>> As described in this mailing list thread:
>>>
>>> http://www2.gr.squid-cache.org/mail-archive/squid-users/199906/0756.html
>>>
>>> all of the entries in my store.log have RELEASE as the action:
>>>
>>> 1223864638.986 RELEASE -1 FFFFFFFF A1FE29E96A44936155BB873BDC882B12 200
>>> 1223864638 -1 375007920 text/html 2197/2197 GET
>>> http://aaa.bbb.ccc.ddd/locations/
>>>
>>> Here is a snipet from the cache.log file:
>>>
>>> 2008/10/12 21:23:36| Done reading /mnt/drive3/squid-cache swaplog (0
>>> entries)
>>> 2008/10/12 21:23:36| Finished rebuilding storage from disk.
>>> 2008/10/12 21:23:36| 0 Entries scanned
>>> 2008/10/12 21:23:36| 0 Invalid entries.
>>> 2008/10/12 21:23:36| 0 With invalid flags.
>>> 2008/10/12 21:23:36| 0 Objects loaded.
>>> 2008/10/12 21:23:36| 0 Objects expired.
>>> 2008/10/12 21:23:36| 0 Objects cancelled.
>>> 2008/10/12 21:23:36| 0 Duplicate URLs purged.
>>> 2008/10/12 21:23:36| 0 Swapfile clashes avoided.
>>> 2008/10/12 21:23:36| Took 0.01 seconds ( 0.00 objects/sec).
>>> 2008/10/12 21:23:36| Beginning Validation Procedure
>>> 2008/10/12 21:23:36| Completed Validation Procedure
>>> 2008/10/12 21:23:36| Validated 25 Entries
>>> 2008/10/12 21:23:36| store_swap_size = 0
>>> 2008/10/12 21:23:37| storeLateRelease: released 0 objects
>>> 2008/10/12 21:24:07| Preparing for shutdown after 2 requests
>>> 2008/10/12 21:24:07| Waiting 30 seconds for active connections to finish
>>> 2008/10/12 21:24:07| FD 14 Closing HTTP connection
>>> 2008/10/12 21:24:38| Shutting down...
>>> 2008/10/12 21:24:38| FD 15 Closing ICP connection
>>> 2008/10/12 21:24:38| aioSync: flushing pending I/O operations
>>> 2008/10/12 21:24:38| aioSync: done
>>> 2008/10/12 21:24:38| Closing unlinkd pipe on FD 12
>>> 2008/10/12 21:24:38| storeDirWriteCleanLogs: Starting...
>>> 2008/10/12 21:24:38| Finished. Wrote 0 entries.
>>> 2008/10/12 21:24:38| Took 0.00 seconds ( 0.00 entries/sec).
>>> CPU Usage: 0.041 seconds = 0.031 user + 0.010 sys
>>> Maximum Resident Size: 0 KB
>>> Page faults with physical i/o: 0
>>> Memory usage for squid via mallinfo():
>>> total space in arena: 3644 KB
>>> Ordinary blocks: 3511 KB 8 blks
>>> Small blocks: 0 KB 1 blks
>>> Holding blocks: 1784 KB 9 blks
>>> Free Small blocks: 0 KB
>>> Free Ordinary blocks: 132 KB
>>> Total in use: 5295 KB 145%
>>> Total free: 132 KB 4%
>>> 2008/10/12 21:24:38| aioSync: flushing pending I/O operations
>>> 2008/10/12 21:24:38| aioSync: done
>>> 2008/10/12 21:24:38| aioSync: flushing pending I/O operations
>>> 2008/10/12 21:24:38| aioSync: done
>>> 2008/10/12 21:24:38| Squid Cache (Version 3.0.STABLE9): Exiting
>>> normally.
>>>
>>> I'm running on RedHat EL 5. With Squid running, I can access the
>>> website just fine and pages load without problems or issues. It's just
>>> nothing is being cached.
>>>
>>> This is my first time configuring Squid as a HTTP accelerator so I
>>> probably missed something when I set it up. Any ideas on what might be
>>> wrong?
>>>
>>> Thanks in advance for your time and assistance! :)
>>>
>>>
>>
>> Q) Do you have any of the routing access controls (http_access,
>> never_direct, cache_peer_access, cache_peer_domain) which make squid pass
>> the accelerated requests back to the web server properly?
>>
>
> I have the default http_access options except I have http_access allow
> all at the end of them:

Ouch. You have a semi-open proxy.
If anyone identifies your public IP they can point a domain DNS at your
IP and have it accelerated. Or even configure port 80 as their proxy IP
and browse through it. A firewall or NAT layer cannot prevent this
happening.

You should at the very least be limiting requests to the domains you are
serving.

I prefer a config like the one listed:
http://wiki.squid-cache.org/SquidFaq/ReverseProxy#head-7fa129a6528d9a5c914f8dd5671668173e39e341

>
> #Recommended minimum configuration:
> #
> # Only allow cachemgr access from localhost
> http_access allow manager localhost
> http_access deny manager
> # Deny requests to unknown ports
> http_access deny !Safe_ports
> # Deny CONNECT to other than SSL ports
> http_access deny CONNECT !SSL_ports
> #
> # We strongly recommend the following be uncommented to protect innocent
> # web applications running on the proxy server who think the only
> # one who can access services on "localhost" is a local user
> http_access deny to_localhost
> #
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
>
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
>
> # And finally deny all other access to this proxy
> #http_access deny all
> http_access allow all
>
> I did this mainly to get things working and I plan on refining these
> options. I do not have never_direct, cache_peer_access,
> cache_peer_domain explicitly set in the config file.
>> DNS should be pointing the domain at Squid so its DIRECT access lookups
>> will normally loop back inwards and fail. The resulting error pages may
>> not be cachable.
>>
>
> I'll be sure to keep this in mind. For now, I'm using the server IP
> address to access it.
>> Q) what does your access.log say about the requests?
>>
>
> I'm using Apache 2.2.3 as the web server and its logs do show I'm
> accessing the pages without problems:
>
> 1.1.1.1 - tom [13/Oct/2008:10:15:39 -0500] "GET /topics/ HTTP/1.1" 200
> 2316 "http://aaa.bbb.ccc.ddd/locations/" "Mozilla/5.0 (X11; U; Linux
> x86_64; en-US; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy)
> Firefox/3.0.3"
>
> as an example. The pages load fine in my browser.
>
>> Q) are the test pages you are requesting cachable?
>> This tester should tell you what and for how long it caches
>> http://www.ircache.net/cgi-bin/cacheability.py
>>
> Good question. I'll use this site to make sure.
>
> Thanks!
>
> Peace...
>
> Tom

Amos

-- 
Please use Squid 2.7.STABLE4 or 3.0.STABLE9