RE: [squid-users] Cache manager analysis

Ok - thanks.
2.HEAD - has this been included in the CentOS repository yet? I believe CentOS only has 2.6
So, before I even look at the optimising sections, this gives me a squid.conf of the following (does this look ok?):

auth_param basic realm Proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
#acl all src 0.0.0.0/0.0.0.0
acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1
acl cacheadmin src 88.xxx.xxx.xxx 127.0.0.1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl Safe_ports port 1863         # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
#http_access allow manager localhost
#IP 127.0.0.1 added to cacheadmin acl above instead
http_access allow manager cacheadmin
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny manager
http_access allow ncsa_users
http_access deny maxuser
#http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port 88.xxx.xxx.xxx:80
hierarchy_stoplist cgi-bin ?
#cache_mem 100MB
#maybe increase further, check top
cache_mem 256MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 40000 16 256
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
#acl QUERY urlpath_regex cgi-bin \?
#cache deny QUERY
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?)  0 0% 0
refresh_pattern .               0       20%     4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
cache_mgr aaa_at_aaa.com
cachemgr_passwd aaa all
visible_hostname ProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid

----------------------------------------
> Date: Sat, 13 Feb 2010 18:03:00 +1300
> From: squid3_at_treenet.co.nz
> To: squid-users_at_squid-cache.org
> Subject: Re: [squid-users] Cache manager analysis
>
> J. Webster wrote:
>> What is the best place to start with in cache analysis?
>> Would it be cache size, memory object size, IO, etc.?
>> I'm looking to optimise the settings for my squid server.
>
> Step 0) migrate to the latest Squid 2.7 or 3.1 or if possible 2.HEAD
> (that one is only nominally beta, it's very stable in reality)
>
> 1) Start by defining 'optimize' ... are you going to prioritize...
> Faster service?
> More bandwidth saving?
> More client connections?
>
> 2a) For faster service, look at DNS delays, disk IO delays, maximizing
> cacheable objects (dynamic objects etc).
>
> 2b) For pure bandwidth savings start with a look at object cacheablity.
> Check dynamics are being cached, ranges are being fetched in full, etc
>
> 3) Then profile all the objects stored over a reasonably long period,
> looking at size. compare with the age of objects being discarded.
>
> 3a) tune the storage limits to prioritize the storage locations. giving
> priority to RAM, then COSS, then AUFS/diskd.
>
> 3b) set the storage limits as high as possible to maximize amount of
> data stored. anywhere.
>
> 4) take a good long look at your access controls and in particular the
> types speedy/fast/slow. You may get some speed benefits from fixing up
> the ordering a bit. regex are killers, remote lookups (helpers, or DNS)
> are second worst.
> (some performance hints below)
>
> 5) repeat from (2b) as often as possible. concentrate traffic which
> seems to logically be storeable but gets a TCP_MISS anyway.
>
> Objects served from cache lead to faster service ties for those objects,
> so the speed vs bandwidth are inter-related somewhat. But there is a
> tipping point somewhere where tuning one starts to impact the other.
>
>
>>
>> Server: about 220GB available for the cache, I'm only using 40000 MB at present as in the config below.
>> system D2812-A2
>> /0 bus D2812-A2
>> /0/0 memory 110KiB BIOS
>> /0/4 processor Intel(R) Core(TM)2 Duo CPU E7300 @ 2.66GHz
>> /0/4/5 memory 64KiB L1 cache
>> /0/4/6 memory 3MiB L2 cache
>> /0/4/0.1 processor Logical CPU
>> /0/4/0.2 processor Logical CPU
>> /0/7 memory 3MiB L3 cache
>> /0/2a memory 1GiB System Memory
>> /0/2a/0 memory 1GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
>> /0/2a/1 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
>> /0/2a/2 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
>> /0/2a/3 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
>> /0/1 processor
>> /0/1/0.1 processor Logical CPU
>> /0/1/0.2 processor Logical CPU
>>
>>
>> Current squid.conf:
>> ---------------------
>> auth_param basic realm Proxy server
>> auth_param basic credentialsttl 2 hours
>> auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
>> authenticate_cache_garbage_interval 1 hour
>> authenticate_ip_ttl 2 hours
>> acl all src 0.0.0.0/0.0.0.0
>
> all src all
>
>> acl manager proto cache_object
>> acl localhost src 127.0.0.1/255.255.255.255
>
> acl localhost src 127.0.0.1
>
>> acl cacheadmin src 88.xxx.xxx.xxx
>> acl to_localhost dst 127.0.0.0/8
>
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
>
>> acl SSL_ports port 443
>> acl Safe_ports port 80 # http
>> acl Safe_ports port 21 # ftp
>> acl Safe_ports port 443 # https
>> acl Safe_ports port 70 # gopher
>> acl Safe_ports port 210 # wais
>> acl Safe_ports port 1025-65535 # unregistered ports
>> acl Safe_ports port 280 # http-mgmt
>> acl Safe_ports port 488 # gss-http
>> acl Safe_ports port 591 # filemaker
>> acl Safe_ports port 777 # multiling http
>> acl Safe_ports port 1863 # MSN messenger
>> acl ncsa_users proxy_auth REQUIRED
>> acl maxuser max_user_ip -s 2
>> acl CONNECT method CONNECT
>> http_access allow manager localhost
>> http_access allow manager cacheadmin
>
> Hint: add the localhost IP to the cacheadmin ACL and drop one full set
> of "allow manager localhost" tests.
>
>> http_access deny manager
>> http_access allow ncsa_users
>
> Hint: drop the authentication down ...
>
>> http_access deny !Safe_ports
>> http_access deny CONNECT !SSL_ports
>> http_access deny to_localhost
>
> ... to here. All the attacks against your proxy for bad ports and
> sources will be dropped quickly by the security blanket settings. Load
> on your auth server will reduce and may speed up it's response time.
>
> Hint 2: if possible, define an ACL or the network ranges where you
> accept logins. Use it like so:
>
> http_access allow localnet ncsa_users
>
> ... once again that speeds up the rejections, and helps by reducing
> the number of times the slow auth lookup needs checking.
>
>> http_access deny maxuser
>> http_access allow localhost
>
> If localhost really is allowed to do anything, move it up above the
> "to_localhost" one.
> Otherwise drop this completely, having the correct auth login details
> will permit links from localhost just as easily as from anywhere else.
>
>> http_access deny all
>> icp_access allow all
>
> Define the networks where peer siblings are trusted. Allwo them and deny
> everything else.
> That will reduce a fair bit of load on your Squid trying to service
> random ICP requests from the general Internet.
>
>> http_port 8080
>> http_port 88.xxx.xxx.xxx:80
>> hierarchy_stoplist cgi-bin ?
>> cache_mem 100 MB
>
> Bump this up as high as you can go without risking memory swapping.
> Objects served from RAM are 100x faster than objects not.
>
>> maximum_object_size_in_memory 50 KB
>> cache_replacement_policy heap LFUDA
>> cache_dir aufs /var/spool/squid 40000 16 256
>
> If you pick 2.x squid to upgrade to, add a COSS directory as well.
> See the recent threads on optimizing COSS for how to tune that.
>
>> maximum_object_size 50 MB
>
> Bump this up too. Holding full ISO CDs and windows service packs can
> boost performance when one is used from the cache. 40GB of disk can
> store a few.
>
>> cache_swap_low 90
>> cache_swap_high 95
>> access_log /var/log/squid/access.log squid
>> cache_log /var/log/squid/cache.log
>> buffered_logs on
>> acl QUERY urlpath_regex cgi-bin \?
>> cache deny QUERY
>
> Drop the QUERY bits above. It's more than halving the things your Squid
> can store.
>
>> refresh_pattern ^ftp: 1440 20% 10080
>> refresh_pattern ^gopher: 1440 0% 1440
>
> Add right here:
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>
>> refresh_pattern . 0 20% 4320
>> quick_abort_min 0 KB
>> quick_abort_max 0 KB
>> acl apache rep_header Server ^Apache
>> broken_vary_encoding allow apache
>> half_closed_clients off
>> cache_mgr aaa_at_aaa.com
>> cachemgr_passwd aaa all
>> visible_hostname ProxyServer
>> log_icp_queries off
>> dns_nameservers 208.67.222.222 208.67.220.220
>> hosts_file /etc/hosts
>> memory_pools off
>
> Might cause efficiency problems if the underlying malloc is not
> optimized. but oh well, up to you.
>
>> forwarded_for off
>> client_db off
>> coredump_dir /var/spool/squid
>>
>
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE7 or 3.0.STABLE23
> Current Beta Squid 3.1.0.16
                                               
_________________________________________________________________
Do you have a story that started on Hotmail? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/
Received on Sun Feb 14 2010 - 08:17:29 MST