Re: [squid-users] Tuning problem in squid

Thanigairajan wrote:
> Hi,
> I have done everything which is said by Kinke,
> the problem has little bit rectified.
> i.e. it is comparatively good but if clients are using new sites(other
> than in cache) it is slow.
>
> My squid -v is as follows
>
> innovation_at_innovation:~$ squid -v
> Squid Cache: Version 2.6.STABLE18
> configure options: '--prefix=/usr' '--exec_prefix=/usr'
> '--bindir=/usr/sbin' '--sbindir=/usr/sbin'
> '--libexecdir=/usr/lib/squid' '--sysconfdir=/etc/squid'
> '--localstatedir=/var/spool/squid' '--datadir=/usr/share/squid'
> '--enable-async-io' '--with-pthreads'
> '--enable-storeio=ufs,aufs,coss,diskd,null' '--enable-linux-netfilter'
> '--enable-arp-acl' '--enable-epoll'
> '--enable-removal-policies=lru,heap' '--enable-snmp'
> '--enable-delay-pools' '--enable-htcp' '--enable-cache-digests'
> '--enable-underscores' '--enable-referer-log' '--enable-useragent-log'
> '--enable-auth=basic,digest,ntlm' '--enable-carp'
> '--enable-follow-x-forwarded-for' '--with-large-files'
> '--with-maxfd=65536' 'i386-debian-linux'
> 'build_alias=i386-debian-linux' 'host_alias=i386-debian-linux'
> 'target_alias=i386-debian-linux' 'CFLAGS=-Wall -g -O2'
> 'LDFLAGS=-Wl,-Bsymbolic-functions' 'CPPFLAGS='
>

Your Squid is kind of aging. We are now up to 2.7.STABLE6 or 3.0.STABLE16.
Even the Debian stable release is up to 2.7.STABLE3 or 3.0.STABLE8 already.

>
>
> Here i am pasting my squid.conf file (ecerpt)
>
> http_port 127.0.0.1:3128
> http_port 192.168.1.6:3128 transparent
> cache_effective_user proxy
> cache_effective_group proxy
> hierarchy_stoplist cgi-bin ?
> acl QUERY urlpath_regex cgi-bin \?
> cache deny QUERY
> acl apache rep_header Server ^Apache
> broken_vary_encoding allow apache
> cache_mem 48 MB

#1: low amount of memory available for recently hit of often-hit objects.

> maximum_object_size 8192 KB
> fqdncache_size 2048
> cache_dir ufs /var/spool/squid 1000 16 256

#2: ufs filesystem. You appear to have Linux therefore use AUFS.

#3: 1000 MB allocated for entire cache storage. Increase this to raise
local hits and thus speed.

> access_log /var/log/squid/access.log squid
> debug_options ALL,1
> log_fqdn off
> hosts_file /etc/hosts
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern . 0 20% 4320
> acl all src 0.0.0.0/0.0.0.0
> acl manager proto cache_object
> acl localhost src 127.0.0.1/255.255.255.255
> acl to_localhost dst 127.0.0.0/8
> acl Safe_ports port 465
> acl Safe_ports port 143
> acl purge method PURGE
> acl CONNECT method CONNECT
> http_access allow manager localhost
> http_access deny manager
> http_access allow purge localhost
> http_access deny purge
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> acl our_networks src 192.168.1.0/24
> acl ceo src 192.168.1.8
> acl ceo src 192.168.1.35
> acl normal_users src 192.168.1.159 192.168.1.160 192.168.1.161 192.168.1.162
> acl filetype url_regex -i .exe .mp3 .vqf .tar.gz .gz .rpm .zip .rar
> .avi .mpeg .mpe .mpg .qt .ram .rm .iso .raw .wav .mov .mp4 .msi
> delay_pools 1
> delay_class 1 2
> delay_parameters 1 -1/-1 100/6000
> delay_access 1 allow filetype normal_users

Hmmmm, as slow as 100 bytes per second perhapse?

That filetype regex will catch most requests.

* Remember that for regex '.' means any character

* Listing a pattern without anchors means it matches anywhere.

* The Squid url_regex pattern matches the entire URL:
protocol,domain,port,path,query-string all of it.

> http_access allow our_networks

NP: Entire network allowed to access the net, before special ranges...

These ...
> http_access allow ceo
> http_access allow normal_users
> http_access deny !normal_users
> http_access deny normal_users bannedsites
> http_access allow localhost
> http_access allow ceo
> http_access allow our_networks

... to here will never match.

> http_access deny all
> http_reply_access allow all
> cache_effective_user proxy
> cache_effective_group proxy
> visible_hostname innovation
> cache_mgr tech_support_at_sybrant.com
> coredump_dir /var/spool/squid
> redirector_bypass on
> redirect_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf
> redirect_children 10
> pipeline_prefetch on
>

Besides the cache_dir and delay_pools issues. I think its likely to be a
squidGuard issue. Simply calling and waiting for a redirector can slow
things down noticeably under load.

I'd also check the squidGuard rules are handled fast.

"squidclient mgr:redirector" looks for "avg service times"

>
> On Fri, Jun 12, 2009 at 8:00 PM, Kinkie <gkinkie_at_gmail.com> wrote:
>> On Fri, Jun 12, 2009 at 4:01 PM, Thanigairajan<methanigai_at_gmail.com> wrote:
>>> Hi ,
>>>
>>> I am facing some performance issues in squid .
>>>
>>> i.e. I have Debian etch with squid,squidguard,shorewall.
>>> Internet is working in normal speed if clients are approx 50 .
>>> If clients are approx 70 -100 it is getting very slow.
>>>
>>> I googled for tuning and done the following things,
>>> redirect_children 10
>>> cache_dir ufs /var/spool/squid 1000 16 256
>> ufs is definitely not suited for anything but testing. Please try aufs instead.
>>
>>> cache_mem 48 MB
>> 48Mb of cache_mem on a 4gb server? This could definitely be raised.
>>
>>> pipeline_prefetch on
>>> fqdncache_size 2048
>>> maximum_object_size 8192 KB
>>>
>>> Can you please suggest me how can i improve much ?
>>>
>>> FYI : We have Leased line .so we are getting constant bandwidth.
>>> We are running the server in desktop HP Compaq with 4GB RAM, Core2Duo
>> Unless your issues can be solved by these simple hints, we need to
>> have more informations, such as the output from "squid -v" and a more
>> complete configuration excerpt.
>>
>> --
>> /kinkie
>
>
>
> --
> Thanks & Regards
> MThanigairajan
>
> The Most Certain Way To Succeed Is To Try One More Time
>
> -- By Edison

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15
   Current Beta Squid 3.1.0.8 or 3.0.STABLE16-RC1