RE: [squid-users] poor performance

On 30-Aug-07 My Secret NSA Wiretap Overheard Lutieri G. Saying :
> Hi!
>
> Today i'm running squid 2.5stable9 in a debian sarg box SUN v20z. All
> works very nice. Although, i need to migrate squid to a new server SUN
> x4100 running FreeBSD. And now begin my problems.
> I was talking about my performance problems with freebsd mailing list.
> But we can't find a solution for my problem described below:
>
> First of all, i'll paste some informations about my new server.
>
>#uname -a
> FreeBSD sd.xyz.com.br 6.2-STABLE FreeBSD 6.2-STABLE #0: Wed Aug 29
> 10:26:18 BRT 2007
> root@sd.xyz.com.br:/usr/src/sys/amd64/compile/LGB amd64

 I have a similiar units running FreeBSD.

>#mount
> /dev/da0s1a on / (ufs, local)
> devfs on /dev (devfs, local)
> /dev/da0s1e on /tmp (ufs, local, soft-updates)
> /dev/da0s1f on /usr (ufs, local, soft-updates)
> /dev/da0s1d on /var (ufs, local, soft-updates)
> /dev/da0s2a on /cache (ufs, local, soft-updates)
> devfs on /var/chroot/named/dev (devfs, local)
>

I am curious why you opted for a seperate slice for the /cache?
 I found that to be slower than just having a seperate partition, as odd as
that seems.

>#df -h
> Filesystem Size Used Avail Capacity Mounted on
> /dev/da0s1a 496M 88M 368M 19% /
> devfs 1.0K 1.0K 0B 100% /dev
> /dev/da0s1e 496M 538K 456M 0% /tmp
> /dev/da0s1f 9.4G 4.5G 4.1G 52% /usr
> /dev/da0s1d 4.7G 1.5G 2.9G 34% /var
> /dev/da0s2a 9.4G 89M 8.6G 1% /cache
> devfs 1.0K 1.0K 0B 100% /var/chroot/named/dev
>
> My kernel was compiled in the day before yesterday with some tunning options:
>
> Commented:
> options INET6 # IPv6
>
> - included:
> options SYSVSHM #SYSV-style shared memory
> options SYSVMSG #SYSV-style message queues
> options SYSVSEM #SYSV-style semaphores
> options SMP # Symmetric
>
> options HZ=2000
> options DEVICE_POLLING # Soft intrrupt's
> options VFS_AIO
> options MAXDSIZ=(4096UL*1024*1024) # Conf para 4Gb
> options MAXSSIZ=(256UL*1024*1024) # E aqui vai pra 128
> options DFLDSIZ=(4096UL*1024*1024) # 4096 tb!
>
> # Message Queues [Based on Squid FAQ]
>
> option MSGMNB=262144 # Number of bytes in a queue
> option MSGMNI=128 # Need to be at least 2 times the number of
> cache_dir entries in the squid
> option MSGSSZ=256 # Size of the message segment in a queue
> option MSGTQL=16384 # Number of max queue identifiers versus 128
> option MSGSEG=2048 # Number of messages segments
>
># Shared Memory [Based on Squid FAQ]
> options SHMMNI=256 # The half of the message queues at least [1 for
> each cache_dir]
> options SHMALL=65536 #
> options SHMMAX=(128UL*1024*1024) #
> options SHMSEG=128
>
>

 Instead of compiling into your kernel you can just use /boot/loader.conf
 Also, you can run AUFS without any of these tuning options.

# test if needed
#kern.ipc.maxsockets=16424
#
kern.ipc.msgmnb=16384
kern.ipc.msgmni=41
kern.ipc.msgseg=2049
kern.ipc.msgssz=64
## For DISKD
kern.ipc.msgtql=2048

kern.ipc.nmbclusters=32768

## Defaults are larger then the settings - why bother
#kern.ipc.shmmax=2097152
#kern.ipc.shmseg=16
#kern.ipc.shmmni=32
#kern.ipc.shmall=4096
#kern.maxfiles=8192

 As you can see, with the larger memory, some settings are already large enough.

> In squid.conf file :
>
> cache_dir diskd /usr/local/squid/cache/cache1 5120 16 256 Q1=128 Q2=100
> cache_dir diskd /usr/local/squid/cache/cache2 5120 16 256 Q1=128 Q2=100

 I'm confused. Above your showed a df with a /cache yet here you are aloting
space on /usr?

Also Q1=72 Q2=64 has worked well for me.

 
> cache_replacement_policy heap LFUDA
> memory_replacement_policy heap GDSF
> cache_mem 1536 MB
> cache_swap_low 65
> cache_swap_high 80
>
> maximum_object_size 64 MB
> minimum_object_size 0 KB
> maximum_object_size_in_memory 2560 Kb

 64 Mb is a pretty large objest size.

cache_mem 1000 MB
cache_swap_low 95
cache_swap_high 98
maximum_object_size 16096 KB
minimum_object_size 0 KB
maximum_object_size_in_memory 500 KB

 These have worked well for me with 4G of ram. Of course this depends on what
else you have running on this box. Also I think with the large variance with
high and low, the unit may spend too much time disconnecting while it deletes
objects to get down to the 65%. I could be wrong, but I found it better to not
have too wide a variance.

 
>#squid -v
> Squid Cache: Version 2.6.STABLE14
> configure options: '--bindir=/usr/local/sbin'
> '--sbindir=/usr/local/sbin' '--datadir=/usr/local/etc/squid'
> '--libexecdir=/usr/local/libexec/squid'
> '--localstatedir=/usr/local/squid' '--sysconfdir=/usr/local/etc/squid'
> '--enable-removal-policies=lru heap' '--disable-linux-netfilter'
> '--disable-linux-tproxy' '--disable-epoll' '--enable-auth=basic ntlm
> digest' '--enable-basic-auth-helpers=DB NCSA PAM MSNT SMB YP'
> '--enable-digest-auth-helpers=password'
> '--enable-external-acl-helpers=ip_user session unix_group
> wbinfo_group' '--enable-ntlm-auth-helpers=SMB'
> '--enable-negotiate-auth-helpers=squid_kerb_auth' '--with-pthreads'
> '--enable-storeio=ufs diskd null aufs' '--enable-delay-pools'
> '--enable-snmp' '--disable-carp' '--enable-ssl' '--with-openssl=/usr'
> '--enable-cache-digests' '--enable-arp-acl'
> '--enable-follow-x-forwarded-for' '--with-large-files'
> '--enable-large-cache-files' '--enable-err-languages=Azerbaijani
> Bulgarian Catalan Czech Danish Dutch English Estonian Finnish French
> German Greek Hebrew Hungarian Italian Japanese Korean Lithuanian
> Polish Portuguese Romanian Russian-1251 Russian-koi8-r Serbian
> Simplify_Chinese Slovak Spanish Swedish Traditional_Chinese Turkish'
> '--enable-default-err-language=English' '--enable-ntlm-fail-open'
> '--prefix=/usr/local' '--mandir=/usr/local/man'
> '--infodir=/usr/local/info/' 'amd64-portbld-freebsd6.2' 'CC=cc'
> 'CFLAGS=-O2 -fno-strict-aliasing -pipe -I/usr/include' 'CPPFLAGS='
> 'LDFLAGS= -rpath=/usr/lib:/usr/local/lib -L/usr/lib'
> 'build_alias=amd64-portbld-freebsd6.2'
> 'host_alias=amd64-portbld-freebsd6.2'
> 'target_alias=amd64-portbld-freebsd6.2'
>
>
> My new box is a X4100 SUN.
> With 4 gigabits NIC.
> Two SAS disks.
> 4Gb RAM

 So your df above was from the old box? Why not show the new unit?
 As someone else suggested. Having one disk be, say as big as needed for OS and
programs and the rest be cache and then the 2nd disk be all cache.

> I tryed use with and without RAID but I got the same problem.

Since you only have 2 disks, a mirror would be pointless unless you need the
uptime, then I recommend it. Geom mirror works very well as a software raid.
 
> Let me explain my environment:
> All my users use IE6 and they have proxy config like this:
> host: proxy.xyz.com.br
> port 3128
>
> Once i need to migrate all users to my new server with FreeBSD ,I only
> change the IP address of proxy register in DNS server.
> Ok. After two minutes i can see some users in the new server log(access.log).
>
> After 10 minutes squidclient mgr:info return me 70 clients and the
> speed connection goes very low for the users.

Perhaps try a netstat to see if you have too many users hanging on. Based on
the amount of dropped connections that may be likely.
You can try this to help. (/etc/sysctl.conf)

### expands port range - changes dropped tcp connections to 5min from 30
net.inet.ip.portrange.last=61000
net.inet.tcp.msl=5000

Also check your kern.ipc.somaxconn
 Also since you compiled in polling, which may or may not be usefull, I think
you also need: kern.polling.enable=1 in your /etc/sysctl.conf

 
> I commented out all acl's and authentication scheme in squid conf
> file. Running squid i'm monitoring with systat -v and iostat but CPU
> and disks is working almost all time as idle.
>
> cache.log doesn't show me errors. only this messages:
> 2007/08/30 08:13:31| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:32| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
> 2007/08/30 08:13:50| httpAccept: FD 39: accept failure: (53) Software
> caused connection abort
>
> I've read that is harmless.
>
> But the low speed continue.
> This is not a DNS problem. I've tested.
>
> Any suggestion?!?!
>
> tanx

 Nicole

 
>
> --
> Att.
> Lutieri G. B.

--
                     |\ __ /|   (`\            
                     | o_o  |__  ) )           
                    //      \\                 
  -  nmh@daemontech.com  -  Powered by FreeBSD  -
------------------------------------------------------
 "The term "daemons" is a Judeo-Christian pejorative.
 Such processes will now be known as "spiritual guides"
  - Politicaly Correct UNIX Page