Re: [squid-users] Squid losing connectivity for 30 seconds

>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am currently facing a problem that I wasn't able to find a
>>>>>>> solution for in the mailing list or on the internet,
>>>>>>> My squid is dying for 30 seconds every one hour at the same
>>>>>>> exact time, squid process will still be running,
>>>>>>> I lose my wccp connectivity, the cache peers detect the squid as
>>>>>>> a dead sibling, and the squid cannot server any requests
>>>>>>> The network connectivity of the sever is not affected (a ping to
>>>>>>> the squid's ip doesn't timeout)
>>>>>>>
>>>>>>> The problem doesn't start immediately when the squid is
>>>>>>> installed on the server (The server is dedicated as a squid)
>>>>>>> It starts when the cache directories starts to fill up,
>>>>>>> I have started my setup with 10 cache directors, the squid will
>>>>>>> start having the problem when the cache directories are above
>>>>>>> 50% filled
>>>>>>> when i change the number of cache directory (9,8,...) the squid
>>>>>>> works for a while then the same problem
>>>>>>> cache_dir aufs /cache1/squid 90000 140 256
>>>>>>> cache_dir aufs /cache2/squid 90000 140 256
>>>>>>> cache_dir aufs /cache3/squid 90000 140 256
>>>>>>> cache_dir aufs /cache4/squid 90000 140 256
>>>>>>> cache_dir aufs /cache5/squid 90000 140 256
>>>>>>> cache_dir aufs /cache6/squid 90000 140 256
>>>>>>> cache_dir aufs /cache7/squid 90000 140 256
>>>>>>> cache_dir aufs /cache8/squid 90000 140 256
>>>>>>> cache_dir aufs /cache9/squid 90000 140 256
>>>>>>> cache_dir aufs /cache10/squid 80000 140 256
>>>>>>>
>>>>>>> I have 1 terabyte of storage
>>>>>>> Finally I created two cache dircetories (One on each HDD) but
>>>>>>> the problem persisted
>>>>>>
>>>>>> You have 2 HDD? but, but, you have 10 cache_dir.
>>>>>> We repeatedly say "one cache_dir per disk" or similar. In
>>>>>> particular one cache_dir per physical drive spindle (for "disks"
>>>>>> made up of multiple physical spindles) wherever possible with
>>>>>> physical drives/spindles mounting separately to ensure the
>>>>>> pairing. Squid performs a very unusual pattern of disk I/O which
>>>>>> stress them down to the hardware controller level and make this
>>>>>> kind of detail critical for anything like good speed. Avoiding
>>>>>> cache_dir object limitations by adding more UFS-based dirs to one
>>>>>> disk does not improve the situation.
>>>>>>
>>>>>> That is a problem which will be affecting your Squid all the time
>>>>>> though, possibly making the source of the pause worse.
>>>>>>
>>>>>> From teh description I believe it is garbage collection on the
>>>>>> cache directories. The pauses can be visible when garbage
>>>>>> collecting any caches over a few dozen GB. The squid default
>>>>>> "swap_high" and "swap_low" values are "5" apart, with at minimum
>>>>>> being a value of 0 apart. These are whole % points of the total
>>>>>> cache size, being erased from disk in a somewhat random-access
>>>>>> style across the cache area. I did mention uncommon disk I/O
>>>>>> patterns, right?
>>>>>>
>>>>>> To be sure what it is, you can use the "strace" tool to the squid
>>>>>> worker process (the second PID in current stable Squids) and see
>>>>>> what is running. But given the hourly regularity and past
>>>>>> experience with others on similar cache sizes, I'm almost certain
>>>>>> its the garbage collection.
>>>>>>
>>>>>> Amos
>>>>>>
>>>>>
>>>>> Hi Amos,
>>>>>
>>>>> Thank you for your fast reply,
>>>>> I have 2 HDD (450GB and 600GB)
>>>>> df -h displays that i have 357Gb and 505GB available
>>>>> In my last test, my cache dir where:
>>>>> cache_swap_low 90
>>>>> cache_swap_high 95
>>>>
>>>> This is not. For anything more than 10-20 GB I recommend setting it
>>>> to no more than 1 apart, possibly the same value if that works.
>>>> Squid has a light but CPU-intensive and possibly long garbage
>>>> removal cycle above cache_swap_low, and a much more aggressive but
>>>> faster and less CPU intensive removal above cache_swap_high. On
>>>> large caches it is better in terms of downtime going straight to
>>>> the aggressive removal and clearing disk space fast, despite the
>>>> bandwidth cost replacing any items the light removal would have left.
>>>>
>>>>
>>>> Amos
>>>>
>>> Hi Amos,
>>>
>>> I have changed the swap_high 90 and swap_low 90 with two cache dir
>>> (one for each HDD), i still have the same problem,
>>> I did an strace (when the problem occured)
>>> ------ ----------- ----------- --------- --------- ----------------
>>> 23.06 0.004769 0 85681 96 write
>>> 21.07 0.004359 0 24658 5 futex
>>> 19.34 0.004001 800 5 open
>>> 6.54 0.001352 0 5101 5101 connect
>>> 6.46 0.001337 3 491 epoll_wait
>>> 5.34 0.001104 0 51938 9453 read
>>> 3.90 0.000806 0 39727 close
>>> 3.54 0.000733 0 86400 epoll_ctl
>>> 3.54 0.000732 0 32357 sendto
>>> 2.02 0.000417 0 56721 recvmsg
>>> 1.84 0.000381 0 24064 socket
>>> 0.96 0.000199 0 56264 fcntl
>>> 0.77 0.000159 0 6366 329 accept
>>> 0.53 0.000109 0 24033 bind
>>> 0.52 0.000108 0 30085 getsockname
>>> 0.21 0.000044 0 11200 stat
>>> 0.21 0.000044 0 6998 359 recvfrom
>>> 0.09 0.000019 0 5085 getsockopt
>>> 0.06 0.000012 0 2887 lseek
>>> 0.00 0.000000 0 98 brk
>>> 0.00 0.000000 0 16 dup2
>>> 0.00 0.000000 0 10314 setsockopt
>>> 0.00 0.000000 0 4 getdents
>>> 0.00 0.000000 0 3 getrusage
>>> ------ ----------- ----------- --------- --------- ----------------
>>> 100.00 0.020685 560496 15343 total
>>>
>>> this is the strace of squid when it is working normally:
>>> ------ ----------- ----------- --------- --------- ----------------
>>> 24.88 0.015887 0 455793 169 write
>>> 13.72 0.008764 0 112185 epoll_wait
>>> 11.67 0.007454 0 256256 27158 read
>>> 8.47 0.005408 0 169133 sendto
>>> 6.94 0.004430 0 159596 close
>>> 6.85 0.004373 0 387359 epoll_ctl
>>> 6.42 0.004102 0 19651 19651 connect
>>> 5.54 0.003538 0 290289 recvmsg
>>> 3.81 0.002431 0 116515 socket
>>> 3.53 0.002254 0 164750 futex
>>> 1.68 0.001075 0 207688 fcntl
>>> 1.53 0.000974 0 95228 23139 recvfrom
>>> 1.29 0.000821 0 33408 12259 accept
>>> 1.14 0.000726 0 46582 stat
>>> 1.11 0.000707 0 110826 bind
>>> 0.85 0.000544 0 137574 getsockname
>>> 0.32 0.000204 0 21642 getsockopt
>>> 0.26 0.000165 0 39502 setsockopt
>>> 0.01 0.000007 0 8092 lseek
>>> 0.00 0.000000 0 248 open
>>> 0.00 0.000000 0 4 brk
>>> 0.00 0.000000 0 88 dup2
>>> 0.00 0.000000 0 14 getdents
>>> 0.00 0.000000 0 6 getrusage
>>> ------ ----------- ----------- --------- --------- ----------------
>>> 100.00 0.063864 2832429 82376 total
>>>
>>> Do you have any suggestions to solve the issue, can I run the
>>> garbage collector more frequently, is it better to change the
>>> cache_dir type from aufs to something else?
>>> Do you see the problem in the strace?
>>>
>>> Thank you,
>>> Elie
>>>
>>>
>> Hi,
>>
>> Please note that squid is facing the same problem even when their is
>> no activity or any clients connected to it
>>
>> Regards
>> Elie
>>
> Hi,
>
> here is the strace result
> -----------------------------------------------------------------------------------------------------
>
> connect(7595, {sa_family=AF_INET, sin_port=htons(80),
> sin_addr=inet_addr("2.22.50.75")}, 16) = -1 EINPROGRESS (Operation now
> in progress)
> epoll_ctl(6, EPOLL_CTL_ADD, 7595, {EPOLLOUT|EPOLLERR|EPOLLHUP,
> {u32=7595, u64=13133095884688465323}}) = 0
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\375\254\21\0002r\316N\0\0\0\0002r\316N\0\0\0\0\262\245\257P\0\0\0\0"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\306\254\21\0000r\316N\0\0\0\0000r\316N\0\0\0\0\377\377\377\377\377\377\377\377"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\262\251\21\0\360\376\315N\0\0\0\0002r\316N\0\0\0\0\377\377\377\377\377\377\377\377"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\2\255\21\0002r\316N\0\0\0\0003r\316N\0\0\0\0\321\376\316N\0\0\0\0"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\3\255\21\0002r\316N\0\0\0\0003r\316N\0\0\0\0002\347\340N\0\0\0\0"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\371\254\21\0002r\316N\0\0\0\0002r\316N\0\0\0\0\377\377\377\377\377\377\377\377"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\n\255\21\0003r\316N\0\0\0\0003r\316N\0\0\0\0\263\245\257P\0\0\0\0"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\v\255\21\0003r\316N\0\0\0\0003r\316N\0\0\0\0\323\370\317N\0\0\0\0"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\f\255\21\0003r\316N\0\0\0\0003r\316N\0\0\0\0003\377\365N\0\0\0\0"...,
> 72) = 72
> futex(0x9f65ec, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9f65e8, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> futex(0x9f65c0, FUTEX_WAKE_PRIVATE, 1) = 1
> write(35,
> "\2\0\0\0\334\251\21\0@\\\316N\0\0\0\0003r\316N\0\0\0\0\377\377\377\377\377\377\377\377"...,
> 72) = 72
> read(165, "!", 256) = 1
> close(379) = 0
> epoll_ctl(6, EPOLL_CTL_DEL, 9525, {0, {u32=9525,
> u64=25160128873375029}}) = 0
> epoll_ctl(6, EPOLL_CTL_ADD, 9525, {EPOLLIN|EPOLLERR|EPOLLHUP,
> {u32=9525, u64=13133094175291483445}}) = 0
> epoll_ctl(6, EPOLL_CTL_MOD, 9525, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP,
> {u32=9525, u64=23686740342482229}}) = 0
> epoll_ctl(6, EPOLL_CTL_MOD, 4554, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP,
> {u32=4554, u64=549755818442}}) = 0
> close(68) = 0
> socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 68
> fcntl(68, F_GETFD) = 0
> fcntl(68, F_SETFD, FD_CLOEXEC) = 0
> setsockopt(68, SOL_IP, 0x13 /* IP_??? */, [1], 4) = 0
> bind(68, {sa_family=AF_INET, sin_port=htons(0),
> sin_addr=inet_addr("95.141.62.17")}, 16) = 0
> fcntl(68, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(68, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> setsockopt(68, SOL_TCP, TCP_NODELAY, [1], 4) = 0
> getsockname(68, {sa_family=AF_INET, sin_port=htons(60303),
> sin_addr=inet_addr("95.141.62.17")}, [16]) = 0
> socket(PF_NETLINK, SOCK_RAW, 0) = 379
> bind(379, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0
> getsockname(379, {sa_family=AF_NETLINK, pid=11594, groups=00000000},
> [12]) = 0
> sendto(379, "\24\0\0\0\26\0\1\3t\232\324N\0\0\0\0\0\0\0\0", 20, 0,
> {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20
> recvmsg(379, {msg_name(12)={sa_family=AF_NETLINK, pid=0,
> groups=00000000},
> msg_iov(1)=[{"0\0\0\0\24\0\2\0t\232\324NJ-\0\0\2\10\200\376\1\0\0\0\10\0\1\0\177\0\0\1"...,
> 4096}], msg_controllen=0, msg_flags=0}, 0) = 108
> recvmsg(379, {msg_name(12)={sa_family=AF_NETLINK, pid=0,
> groups=00000000},
> msg_iov(1)=[{"@\0\0\0\24\0\2\0t\232\324NJ-\0\0\n\200\200\376\1\0\0\0\24\0\1\0\0\0\0\0"...,
> 4096}], msg_controllen=0, msg_flags=0}, 0) = 128
> recvmsg(379, {msg_name(12)={sa_family=AF_NETLINK, pid=0,
> groups=00000000},
> msg_iov(1)=[{"\24\0\0\0\3\0\2\0t\232\324NJ-\0\0\0\0\0\0\1\0\0\0\24\0\1\0\0\0\0\0"...,
> 4096}], msg_controllen=0, msg_flags=0}, 0) = 20
> close(379) = 0
> connect(68, {sa_family=AF_INET, sin_port=htons(80),
> sin_addr=inet_addr("2.22.50.65")}, 16) = -1 EINPROGRESS (Operation now
> in progress)
> epoll_ctl(6, EPOLL_CTL_ADD, 68, {EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=68,
> u64=13133092173836714052}}) = 0
> epoll_ctl(6, EPOLL_CTL_MOD, 3658, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP,
> {u32=3658, u64=549755817546}}) = 0
> sendto(10, "\33\221\1\0\0\1\0\0\0\0\0\0\nliveupdate\vgocyberl"..., 44,
> 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 44
> sendto(10, "\275\251\1\0\0\1\0\0\0\0\0\0\2tc\2v3\6cache3\1c\4pack"...,
> 48, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 48
> sendto(10,
> "\267\355\1\0\0\1\0\0\0\0\0\0\3crl\6thawte\3com\0\0\34\0\1", 32, 0,
> {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 32
> sendto(10, "\312\237\1\0\0\1\0\0\0\0\0\0\2p4\rbhzfrwddqtb66\20cg"...,
> 79, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 79
> sendto(10,
> "\261\216\1\0\0\1\0\0\0\0\0\0\4apps\10facebook\3com\0\0"..., 35, 0,
> {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 35
> sendto(10, "\237\10\1\0\0\1\0\0\0\0\0\0\6sn121w\6snt121\4mail\4"...,
> 45, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 45
> sendto(10, "FR\1\0\0\1\0\0\0\0\0\0\3api\6js-kit\3com\0\0\1\0\1", 32,
> 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 32
> sendto(10, "\207$\1\0\0\1\0\0\0\0\0\0\1l\naddthiscdn\3com\0\0\1"...,
> 34, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 34
> sendto(10, "\265#\1\0\0\1\0\0\0\0\0\0\5usage\7hosting\7toolb"..., 60,
> 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 60
> sendto(10, "hZ\1\0\0\1\0\0\0\0\0\0\3www\ntargetgulf\3com\0"..., 36, 0,
> {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 36
> sendto(10, "\355\345\1\0\0\1\0\0\0\0\0\0\10services\4apps\7condu"...,
> 43, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 43
> sendto(10, "O\254\1\0\0\1\0\0\0\0\0\0\0060-jw-w\7channel\10face"...,
> 45, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 45
> sendto(10, "\330\310\1\0\0\1\0\0\0\0\0\0\2t1\7gstatic\3com\0\0\1\0\1",
> 32, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 32
> sendto(10, "\16Z\1\0\0\1\0\0\0\0\0\0\4mail\3aol\3com\0\0\1\0\1", 30,
> 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 30
> sendto(10,
> "\327\216\1\0\0\1\0\0\0\0\0\0\3www\10bisara7a\3com\0\0\1"..., 34, 0,
> {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 34
> sendto(10, "o\201\1\0\0\1\0\0\0\0\0\0\003317\7channel\10faceboo"...,
> 42, 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 42
> sendto(10, "\344?\1\0\0\1\0\0\0\0\0\0\7email03\fsecureserve"..., 42,
> 0, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("10.249.209.102")}, 16) = 42
> sendto(21,
> "\0\0\0\n\2\0\0|\0\0\0\4\0\0\0\0\0\1\0\30\0017\360\6\0\0\0002\0P\0\0"...,
> 132, 0, NULL, 0) = 132
> sendto(21,
> "\0\0\0\n\2\0\0|\0\0\0\4\0\0\0\0\0\1\0\30\1<\360\6\0\0\0\21\0P\0\0"..., 132,
> 0, NULL, 0) = 132
> open("/cache2/pool6/squid/13/AB",
> O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 379
> getdents(379, /* 254 entries */, 32768) = 8112
> getdents(379, /* 0 entries */, 32768) = 0
> close(379) = 0
> read(165, "!", 256) = 1
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Squid is freezing at this point
> Please advise
>
> Thank you
> Elie
>
Hi,

Here is my compilation options
--------------------------------------------------------------------------------------------------------------------------------------------------------
./configure '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin'
'--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share'
'--includedir=/usr/include' '--libdir=/usr/lib64'
'--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib'
'--mandir=/usr/share/man' '--infodir=/usr/share/info'
'--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid'
'--localstatedir=/var' '--datadir=/usr/share/squid'
'--sysconfdir=/etc/squid' '--with-logdir=/var/log/squid'
'--with-pidfile=/var/run/squid.pid' '--disable-dependency-tracking'
'--enable-arp-acl' '--enable-follow-x-forwarded-for'
'--enable-auth=basic,digest,negotiate'
'--enable-external-acl-helpers=ip_user,unix_group,wbinfo_group'
'--enable-cache-digests' '--enable-cachemgr-hostname=localhost'
'--enable-delay-pools' '--enable-epoll' '--enable-icap-client'
'--enable-ident-lookups' '--enable-linux-netfilter'
'--enable-referer-log' '--enable-removal-policies=lru' '--enable-snmp'
'--enable-ssl' '--enable-storeio=aufs,ufs' '--enable-wccpv2'
'--enable-esi' '--with-aio' '--with-default-user=proxy'
'--with-filedescriptors=65536' '--with-dl' '--with-pthreads'
'--with-libcap' '--with-netfilter-conntrack' '--with-openssl'
'--enable-inline' '--enable-uselect' '--enable-disk-io' '--disable-htcp'
'--with-gnu-ld' '--with-build-environment=default' '--enable-carp'
'--enable-async-io=26' --with-squid=/home/squid-3.1.15
--enable-ltdl-convenience
---------------------------------------------------------------------------------------------------------------------------------------------------------
Here is my squid.conf:
---------------------------------------------------------------------------------------------------------------------------------------------------------
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl clients src x.x.x.x
#icp acl
acl squidFarm src x.x.x.x
acl self src x.x.x.x

#ICAP acl
acl icap_port1 myportname 3144
acl icap_port2 myportname 3145
acl icap_port3 myportname 3146
acl icap_port5 myportname 3148

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access allow localhost
#prevent digest loop
http_access deny self
http_access allow clients
http_access deny all
http_reply_access allow all
#icp_access allow all
icp_port 3130
icp_access allow squidFarm
icp_access deny all

http_port 3129 tproxy
http_port 3128 transparent
http_port 3144 tproxy
http_port 3146 tproxy
http_port 3145 tproxy
http_port 3148 tproxy

forwarded_for off
via off
visible_hostname x.x.x.x
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid
# Image files
refresh_pattern -i \.png$ 10080 90% 43200
refresh_pattern -i \.gif$ 10080 90% 43200
refresh_pattern -i \.jpg$ 10080 90% 43200
refresh_pattern -i \.jpeg$ 10080 90% 43200
refresh_pattern -i \.bmp$ 10080 90% 43200
refresh_pattern -i \.tif$ 10080 90% 43200
refresh_pattern -i \.tiff$ 10080 90% 43200

# Compressed files
refresh_pattern -i \.zip$ 10080 90% 43200
refresh_pattern -i \.rar$ 10080 90% 43200
refresh_pattern -i \.tar$ 10080 90% 43200
refresh_pattern -i \.gz$ 10080 90% 43200
refresh_pattern -i \.tgz$ 10080 90% 43200
refresh_pattern -i \.z$ 10080 90% 43200
refresh_pattern -i \.arj$ 10080 90% 43200
refresh_pattern -i \.lha$ 10080 90% 43200
refresh_pattern -i \.lzh$ 10080 90% 43200

# Binary files
refresh_pattern -i \.exe$ 10080 90% 43200
refresh_pattern -i \.msi$ 10080 90% 43200

# Multimedia files
refresh_pattern -i \.mp3$ 10080 90% 43200
refresh_pattern -i \.wav$ 10080 90% 43200
refresh_pattern -i \.mid$ 10080 90% 43200
refresh_pattern -i \.midi$ 10080 90% 43200
refresh_pattern -i \.ram$ 10080 90% 43200
refresh_pattern -i \.ra$ 10080 90% 43200
refresh_pattern -i \.mov$ 10080 90% 43200
refresh_pattern -i \.avi$ 10080 90% 43200
refresh_pattern -i \.wmv$ 10080 90% 43200
refresh_pattern -i \.mpg$ 10080 90% 43200
refresh_pattern -i \.mpg$ 10080 90% 43200
refresh_pattern -i \.mpg$ 10080 90% 43200
refresh_pattern -i \.mpeg$ 10080 90% 43200
refresh_pattern -i \.swf$ 10080 90% 43200

# Document files
refresh_pattern -i \.pdf$ 10080 90% 43200
refresh_pattern -i \.ps$ 10080 90% 43200
refresh_pattern -i \.doc$ 10080 90% 43200
refresh_pattern -i \.ppt$ 10080 90% 43200
refresh_pattern -i \.pps$ 10080 90% 43200
#windows update refresh paterns
refresh_pattern windowsupdate.com/.*\.(cab|exe|psf) 4320 100% 43200
reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe|psf) 4320 100% 43200
reload-into-ims
refresh_pattern armdl.adobe.com/.*\.(cab|msp|msi) 4320 100% 43200
reload-into-ims
#default refresh paterns
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

client_db on
cache_mem 256 MB
cache_swap_low 90
cache_swap_high 90
maximum_object_size 512 MB
maximum_object_size_in_memory 20 KB
cache_dir aufs /cache1/squid 320000 480 256
cache_dir aufs /cache2/squid 480000 700 256

#logformat modified %tl Request time:%tr Status:%Ss/%03>Hs Client:%>a
URL:%ru Server:%<A Type:%mt
logformat modified %tl %>a %ru %<A
access_log /var/log/squid/access.log modified
cache_log /var/log/squid/cache.log
cache_store_log none
coredump_dir /var/spool/squid
half_closed_clients off

snmp_port x
acl cacti src x.x.x.x
acl snmpcommunity snmp_community xxxx
snmp_access allow snmpcommunity xxxx
snmp_access allow snmpcommunity localhost
snmp_access deny all

wccp2_router x.x.x.x
wccp2_forwarding_method l2
wccp2_return_method l2
wccp2_service dynamic x
wccp2_service_info x protocol=tcp flags=src_ip_hash priority=240 ports=80
wccp2_service dynamic x
wccp2_service_info x protocol=tcp flags=dst_ip_hash,ports_source
priority=240 ports=80
wccp2_assignment_method mask

#icp configuration
maximum_icp_query_timeout 30
cache_peer x.x.x.x sibling 3128 3130 proxy-only no-tproxy
cache_peer x.x.x.x sibling 3128 3130 proxy-only no-tproxy
cache_peer x.x.x.x sibling 3128 3130 proxy-only no-tproxy
log_icp_queries off
miss_access allow squidFarm
miss_access deny all

# ICAP configuration
icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_encode off
icap_client_username_header X-Client-Username
icap_preview_enable on
icap_preview_size 1024

logformat squid %tl %icap::tt %icap::tr %>a %icap::rm %icap::ru
icap_log /var/log/squid/icap.log squid
icap_service service_req reqmod_precache bypass=1 icap://x.x.x.x:x/reqmod
adaptation_access service_req allow icap_port1
icap_service service_req_2 reqmod_precache bypass=1 icap://x.x.x.x:x/reqmod
adaptation_access service_req_2 allow icap_port2
icap_service service_req_3 reqmod_precache bypass=1 icap://x.x.x.x:x/reqmod
adaptation_access service_req_3 allow icap_port3
icap_service service_req_5 reqmod_precache bypass=1 icap://x.x.x.x:x/reqmod
adaptation_access service_req_5 allow icap_port5
---------------------------------------------------------------------------------------------------------------------------------------------------------

Please advise,
Thank you
Elie
Received on Thu Dec 01 2011 - 14:17:08 MST