RE: [squid-users] Some pages loading very slow in 3.1.10 Stable

On Tue, 25 Jan 2011 05:08:00 +1100, Saiful Alam <saifulmr_at_hotmail.com>
wrote:
> Dear Everyone,
> It is now 12AM here (GMT +0600) and very peak time for our ISP. Most of
> our users are now online. I took a snapshot from my cachemgr to show you
> guys.
>
> Squid Object Cache: Version 3.1.10
>
> Start Time:Mon, 24 Jan 2011 13:58:44 GMT
> Current Time:Mon, 24 Jan 2011 18:04:49 GMT
>
> Connection information for squid:
> Number of clients accessing cache: 570
> Number of HTTP requests received: 1468986
> Number of ICP messages received: 0
> Number of ICP messages sent: 0
> Number of queued ICP replies: 0
> Number of HTCP messages received: 0
> Number of HTCP messages sent: 0
> Request failure ratio: 0.00
> Average HTTP requests per minute since start: 5969.4
> Average ICP messages per minute since start: 0.0
> Select loop called: 36711620 times, 0.402 ms avg
> Cache information for squid:
> Hits as % of all requests: 5min: 35.1%, 60min: 35.6%
> Hits as % of bytes sent: 5min: 25.6%, 60min: 31.4%
> Memory hits as % of hit requests: 5min: 4.5%, 60min: 3.9%
> Disk hits as % of hit requests: 5min: 63.3%, 60min: 65.4%
> Storage Swap size: 142110940 KB
> Storage Swap capacity: 90.0% used, 10.0% free
> Storage Mem size: 152228 KB
> Storage Mem capacity: 100.1% used, 0.0% free
> Mean Object Size: 18.00 KB

152 MB of RAM shared between 570 users doe not look so good. If you can
increase that things should improve a bit.

> Requests given to unlinkd: 0
> Median Service Times (seconds) 5 min 60 min:
> HTTP Requests (All): 0.44492 0.44492
> Cache Misses: 0.76407 0.80651
> Cache Hits: 0.01309 0.01745
> Near Hits: 0.35832 0.37825
> Not-Modified Replies: 0.00000 0.00000
> DNS Lookups: 0.06963 0.09971
> ICP Queries: 0.00000 0.00000
> Resource usage for squid:
> UP Time: 14765.225 seconds
> CPU Time: 2012.280 seconds
> CPU Usage: 13.63%
> CPU Usage, 5 minute avg: 14.37%
> CPU Usage, 60 minute avg: 15.29%
> Process Data Segment Size via sbrk(): 1778280 KB
> Maximum Resident Size: 7213216 KB
> Page faults with physical i/o: 3
> Memory usage for squid via mallinfo():
> Total space in arena: 1778412 KB
> Ordinary blocks: 1750711 KB 20784 blks
> Small blocks: 0 KB 0 blks
> Holding blocks: 44844 KB 13 blks
> Free Small blocks: 0 KB
> Free Ordinary blocks: 27700 KB
> Total in use: 1795555 KB 98%
> Total free: 27700 KB 2%
> Total size: 1823256 KB
> Memory accounted for:
> Total accounted: 1325755 KB 73%
> memPool accounted: 1325755 KB 73%
> memPool unaccounted: 497500 KB 27%
> memPoolAlloc calls: 373493434
> memPoolFree calls: 375838488
> File descriptor usage for squid:
> Maximum number of file descriptors: 65535
> Largest file desc currently in use: 4467
> Number of file desc currently in use: 3790
> Files queued for open: 0
> Available number of file descriptors: 61745
> Reserved number of file descriptors: 100
> Store Disk files open: 60
> Internal Data Structures:
> 7896282 StoreEntries
> 16070 StoreEntries with MemObjects
> 15496 Hot Object Cache Items
>
>
> 7895225 on-disk objects
> Please see and check if there is anything which is not NORMAL, or might
be
> reason for problems discussed below.
>
> Regards
> Saiful
>
> ----------------------------------------
>> From: saifulmr_at_hotmail.com
>> To: squid-users_at_squid-cache.org
>> Subject: RE: [squid-users] Some pages loading very slow in 3.1.10
Stable
>> Date: Tue, 25 Jan 2011 05:00:30 +1100
>>
>>
>> Hi Amor,
>> I have recompiled Squid with the following parameters.
>>
>> ./configure --prefix=/usr/local/squid --enable-async-io=8
>> --enable-storeio="ufs,aufs,diskd" --enable-removal-policies="lru,heap"
>> --enable-delay-pools --enable-cache-digests --enable-underscores
>> --enable-icap-client --enable-follow-x-forwarded-for --enable-arp-acl
>> --enable-esi --enable-zph-qos --disable-translation --with-large-files
>> --with-filedescriptors=65536 --disable-ipv6 --enable-linux-netfilter
>>
>> DISK I/O is just like before, on average 5% I would say.
>> I am using the default removal policy, not heap.
>>
>> Well after recompiling it, I can see the Median DNS Lookup time reduced
>> by 10 times are the browsing experience is improved than before. Thank
>> you for that. But those two domains i.e www.music.com.bd and
>> www.djmaza.com problem is not solved. I cannot determine exactly what
is
>> stopping it to load the site fast, indeed it loads may be 10 minutes
>> later. To my (amateur) understanding, the problem is with the
>> ads.clicksor.com and the facebook widgets it tries to load, but I am
not
>> 100% sure.
>>
>> Can anyone try to load the site again and see if the problem persists,
>> or how should I exempt those URLs from being caching.
>>
>> I think the URLS are :-
>> 1)
>> http://ads.clicksor.com/showAd.php?pid=102683&adtype=1&sid=152482&zone=
>> 2)
>> http://ads.clicksor.com/showAd.php?pid=102683&adtype=9&sid=152482&zone=

These permit caching for a short while which should be okay. However they
are being served up by a broken system. It appears to be using at least two
servers, whose clocks are varying amounts of time out of sync with each
other and the rest of the world. Also sends broken Vary: headers, maybe
resulting in garbage data transfers to some clients.

You can block this with:
  acl clicksor dstdomain ads.clicksor.com
  http_access deny clicksor

>> 3)
>>
http://www.facebook.com/ajax/connect/activity_widget.php?__a=1&site=www.music.com.bd&width=350&height=225&header=false&colorscheme=light&border_color=%23006a4e&post_form_id=7c21c99b9bd83930bb871649bf465fcf&user=505688880&nb_activities=5&newest=0
>>
>> URL 3 waiting time sometimes goes more than 800ms. Please advise.

That seems normal for facebook in my experience. Nasty but under a second
for what it is doing (two object requests, a script execution and a server
login cycle, then another page load) is not that bad.

This is already blocked from caching by facebook. You can do an
http_access deny on the URL if you want to. I think its not worth that
though.
 acl facebook dstdomain www.facebook.com
 acl widgets urlpath_regex ^/ajax/connect/activity_widget.php
 http_access deny facebook widgets

Remembering that order is important, so both of the blocks need to go
above where the localnet gets allowed.

Amos
Received on Mon Jan 24 2011 - 23:01:27 MST