Re: [squid-users] Squid performance issues

Felipe W Damasio wrote:
> Hi Mr. Robertson,
>
> 2010/1/26 Chris Robertson <crobertson_at_gci.net>:
>>> Do you have any idea or any other data I can collect to try and
>>> track down this?
>>>
>> Check your log rotation schedule. Is it possible that logs are being
>> rotated at midnight? I think that the swap.state file is rewritten when
>> "squid -k rotate" is called. Check the beginning of your cache.log to
>> verify.
>
> I don't use -k rotate.
>
> At midnight, the only thing that changes is that the traffic shapper
> of the ISP let's everything run loose, ie. it allows http requests go
> through the roof.
>
> The youtube requests, which is what I really care about, goes form
> and avg of 20Mbps of traffic (shaped), to around 50Mbps unshapped.
>
> This is, I suppose, what's causing squid to slowdown....but squid
> is able to handle this kind of traffic increase, isn't it?

Should yes. I have theory though...

  ... One of the major differences between 2.x and 3.x is that for
objects in memory 2.x must walk a list of chunks 4KNB in size from the
start of the file every time it sends a bit. 3.x is able to keep its
last read position.
This can cause 2.x to use a lot of CPU time walking the objects if Squid
is forced to send them in small bits at a time.

Since its youtube (4-8MB objects) you are talking about and you only do
memory caching I suspect the problem is that a lot of new requests
arrive at around the same time for large objects. Squid then has to wait
for each bit of the reply to trickle in and ends up spending a lot of
time switching between replies and walking the object to find the next
bit to send out.

You can test this by trying 3.x. In particular 3.1 which has the better
memory caching of the 3.x line. (I'm releasing 3.1.0.16 this weekend if
you can wait that long).

>
> I don't think is the correct behavior squid at this time slowdown
> from 0.04s to load a HTML file to 23-40 seconds
>
> I'll run iostat and vmstat tonight to see if I get more info to
> track this down, and I'll send them to the list tomorrow.

strace may provide some more info if those are unclear.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21
   Current Beta Squid 3.1.0.15