Re: [squid-users] 100% CPU when bigger files are downloading

On 07/23/2013 10:01 PM, Tim Bates wrote:
> OK, so I've got Squid 3.1.6 (from Debian Wheezy) running on a OpenVZ
> container. There are very few users, often just one at a time - we use
> this for caching and filtering customer computers being repaired in a
> small computer repair shop.
>
> Any time we download anything over a couple of megabytes in size, the
> CPU hits 100%, and the download gets progressively slower. Starts off
> around the network speed (100mbps) but over a period of about 30 seconds
> it will drop down to as low as 20KB/s (200kbps). Sometimes it gets even
> worse...

Hello Tim,

    This may be a known problem attributed to Squid memory storage code
that is extremely inefficient at handling huge cached responses. When it
was first reported a few months ago, I was able to reproduce it by
caching 10+MB files in Squid memory cache.

> maximum_object_size 350 MB

You configuration should not allow large files into the memory cache
because you do not increase maximum_object_size_in_memory from its 512KB
default. There are several possibilities here, including:

1) You are suffering from some old v3.1 bug unrelated to the issue
discussed above. Upgrade to the latest v3.3 to verify.

2) You are suffering from the issue discussed above due to v3.1 bugs
that incorrectly limit the size of entries in the memory cache. Upgrade
to the latest v3.3 to verify.

3) You are suffering from the issue discussed above. Our initial
understanding that the problem is limited to memory-cached entries needs
to be adjusted to include large disk-cached entries as well. Triage by
developers is probably needed to verify that (I am not sure you can
easily confirm the problem using just logs and other admin-level tools).

If upgrade does not help, you should report this problem to bugzilla.

There are some hacks that mitigate the problem somewhat at the expense
of possibly incorrectly treating range and other special requests, but
serious development is required to fix the issue discussed above. I
cannot offer any patches at this time, unfortunately.

HTH,

Alex.
Received on Wed Jul 24 2013 - 16:21:55 MDT