Re: [squid-users] FATAL: Too many queued redirector requests

Hi, I decreased the number of url_rewrite_children and upgraded squid
to version 3.1.12, now after two days in weekend (Sat and Sun) the
memory consuming has not exceeded the amount of defined cache_mem and
it seems to be working properly :)

Page faults with physical i/o: 0

I still monitoring it on working days to see what happen, but I thinks
the problem may be caused by too much redirector programs spawning.

Thank you.
Kien Le.

On Fri, Jun 10, 2011 at 1:27 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On 09/06/11 20:16, Le Trung Kien wrote:
>>
>> Hi, I don't know how to handle memory for squid. Squid automatically
>> breaks down itself and restarts:
>
> http://wiki.squid-cache.org/SquidFaq/SquidMemory
>
>>
>> In my squid.conf I set:
>>
>> url_rewrite_children 120 startup=30 idle=100 concurrency=0
>
> You need 3.2 for the dynamic startup capabilities. 3.1 only uses the first
> value (120 helpers to start *immediately*).
>
> Given that is actually a reasonable number of helpers I think the memory
> consumption elsewhere is probably killing them.
>
>
>
>> max_filedesc 7168
>> cache_swap_low 60
>> cache_swap_high 80
>
> If you are forced to do that with swap values it means you cache management
> is badly calculated.
>
> There is no use allocating for example 100GB of disk storage when
> cache_swap_low forces 40% of it not to be used. And Squid pausing
> occasionally while 20% of it (80%-60%) is erased can also cause visible
> slowdown to clients.
>
> The memory FAQ page linked above outlines how to estimate cache sizes.
>
>>
>> I notice that just one pending requests queued:
>>
>> 2011/06/09 11:00:17| WARNING: All redirector processes are busy.
>> 2011/06/09 11:00:17| WARNING: 1 pending requests queued
>>
>> and follow is more details of cache.log:
>>
>> 2011/06/09 11:00:16| WARNING: Cannot run
>> '/opt/squid-3.1.10/urlrewriter_new.pl' process.
>> 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory
>
>
>> 2011/06/09 11:00:16| WARNING: Cannot run
>> '/opt/squid-3.1.10/urlrewriter_new.pl' process.
>> 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory
>> 2011/06/09 11:00:16| WARNING: Cannot run
>> '/opt/squid-3.1.10/urlrewriter_new.pl' process.
>> 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory
>> 2011/06/09 11:00:16| WARNING: Cannot run
>> '/opt/squid-3.1.10/urlrewriter_new.pl' process.
>> 2011/06/09 11:00:17| WARNING: All redirector processes are busy.
>> 2011/06/09 11:00:17| WARNING: 1 pending requests queued
>> 2011/06/09 11:00:17| storeDirWriteCleanLogs: Starting...
>> 2011/06/09 11:00:17| WARNING: Closing open FD  177
>> 2011/06/09 11:00:17|   Finished.  Wrote 16367 entries.
>> 2011/06/09 11:00:17|   Took 0.01 seconds (1308941.14 entries/sec).
>> FATAL: Too many queued redirector requests
>> Squid Cache (Version 3.1.10): Terminated abnormally.
>
> Your machine seems not to be capable of running 120 or these rewriter
> processes. You need more available to them RAM or less maximum helpers.
>
>
>> <----------------------------------  CRASH ***
>> CPU Usage: 6631.678 seconds = 4839.266 user + 1792.412 sys
>> Maximum Resident Size: 0 KB
>> Page faults with physical i/o: 3299
>
> "Page faults" - your system is swapping badly to get that many just on
> startup. If this continues it will kill Squid performance later even if it
> gets past the helpers.
>
> Amos
> --
> Please be using
>  Current Stable Squid 2.7.STABLE9 or 3.1.12
>  Beta testers wanted for 3.2.0.8 and 3.1.12.2
>
Received on Mon Jun 13 2011 - 02:23:15 MDT