Help: Squid performance bombs when many sites are unreachable

Hi,

It seems that when a significant number of origin servers cannot be reached
then the performance of Squid takes a dive.

We are currently experiencing a routing issue whereby sites that sit behind
Exodus cannot be reached. These include Hotbot and some of Hotmail amongst
others. These are popular sites and the content is dynamic, so it isn't
cached. It appears that pending requests for these unreachable sites are
holding off requests for sites that can be reached.

A syslog extract looks like this:

Apr 27 11:07:17 adder squid[142]: WARNING: All redirector processes are busy.
Apr 27 11:07:17 adder squid[142]: WARNING: 8 pending requests queued
Apr 27 11:07:17 adder squid[142]: WARNING: All dnsserver processes are busy.
Apr 27 11:07:17 adder squid[142]: WARNING: 5 pending requests queued
Apr 27 11:17:17 adder squid[142]: WARNING: All redirector processes are busy.
Apr 27 11:17:17 adder squid[142]: WARNING: 1762 pending requests queued
Apr 27 11:17:20 adder squid[142]: Too many queued redirector requests
Apr 27 11:17:20 adder squid[142]: Too many queued redirector requests
Apr 27 11:17:23 adder squid[285]: Starting Squid Cache version 2.1.RELEASE
for i686-pc-linux-gnu...
Apr 27 11:17:28 adder squid[285]: WARNING: All redirector processes are busy.
Apr 27 11:17:28 adder squid[285]: WARNING: 8 pending requests queued
Apr 27 11:17:39 adder squid[285]: WARNING: All dnsserver processes are busy.
Apr 27 11:17:39 adder squid[285]: WARNING: 5 pending requests queued

The throughput is down to less than 10% of normal and even local (and
cached) web sites are taking extended periods to be delivered. I can't see
any obvious timeout in squid.conf that would improve the situation.

DNS lookups are (currently) OK for the unreachable sites since these have
been cached by named.

We force our users to go via a proxy by blocking port 80 access to the Net,
so passthrough is not an option.

Any suggestions greatly appreciated.

Thanks,
Simon.

-------------------------------------------------------------------------
Simon Rainey Direct Line: 01235 823238
Principal Internet Consultant Fax: 01235 823424
RM Internet for Learning E-mail: srainey@rmplc.net
New Mill House, 183 Milton Park, Abingdon, Oxfordshire, OX14 4SE, England
Received on Tue Apr 27 1999 - 05:39:18 MDT