Re: Squid performance bombs when many sites are unreachable

Simon Rainey wrote:

> I tried increasing the number of dnsserver and redirector processes to 24
> each before we got Exodus connectivity back but there was only negligible
> improvement. As soon as Exodus came bake we were rolling again.

Redirectors should be unaffected unless your redirector does things
which depends on DNS lookups or similar network related things.

> What I'm concerned about is this. Suppose our transatlantic links go down
> (we're based in the UK) so we lose all US connectivity. Now maybe 80% of
> requests are for US sites. If Squid (or the dnsserver) is going to block
> because it can't reach those sites then it will eventually grind to a halt

Now, that is a problem. Squid is an application gateway, and as such it
is very sensitive to network outage. The main problem is that the number
of pending TCP connections quickly build up when a major host is
unreachable. There are a couple of things you may do to limit the
impact:

* Turn off the support for half closed clients ("half_closed_clients
off"). When this is done Squid immediately aborts the connection when
the client aborts. While half closed client connections are enabled it
is impossible (a TCP restriction) to abort a connection until there is
data to send to the client, and when a host is unreachable then no data
is sent until the server connection times out (which usually is much
later than the client gets tired and moves on doing something else).
Very few (if any) browsers depends on half closed connections (I think
early versions of a browser from IBM needed it, and command pipes using
telnet also requires it).

* Lower the connect timeout.

* Set maximum_single_addr_tries to 1. Nowdays (Squid 2.2) this directive
only adds to the delay seen when the host is unreachable.

* Have a large margin on the number of dnsservers and TCP connections
the cache server can support.

--
Henrik Nordstrom
Spare time Squid hacker