[squid-users] Excessive File descriptor usage on Squid 2.4

Hi All,
        I have been having problems with our main squid accelerator which front
ends a collection of web servers.

Basically I have been running out of file descriptors, this is on an
accelerator which is peeking at only around 90 to 100 requests per second.
The web servers that it front ends are all connected to a switched 100Mb
backbone and the web servers do not appear to be stressed in any way.

I know you can increase how many are available however I consider that (in
my case) to be curing the symptom not the cause. I have been running 2.4
STABLE 2 on a 2.4 Linux kernel for several months with this problem, which
would occasionally cause me to get paged when Netsaint (out network
monitoring program) was getting a connection blocked for too long.

Looking at the Process File descriptor Allocation in the cache manager I am
seeing huge numbers of connections stuck in a "Reading next request" state,
by huge numbers I mean in the 700 to 800 range!. Here is a section :-

 751 Socket 1 1770* 3914 128.8.71.30.35627 Reading next
request
 752 Socket 1 2094* 39739 24.156.173.164.1107 Reading next
request
 753 Socket 0 463* 228 200.38.237.51.1981 Reading next
request
 755 Socket 0 298* 6058 200.226.210.228.1418 Reading next
request
 759 Socket 0 314* 412 217.39.21.173.4540 Reading next
request
 761 Socket 0 608* 5471 194.239.162.246.1568 Reading next
request
 762 Socket 1 373* 251 148.202.51.100.4406 Reading next
request
 763 Socket 1 1062* 2816 128.8.71.30.35628 Reading next
request

Note the above was when the server was running squid 2.4 stable 2.

I had previously been running Squid 2.3 STABLE4 on a 2.2 Linux kernel and
did not see this problem. Before installing an older version of squid, I
updated our server to Squid 2.4 STABLE 4 to see if it had the same problem,
which it did.

I then downloaded and installed Squid 2.3 STABLE5 in parallel with the 2.4
squid so that I could switch between them. When running with Squid 2.3 I
don't see the problem at all!. Where as with 2.4 my used file descriptors
would go up to 700 odd in only a few minutes, with 2.3 it only gets up to
about 180 after several hours!.

I would like to return to running 2.4 if I can as I was using referrer
logging which is not in 2.3 :-(

I get the impression that this might be related to HTTP KeepAlive, at least
the status string "Reading next request" seems to suggest that. I don't
remember seeing that string in Squid 2.3, has KeepAlive behaviour changed
between 2.3 and 2.4?

I have not tried playing around with the request_timeout values, which are
still at their default of 30 seconds, I am not too keen to reduce them. This
coupled with the fact that I only really see this on a production server and
can't afford the disruption of trying lots of things out to see if they
help.

Either way, has some one else seen this problem, did you manage to solve it?
any help would be very much appreciated.

Thanks!

--
Simon Morley (simon@xaraonline.com)
Network Manager for Xara Online (http://www.xaraonline.com/)
DDI: 01442 351024
"Thou shalt not follow the Null Pointer, for at it's end Madness and Chaos
lie."