Re: [squid-users] Re: FD leak

Replying to my own post.

I originally thought the leak was a result of overloading, but I'm not
getting those errors on this squid and it's still leaking. Badly.

Take this guy:
lr-x------ 1 root root 64 Aug 21 21:33 184 ->
/var/squid2/00/6E/00006E4F (deleted)

Yes, it really is deleted. store_log even says so:
998438698.197 RELEASE 01 00006E4F 0CDD11B03A059B9BE951FB4375E0A8A6 200
998438668 990724035 -1 application/zip 2472733/128712 GET
http://site/zips/m37b15b.zip
998438713.000 SWAPOUT 01 00006E4F 14003310C8016707FEBEFA142203FE16 200
998438671 990724035 -1 application/zip 2472733/2472733 GET
http://site/zips/m37b15b.zip
998440834.351 RELEASE 01 00006E4F 44BA95077C4BAA2E9ECDCD18379CBF81 200
998439027 990724035 -1 application/zip 2472733/2472733 GET
http://site/zips/m37b15b.zip

Some are REALLY bad:
lr-x------ 1 root root 64 Aug 21 21:43 167 ->
/var/squid2/00/6A/00006AE0
lr-x------ 1 root root 64 Aug 21 21:43 221 ->
/var/squid2/00/6A/00006AE0
lr-x------ 1 root root 64 Aug 21 21:43 235 ->
/var/squid2/00/6A/00006AE0
lr-x------ 1 root root 64 Aug 21 21:43 287 ->
/var/squid2/00/6A/00006AE0
lr-x------ 1 root root 64 Aug 21 21:43 377 ->
/var/squid2/00/6A/00006AE0
lr-x------ 1 root root 64 Aug 21 21:43 384 ->
/var/squid2/00/6A/00006AE0
lr-x------ 1 root root 64 Aug 21 21:43 614 ->
/var/squid2/00/6A/00006AE0
lr-x------ 1 root root 64 Aug 21 21:43 645 ->
/var/squid2/00/6A/00006AE0

This particular one is interesting because it has only been swapped out
once (never released). It also introduces another trend that seems to
hold: all of the leaked descriptors are open for reads. Writes do not
leak.

The problem actually seems worse than it was when I first reported it.
This squid has been up for 4 hours and has probably leaked 100 descriptors
or so.

        -- Brian

On Tuesday 21 August 2001 06:18 pm, you wrote:
> It took awhile to get a new version in production.
>
> File descriptor usage for squid:
> Maximum number of file descriptors: 4096
> Largest file desc currently in use: 1390
> Number of file desc currently in use: 978
> Files queued for open: 0
> Available number of file descriptors: 3118
> Reserved number of file descriptors: 100
> Store Disk files open: 916
>
> This is squid-2.4-200108182300. I'd say it still leaks.
>
> This particular system is Debian woody with a 9gig and an 18gig disk,
> and it serves as our multimedia and image parent cache.
>
> Squid was compiled on Debian 2.2 (potato) w/gcc 2.95.2 using
> ./configure --disable-ident-lookups --disable-unlinkd \
> --with-aio-threads=20 --with-pthreads --enable-storeio='ufs,aufs'\
> --enable-removal-policies='lru,heap'
>
> The problem does not occur with diskd.
>
> From here, I'm kind of lost. I'm not sure how to proceed with hunting
> this problem down. Any suggestions?
>
> -- Brian
>
> On Wednesday 25 July 2001 02:01 am, Adrian Chadd wrote:
> > On Tue, Jul 24, 2001, Brian wrote:
> > > The error means the IO queue has built up to an unusual level. With
> > > hard drives, the drive can fall behind and leave all of the threads
> > > blocked for too long. In a case like this, a flash of load is
> > > probably building up the queue for a moment. More threads would
> > > allow more requests in progress.
> > >
> > > Our squid httpd-accels produced several congestion and overloading
> > > messages an hour without harm (well... actually it seems to leak
> > > file descriptors around the overloading point). I finally tweaked
> > > it so requests wouldn't be saved to disk until squid got at least 2
> > > requests for it.
> >
> > Leaky fds? Can you reproduce it with the latest squid-2.4 snapshot?
> > that might make it easier for us squid hackers to track down any bug
> > and squish it.
Received on Tue Aug 21 2001 - 19:55:23 MDT