[squid-users] Squid strangely using more memory than it should

Hi,
  I upgraded our two site caches from FreeBSD 3.5 and squid 2.3-STABLE4 to
FreeBSD 4.4-STABLE and squid 2.4-STABLE3 using diskd, about 2 weeks ago.
They'd been running fine for about 2 years with the FreeBSD and the squid
being periodically brought up to date.
  Config file options seem okay, but squid seems to run happily for a
while then suddenly start consuming lots of memory. Both machines have 1GB
RAM and the non-default config options for storage are:

cache_mem 256 MB
maximum_object_size 256 MB
cache_dir diskd /cache1 7300 16 256
cache_dir diskd /cache2 7300 16 256
cache_dir diskd /cache3 7300 16 256
cache_dir diskd /cache4 7300 16 256

  One machine ran for 10 days seemingly without a hitch. In that
time the process gradually got larger and larger as it's cache filled up.
It slowly crept up to the following sizes:

/dev/da1s1a 8822066 6827371 1288930 84% /cache1
/dev/da2s1a 8822066 6828831 1287470 84% /cache2
/dev/da3s1a 8822066 6826014 1290287 84% /cache3
/dev/da4s1a 8822066 6828091 1288210 84% /cache4

It's been at this amount of disk usage for a while now so I think it's
filled it's cache.
  Squid used 256MB of memory as I told it and hung around that mark:

$ client mgr:info | grep size
        Storage Swap size: 25577703 KB
        Storage Mem size: 262136 KB

However, now after running for a day or so at the 256MB mark it will
suddenly start to grow without limit. i.e. it started again about 30 mins
ago and is now at:

$ client mgr:info | grep size
        Storage Swap size: 26910658 KB
        Storage Mem size: 579148 KB

then a few minutes later:

$ client mgr:info | grep size
        Storage Swap size: 26906399 KB
        Storage Mem size: 600240 KB

It does this until it hits the resource limit and dies. During this
accelerated growth there are many, many client timeouts and cache response
is very poor.
  The other cache was upgraded second and has still to reach the above
disk usage. Maybe that will exhibit the same problem when it's cache has
been filled for a while.
  I just shutdown the cache before it consumed all memory and died. I saw
the error:

assertion failed: stmem.c:85: "current_offset == target_offset"

Instead of the usual 30 secs wait it took 5 mins before it started writing
out swap.state entries. Until it finally finished with:

CPU Usage: 13914.695 seconds = 9469.761 user + 4444.934 sys
Maximum Resident Size: 865888 KB
Page faults with physical i/o: 26774
2001/12/10 15:28:23| Open FD 17 squid -> diskd
2001/12/10 15:28:23| Open FD 20 squid -> diskd
2001/12/10 15:28:23| Open FD 23 squid -> diskd
2001/12/10 15:28:23| Open FD 26 squid -> diskd
2001/12/10 15:28:23| Squid Cache (Version 2.4.STABLE3): Exiting normally.

  I've checked the FAQ, etc. and my config, but can't see what's wrong.
It's beginning to seem like a bug to me.
  Any ideas what is causing this?
  Cheers.

Mark Powell - UNIX System Administrator - The University of Salford
Academic Information Services, Clifford Whitworth Building,
Salford University, Manchester, M5 4WT, UK.
Tel: +44 161 295 5936 Fax: +44 161 295 5888 www.pgp.com for PGP key
Received on Mon Dec 10 2001 - 08:36:23 MST