Re: [squid-users] running out of filedescriptors

Landy Landy wrote:
> I restarted squid and it comes up ok. But I still get:
>
> With 1024 file descriptors available
>
> 2010/01/26 10:27:43| Starting Squid Cache version 3.0.STABLE20 for i686-pc-linux-gnu...
> 2010/01/26 10:27:43| Process ID 2768
> 2010/01/26 10:27:43| With 1024 file descriptors available
> 2010/01/26 10:27:43| DNS Socket created at 0.0.0.0, port 32808, FD 7
> 2010/01/26 10:27:43| Adding nameserver 196.3.81.5 from squid.conf
> 2010/01/26 10:27:43| Adding nameserver 200.88.127.22 from squid.conf
> 2010/01/26 10:27:43| Adding nameserver 196.3.81.132 from squid.conf
> 2010/01/26 10:27:43| helperOpenServers: Starting 10/10 'python' processes
> 2010/01/26 10:27:44| Unlinkd pipe opened on FD 22
> 2010/01/26 10:27:44| Swap maxSize 40960000 + 524288 KB, estimated 864256 objects
> 2010/01/26 10:27:44| Target number of buckets: 43212
> 2010/01/26 10:27:44| Using 65536 Store buckets
> 2010/01/26 10:27:44| Max Mem size: 524288 KB
> 2010/01/26 10:27:44| Max Swap size: 40960000 KB
> 2010/01/26 10:27:45| Version 1 of swap file with LFS support detected...
> 2010/01/26 10:27:45| Rebuilding storage in /var/log/squid3.1/cache (DIRTY)
> 2010/01/26 10:27:45| Using Round Robin store dir selection
> 2010/01/26 10:27:45| Current Directory is /
> 2010/01/26 10:27:45| Loaded Icons.
> 2010/01/26 10:27:45| Accepting transparently proxied HTTP connections at 172.16.0.1, port 3128, FD 24.
> 2010/01/26 10:27:45| HTCP Disabled.
> 2010/01/26 10:27:45| Ready to serve requests.
> 2010/01/26 10:27:45| Store rebuilding is 0.35% complete
> 2010/01/26 10:28:00| Store rebuilding is 85.83% complete
> 2010/01/26 10:28:04| Done reading /var/log/squid3.1/cache swaplog (1173452 entries)
> 2010/01/26 10:28:04| Finished rebuilding storage from disk.
> 2010/01/26 10:28:04| 969488 Entries scanned
> 2010/01/26 10:28:04| 0 Invalid entries.
> 2010/01/26 10:28:04| 0 With invalid flags.
> 2010/01/26 10:28:04| 766226 Objects loaded.
> 2010/01/26 10:28:04| 0 Objects expired.
> 2010/01/26 10:28:04| 203262 Objects cancelled.
> 2010/01/26 10:28:04| 0 Duplicate URLs purged.
> 2010/01/26 10:28:04| 0 Swapfile clashes avoided.
> 2010/01/26 10:28:04| Took 19.63 seconds (39024.87 objects/sec).
> 2010/01/26 10:28:04| Beginning Validation Procedure
> 2010/01/26 10:28:05| 524288 Entries Validated so far.
> 2010/01/26 10:28:06| 1048576 Entries Validated so far.
> 2010/01/26 10:28:06| 1310720 Entries Validated so far.
> 2010/01/26 10:28:06| Completed Validation Procedure
> 2010/01/26 10:28:06| Validated 1532463 Entries
> 2010/01/26 10:28:06| store_swap_size = 32693644
> 2010/01/26 10:28:06| storeLateRelease: released 0 objects
> 2010/01/26 10:28:59| Killing RunCache, pid 2766
> 2010/01/26 10:28:59| Preparing for shutdown after 0 requests
> 2010/01/26 10:28:59| Waiting 30 seconds for active connections to finish
> 2010/01/26 10:28:59| FD 24 Closing HTTP connection
> 2010/01/26 10:40:59| Starting Squid Cache version 3.0.STABLE20 for i686-pc-linux-gnu...
> 2010/01/26 10:40:59| Process ID 3549
> 2010/01/26 10:40:59| With 1024 file descriptors available
> 2010/01/26 10:40:59| Performing DNS Tests...
> 2010/01/26 10:40:59| Successful DNS name lookup tests...
> 2010/01/26 10:40:59| DNS Socket created at 0.0.0.0, port 32822, FD 7
> 2010/01/26 10:40:59| Adding nameserver 196.3.81.5 from squid.conf
> 2010/01/26 10:40:59| Adding nameserver 200.88.127.22 from squid.conf
> 2010/01/26 10:40:59| Adding nameserver 196.3.81.132 from squid.conf
> 2010/01/26 10:40:59| helperOpenServers: Starting 10/10 'python' processes
> 2010/01/26 10:41:00| Unlinkd pipe opened on FD 22
> 2010/01/26 10:41:00| Swap maxSize 40960000 + 786432 KB, estimated 869717 objects
> 2010/01/26 10:41:00| Target number of buckets: 43485
> 2010/01/26 10:41:00| Using 65536 Store buckets
> 2010/01/26 10:41:00| Max Mem size: 786432 KB
> 2010/01/26 10:41:00| Max Swap size: 40960000 KB
> 2010/01/26 10:41:00| Version 1 of swap file with LFS support detected...
> 2010/01/26 10:41:00| Rebuilding storage in /var/log/squid3.1/cache (DIRTY)
> 2010/01/26 10:41:00| Using Round Robin store dir selection
> 2010/01/26 10:41:00| Current Directory is /root
> 2010/01/26 10:41:00| Loaded Icons.
> 2010/01/26 10:41:00| Accepting transparently proxied HTTP connections at 172.16.0.1, port 3128, FD 24.
> 2010/01/26 10:41:00| HTCP Disabled.
> 2010/01/26 10:41:00| Ready to serve requests.
> 2010/01/26 10:41:01| WARNING: redirector #1 (FD 8) exited
> 2010/01/26 10:41:01| Store rebuilding is 0.35% complete
> 2010/01/26 10:41:14| Done reading /var/log/squid3.1/cache swaplog (1172750 entries)
> 2010/01/26 10:41:14| Finished rebuilding storage from disk.
> 2010/01/26 10:41:14| 969488 Entries scanned
> 2010/01/26 10:41:14| 0 Invalid entries.
> 2010/01/26 10:41:14| 0 With invalid flags.
> 2010/01/26 10:41:14| 766226 Objects loaded.
> 2010/01/26 10:41:14| 0 Objects expired.
> 2010/01/26 10:41:14| 203262 Objects cancelled.
> 2010/01/26 10:41:14| 0 Duplicate URLs purged.
> 2010/01/26 10:41:14| 0 Swapfile clashes avoided.
> 2010/01/26 10:41:14| Took 13.59 seconds (56399.78 objects/sec).
> 2010/01/26 10:41:14| Beginning Validation Procedure
> 2010/01/26 10:41:14| 524288 Entries Validated so far.
> 2010/01/26 10:41:14| 1048576 Entries Validated so far.
> 2010/01/26 10:41:14| 1310720 Entries Validated so far.
> 2010/01/26 10:41:14| Completed Validation Procedure
> 2010/01/26 10:41:14| Validated 1532463 Entries
> 2010/01/26 10:41:14| store_swap_size = 32693644
> 2010/01/26 10:41:14| storeLateRelease: released 0 objects
> 2010/01/26 10:41:54| Squid is already running! Process ID 3549
> 2010/01/26 10:59:47| Squid is already running! Process ID 3549
> 2010/01/26 11:00:06| Squid is already running! Process ID 3549
> 2010/01/26 11:37:07| Killing RunCache, pid 3547
> 2010/01/26 11:37:07| Preparing for shutdown after 0 requests
> 2010/01/26 11:37:07| Waiting 30 seconds for active connections to finish
> 2010/01/26 11:37:07| FD 24 Closing HTTP connection
> 2010/01/26 11:37:07| WARNING: redirector #2 (FD 9) exited
> 2010/01/26 11:37:07| WARNING: redirector #4 (FD 11) exited
> 2010/01/26 11:37:07| WARNING: redirector #5 (FD 12) exited
> 2010/01/26 11:37:07| WARNING: redirector #6 (FD 13) exited
> 2010/01/26 11:37:07| WARNING: redirector #7 (FD 14) exited
> 2010/01/26 11:37:07| Too few redirector processes are running
> 2010/01/26 11:37:07| Starting new helpers
> 2010/01/26 11:37:07| helperOpenServers: Starting 6/10 'python' processes
> 2010/01/26 11:37:07| WARNING: redirector #3 (FD 10) exited
> 2010/01/26 11:37:07| WARNING: redirector #10 (FD 17) exited
> 2010/01/26 11:37:07| WARNING: redirector #9 (FD 16) exited
> 2010/01/26 11:37:07| WARNING: redirector #8 (FD 15) exited
> 2010/01/26 11:40:41| Starting Squid Cache version 3.0.STABLE20 for i686-pc-linux-gnu...
> 2010/01/26 11:40:41| Process ID 2842
> 2010/01/26 11:40:41| With 1024 file descriptors available
> 2010/01/26 11:40:41| Performing DNS Tests...
> 2010/01/26 11:40:42| Successful DNS name lookup tests...
> 2010/01/26 11:40:42| DNS Socket created at 0.0.0.0, port 32779, FD 7
> 2010/01/26 11:40:42| Adding nameserver 196.3.81.5 from squid.conf
> 2010/01/26 11:40:42| Adding nameserver 200.88.127.22 from squid.conf
> 2010/01/26 11:40:42| Adding nameserver 196.3.81.132 from squid.conf
> 2010/01/26 11:40:42| helperOpenServers: Starting 10/10 'python' processes
> 2010/01/26 11:40:42| Unlinkd pipe opened on FD 22
> 2010/01/26 11:40:42| Swap maxSize 40960000 + 786432 KB, estimated 869717 objects
> 2010/01/26 11:40:42| Target number of buckets: 43485
> 2010/01/26 11:40:42| Using 65536 Store buckets
> 2010/01/26 11:40:42| Max Mem size: 786432 KB
> 2010/01/26 11:40:42| Max Swap size: 40960000 KB
> 2010/01/26 11:40:43| Version 1 of swap file with LFS support detected...
> 2010/01/26 11:40:43| Rebuilding storage in /var/log/squid3.1/cache (DIRTY)
> 2010/01/26 11:40:43| Using Round Robin store dir selection
> 2010/01/26 11:40:43| Current Directory is /
> 2010/01/26 11:40:43| Loaded Icons.
> 2010/01/26 11:40:43| Accepting transparently proxied HTTP connections at 172.16.0.1, port 3128, FD 24.
> 2010/01/26 11:40:43| HTCP Disabled.
> 2010/01/26 11:40:43| Ready to serve requests.
> 2010/01/26 11:40:44| Store rebuilding is 0.35% complete
> 2010/01/26 11:40:59| Store rebuilding is 88.37% complete
> 2010/01/26 11:41:02| Done reading /var/log/squid3.1/cache swaplog (1172750 entries)
> 2010/01/26 11:41:02| Finished rebuilding storage from disk.
> 2010/01/26 11:41:02| 969488 Entries scanned
> 2010/01/26 11:41:02| 0 Invalid entries.
> 2010/01/26 11:41:02| 0 With invalid flags.
> 2010/01/26 11:41:02| 766226 Objects loaded.
> 2010/01/26 11:41:02| 0 Objects expired.
> 2010/01/26 11:41:02| 203262 Objects cancelled.
> 2010/01/26 11:41:02| 0 Duplicate URLs purged.
> 2010/01/26 11:41:02| 0 Swapfile clashes avoided.
> 2010/01/26 11:41:02| Took 18.63 seconds (41137.06 objects/sec).
> 2010/01/26 11:41:02| Beginning Validation Procedure
> 2010/01/26 11:41:02| 524288 Entries Validated so far.
> 2010/01/26 11:41:03| 1048576 Entries Validated so far.
> 2010/01/26 11:41:03| 1310720 Entries Validated so far.
> 2010/01/26 11:41:03| Completed Validation Procedure
> 2010/01/26 11:41:03| Validated 1532463 Entries
> 2010/01/26 11:41:03| store_swap_size = 32693644
> 2010/01/26 11:41:03| storeLateRelease: released 0 objects
>
>
>
>

Unless I'm totally missing it there isn't anything in this particular
log that indicates a shortage of file descriptors. I do notice your
redirectors are exiting though. I'd suggest running squid without the
redirector just to see if the cache itself will stay up and process
traffic that way. If it does it implies the problem is with the redirector.

If you actually *are* running out of file descriptors and it just isn't
identified in this particular log, and if you can't do anything with
ulimit and recompiling squid to use more FD's as others have suggested,
you can try setting persistent connections to off as that will reduce
the FD pressure on squid.

server_persistent_connections off
client_persistent_connections off

HTH.

-- 
Mike Rambo
NOTE: In order to control energy costs the light at the end
of the tunnel has been shut off until further notice...