Re: Increase caching time

From: Roger Peña Escobio <roger@dont-contact.us>
Date: Fri, 19 Mar 1999 13:32:42 -0500

>Yes I want images to be cached for 150 days and no matter what I dont want
>to expire. What I noticed was that the cache was filled 5 GB then the next
>day I have 1.5GB left ! All the objects simply disappeared. Would
>override-expire work on squid2.1-PATCH2.
the same happened to me, when I look in cache.log i find something like
this:

>>>>
1999/03/11 15:34:56| Squid Cache (Version 2.1.RELEASE): Exiting normally.
1999/03/11 15:35:08| Starting Squid Cache version 2.1.RELEASE for
sparc-unknown-linux-gnu...
1999/03/11 15:35:08| Process ID 2708
1999/03/11 15:35:08| With 256 file descriptors available
1999/03/11 15:35:08| Performing DNS Tests...
1999/03/11 15:35:08| Successful DNS name lookup tests...
1999/03/11 15:35:08| helperOpenServers: Starting 25 'dnsserver' processes
1999/03/11 15:35:11| Unlinkd pipe opened on FD 33
1999/03/11 15:35:11| Swap maxSize 2795520 KB, estimated 215040 objects
1999/03/11 15:35:11| Target number of buckets: 4300
1999/03/11 15:35:11| Using 8192 Store buckets, replacement runs every 10
seconds
1999/03/11 15:35:11| Max Mem size: 8192 KB
1999/03/11 15:35:11| Max Swap size: 2795520 KB
1999/03/11 15:35:11| Rebuilding storage in Cache Dir #0 (CLEAN)
1999/03/11 15:35:11| Rebuilding storage in Cache Dir #1 (CLEAN)
1999/03/11 15:35:11| Rebuilding storage in Cache Dir #2 (CLEAN)
1999/03/11 15:35:11| Loaded Icons.
1999/03/11 15:35:11| Accepting HTTP connections on port 3128, FD 60.
1999/03/11 15:35:11| commBind: Cannot bind socket FD 61 to *:1080: (48)
Address already in use
1999/03/11 15:35:11| Accepting ICP messages on port 3130, FD 61.
1999/03/11 15:35:11| Configuring Sibling 169.158.166.219/3128/3130
1999/03/11 15:35:11| Configuring Sibling 169.158.165.4/3128/3130
1999/03/11 15:35:11| Configuring Sibling 169.158.164.35/3128/3130
1999/03/11 15:35:11| Ready to serve requests.
1999/03/11 15:35:13| Starting Squid Cache version 2.1.RELEASE for
sparc-unknown-linux-gnu...
1999/03/11 15:35:13| Process ID 2737
1999/03/11 15:35:13| With 256 file descriptors available
1999/03/11 15:35:13| Performing DNS Tests...
1999/03/11 15:35:13| Successful DNS name lookup tests...
1999/03/11 15:35:13| helperOpenServers: Starting 25 'dnsserver' processes
1999/03/11 15:35:18| Unlinkd pipe opened on FD 33
1999/03/11 15:35:18| Swap maxSize 2795520 KB, estimated 215040 objects
1999/03/11 15:35:18| Target number of buckets: 4300
1999/03/11 15:35:18| Using 8192 Store buckets, replacement runs every 10
seconds
1999/03/11 15:35:18| Max Mem size: 8192 KB
1999/03/11 15:35:18| Max Swap size: 2795520 KB
1999/03/11 15:35:18| Rebuilding storage in Cache Dir #0 (DIRTY)
1999/03/11 15:35:18| Rebuilding storage in Cache Dir #1 (DIRTY)
1999/03/11 15:35:18| Rebuilding storage in Cache Dir #2 (DIRTY)
1999/03/11 15:35:18| Loaded Icons.
1999/03/11 15:35:18| commBind: Cannot bind socket FD 60 to *:3128: (48)
Address already in use
1999/03/11 15:35:18| commBind: Cannot bind socket FD 60 to *:1080: (48)
Address already in use
FATAL: Cannot open HTTP Port
Squid Cache (Version 2.1.RELEASE): Terminated abnormally.
CPU Usage: 1.710 seconds
Maximum Resident Size: 0 KB
Page faults with physical i/o: 297
Memory usage for squid via mallinfo():
 total space in arena: 2020 KB
 Ordinary blocks: 2013 KB 1 blks
 Small blocks: 0 KB 0 blks
 Holding blocks: 0 KB 0 blks
 Free Small blocks: 0 KB
 Free Ordinary blocks: 6 KB
 Total in use: 2013 KB 100%
 Total free: 6 KB 0%
1999/03/11 15:35:23| Starting Squid Cache version 2.1.RELEASE for
sparc-unknown-linux-gnu...
1999/03/11 15:35:23| Process ID 2764
1999/03/11 15:35:23| With 256 file descriptors available
1999/03/11 15:35:23| Performing DNS Tests...
1999/03/11 15:35:23| Successful DNS name lookup tests...
1999/03/11 15:35:23| helperOpenServers: Starting 25 'dnsserver' processes
1999/03/11 15:35:28| Unlinkd pipe opened on FD 33
1999/03/11 15:35:28| Swap maxSize 2795520 KB, estimated 215040 objects
1999/03/11 15:35:28| Target number of buckets: 4300
1999/03/11 15:35:28| Using 8192 Store buckets, replacement runs every 10
seconds
1999/03/11 15:35:28| Max Mem size: 8192 KB
1999/03/11 15:35:28| Max Swap size: 2795520 KB
1999/03/11 15:35:28| Rebuilding storage in Cache Dir #0 (DIRTY)
1999/03/11 15:35:28| Rebuilding storage in Cache Dir #1 (DIRTY)
1999/03/11 15:35:28| Rebuilding storage in Cache Dir #2 (DIRTY)
1999/03/11 15:35:28| Loaded Icons.
1999/03/11 15:35:28| commBind: Cannot bind socket FD 60 to *:3128: (48)
Address already in use
1999/03/11 15:35:28| commBind: Cannot bind socket FD 60 to *:1080: (48)
Address already in use
FATAL: Cannot open HTTP Port
Squid Cache (Version 2.1.RELEASE): Terminated abnormally.
CPU Usage: 1.670 seconds
Maximum Resident Size: 0 KB
Page faults with physical i/o: 295
Memory usage for squid via mallinfo():
 total space in arena: 2020 KB
 Ordinary blocks: 2013 KB 1 blks
 Small blocks: 0 KB 0 blks
 Holding blocks: 0 KB 0 blks
 Free Small blocks: 0 KB
 Free Ordinary blocks: 6 KB
 Total in use: 2013 KB 100%
 Total free: 6 KB 0%
1999/03/11 15:35:32| 65536 Entries read so far.
1999/03/11 15:35:33| Done reading Cache Dir #1 swaplog (23063 entries)
1999/03/11 15:35:33| Done reading Cache Dir #0 swaplog (23183 entries)
>>>>>
and the squid can't start until a few of seconds(in the middle we have
messages like the last one):
>>>>>
1999/03/11 15:36:11| Restarting Squid Cache (version 2.1.RELEASE)...
1999/03/11 15:36:11| FD 60 Closing HTTP connection
1999/03/11 15:36:11| FD 61 Closing ICP connection
1999/03/11 15:37:31| Starting Squid Cache version 2.1.RELEASE for
sparc-unknown-linux-gnu...
1999/03/11 15:37:31| Process ID 2877
1999/03/11 15:37:31| With 256 file descriptors available
1999/03/11 15:37:31| Performing DNS Tests...
1999/03/11 15:37:31| Successful DNS name lookup tests...
1999/03/11 15:37:31| helperOpenServers: Starting 25 'dnsserver' processes
1999/03/11 15:37:34| Unlinkd pipe opened on FD 33
1999/03/11 15:37:34| Swap maxSize 2795520 KB, estimated 215040 objects
1999/03/11 15:37:34| Target number of buckets: 4300
1999/03/11 15:37:34| Using 8192 Store buckets, replacement runs every 10
seconds
1999/03/11 15:37:34| Max Mem size: 8192 KB
1999/03/11 15:37:34| Max Swap size: 2795520 KB
1999/03/11 15:37:34| Rebuilding storage in Cache Dir #0 (DIRTY)
1999/03/11 15:37:34| Rebuilding storage in Cache Dir #1 (DIRTY)
1999/03/11 15:37:34| Rebuilding storage in Cache Dir #2 (DIRTY)
1999/03/11 15:37:34| Loaded Icons.
1999/03/11 15:37:34| Accepting HTTP connections on port 3128, FD 60.
1999/03/11 15:37:34| commBind: Cannot bind socket FD 61 to *:1080: (48)
Address already in use
1999/03/11 15:37:34| Accepting ICP messages on port 3130, FD 61.
1999/03/11 15:37:34| Configuring Sibling 169.158.166.219/3128/3130
1999/03/11 15:37:34| Configuring Sibling 169.158.165.4/3128/3130
1999/03/11 15:37:34| Configuring Sibling 169.158.164.35/3128/3130
1999/03/11 15:37:34| Ready to serve requests.
1999/03/11 15:37:38| Done reading Cache Dir #1 swaplog (6911 entries)
1999/03/11 15:37:38| Done reading Cache Dir #0 swaplog (7078 entries)
1999/03/11 15:37:38| Done reading Cache Dir #2 swaplog (7621 entries)
1999/03/11 15:37:38| Finished rebuilding storage disk.
1999/03/11 15:37:38| 21606 Entries read from previous logfile.
1999/03/11 15:37:38| 0 Entries scanned from swap files.
1999/03/11 15:37:38| 0 Invalid entries.
1999/03/11 15:37:38| 0 With invalid flags.
1999/03/11 15:37:38| 21603 Objects loaded.
1999/03/11 15:37:38| 0 Objects expired.
1999/03/11 15:37:38| 3 Objects cancelled.
1999/03/11 15:37:38| 1 Duplicate URLs purged.
1999/03/11 15:37:38| 0 Swapfile clashes avoided.
1999/03/11 15:37:38| Took 4 seconds (5400.8 objects/sec).
1999/03/11 15:37:38| Beginning Validation Procedure
1999/03/11 15:37:41| Completed Validation Procedure
1999/03/11 15:37:41| Validated 21602 Entries
1999/03/11 15:37:41| store_swap_size = 179689k
1999/03/11 15:39:05| Restarting Squid Cache (version 2.1.RELEASE)...
1999/03/11 15:39:05| FD 60 Closing HTTP connection
1999/03/11 15:39:05| FD 61 Closing ICP connection
1999/03/11 15:39:08| Starting Squid Cache version 2.1.RELEASE for
sparc-unknown-linux-gnu...
1999/03/11 15:39:08| Process ID 2906
>>>>>
but look that the number of object in cache dir go down from ~70000 to 21000
, ¿ any light about what happen here? one interesting things is the
restarting of cache so often .... is it normal?

i'll attach the entire cache.log for a close look of the problems

thanks in advance
Roger

Received on Fri Mar 19 1999 - 16:13:02 MST

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:45:21 MST