[squid-users] squid exceeding total system swap limit

From: atit jariwala <atit@dont-contact.us>
Date: Sat, 10 May 2003 15:18:09 +0530

below is my system config for squid 2.5 stable1

Intel(R) Pentium(R) 4 CPU 1.70GHz
MemTotal: 1029848 kB

12200 squid 15 0 685M 662M 1184 R 0 19.2 65.8 86:40 squid

mem_node 4104 56937 228193 228326 0.00 41

cpu util 20-30 %

snips of vmstat within span of 10 minutes

procs memory swap io system cpu

 r b w swpd free buff cache si so bi bo in cs us sy
id

2 0 1 43124 10816 4732 222268 0 0 32 0 711 190 13 15
72

 0 0 0 43124 10800 4740 222228 0 0 32 0 690 262 22 7
71

 1 0 0 49944 10744 4224 217552 0 0 26 50 653 169 12 7
81

 0 1 0 49944 10880 4332 216856 0 0 50 92 630 175 16 8
76

.......

 1 0 0 61680 16724 4868 207840 7 4 65 76 89 46 22 8
70
 0 0 0 61680 16684 4876 207872 0 0 32 128 737 246 21 10
69

.....
 0 0 0 67424 11072 5660 203308 7 4 65 76 90 46 22 8
70
 1 0 0 67424 11000 5664 203332 0 0 16 0 848 173 18 2
80
 0 0 0 67424 10932 5680 203384 0 0 40 0 752 192 16 9
75
 0 0 1 67424 10860 5680 203428 0 0 20 128 693 169 17 3
80

Cache Manager statistics

Squid Object Cache: Version 2.5.STABLE1

Start Time: Fri, 09 May 2003 13:17:38 GMT
Current Time: Sat, 10 May 2003 09:22:08 GMT

Connection information for squid:
 Number of clients accessing cache: 207
 Number of HTTP requests received: 620708
 Number of ICP messages received: 0
 Number of ICP messages sent: 0
 Number of queued ICP replies: 0
 Request failure ratio: 0.00%
 Average HTTP requests per minute since start: 515.3
 Average ICP messages per minute since start: 0.0
 Select loop called: 4308984 times, 16.772 ms avg
Cache information for squid:
 Request Hit Ratios: 5min: 30.2%, 60min: 36.1%
 Byte Hit Ratios: 5min: 23.7%, 60min: 28.6%
 Request Memory Hit Ratios: 5min: 5.7%, 60min: 5.8%
 Request Disk Hit Ratios: 5min: 41.0%, 60min: 39.3%
 Storage Swap size: 16999348 KB
 Storage Mem size: 231940 KB
 Mean Object Size: 13.15 KB
 Requests given to unlinkd: 6552
Median Service Times (seconds) 5 min 60 min:
 HTTP Requests (All): 0.02190 0.02069
 Cache Misses: 0.03622 0.02592
 Cache Hits: 0.02190 0.02069
 Near Hits: 0.80651 0.89858
 Not-Modified Replies: 0.01387 0.01235
 DNS Lookups: 0.01046 0.00949
 ICP Queries: 0.00000 0.00000
Resource usage for squid:
 UP Time: 72270.709 seconds
 CPU Time: 5379.180 seconds
 CPU Usage: 7.44%
 CPU Usage, 5 minute avg: 20.18%
 CPU Usage, 60 minute avg: 19.11%
 Maximum Resident Size: 0 KB
 Page faults with physical i/o: 314
Memory usage for squid via mallinfo():
 Total space in arena: 716877 KB
 Ordinary blocks: 613831 KB 80871 blks
 Small blocks: 0 KB 0 blks
 Holding blocks: 1016 KB 2 blks
 Free Small blocks: 0 KB
 Free Ordinary blocks: 103045 KB
 Total in use: 614847 KB 86%
 Total free: 103045 KB 14%
 Total size: 717893 KB
Memory accounted for:
 Total accounted: -1 KB
 memPoolAlloc calls: 73778524
 memPoolFree calls: 68747978
File descriptor usage for squid:
 Maximum number of file descriptors: 4096
 Largest file desc currently in use: 398
 Number of file desc currently in use: 287
 Files queued for open: 0
 Available number of file descriptors: 3809
 Reserved number of file descriptors: 100
 Store Disk files open: 15
Internal Data Structures:
 1294196 StoreEntries
  46653 StoreEntries with MemObjects
  46617 Hot Object Cache Items
 1292709 on-disk objects

---------------------------------------------------

This is at other machine having squid 2.4 stable1

Intel(R) Pentium(R) III CPU family 1266MHz
MemTotal: 1031548 kB

Cache Manager menu

Squid Object Cache: Version 2.4.STABLE1

Start Time: Mon, 05 May 2003 14:16:18 GMT
Current Time: Sat, 10 May 2003 09:19:40 GMT

Connection information for squid:
 Number of clients accessing cache: 679
 Number of HTTP requests received: 4753308
 Number of ICP messages received: 0
 Number of ICP messages sent: 0
 Number of queued ICP replies: 0
 Request failure ratio: 0.00%
 HTTP requests per minute: 688.5
 ICP messages per minute: 0.0
 Select loop called: 29809845 times, 13.895 ms avg
Cache information for squid:
 Request Hit Ratios: 5min: 51.9%, 60min: 53.1%
 Byte Hit Ratios: 5min: 26.1%, 60min: 30.3%
 Request Memory Hit Ratios: 5min: 10.9%, 60min: 14.3%
 Request Disk Hit Ratios: 5min: 33.1%, 60min: 35.4%
 Storage Swap size: 11059188 KB
 Storage Mem size: 262144 KB
 Mean Object Size: 12.20 KB
 Requests given to unlinkd: 320901
Median Service Times (seconds) 5 min 60 min:
 HTTP Requests (All): 0.10857 0.17711
 Cache Misses: 0.61549 0.61549
 Cache Hits: 0.01387 0.01387
 Near Hits: 0.61549 0.58309
 Not-Modified Replies: 0.00379 0.00562
 DNS Lookups: 0.00669 0.00669
 ICP Queries: 0.00000 0.00000
Resource usage for squid:
 UP Time: 414202.149 seconds
 CPU Time: 30576.660 seconds
 CPU Usage: 7.38%
 CPU Usage, 5 minute avg: 16.34%
 CPU Usage, 60 minute avg: 13.94%
 Maximum Resident Size: 0 KB
 Page faults with physical i/o: 792
Memory usage for squid via mallinfo():
 Total space in arena: 493415 KB
 Ordinary blocks: 406628 KB 35911 blks
 Small blocks: 0 KB 0 blks
 Holding blocks: 308 KB 2 blks
 Free Small blocks: 0 KB
 Free Ordinary blocks: 86787 KB
 Total in use: 406936 KB 82%
 Total free: 86787 KB 18%
Memory accounted for:
 Total accounted: 364891 KB
 memPoolAlloc calls: 545399468
 memPoolFree calls: 541715100
File descriptor usage for squid:
 Maximum number of file descriptors: 1024
 Largest file desc currently in use: 647
 Number of file desc currently in use: 552
 Files queued for open: 0
 Available number of file descriptors: 472
 Reserved number of file descriptors: 100
 Store Disk files open: 51
Internal Data Structures:
 907531 StoreEntries
  41477 StoreEntries with MemObjects
  41387 Hot Object Cache Items
 906327 on-disk objects
----------------------------------------------

as per documentaton 2.4 stable has pipeline prefetch on by default

and for squid 2.5 stable1 i have kept it is on for response time boosting

below if part of my squid.conf at 2.5 version

cache_dir ufs /var/spool/squid 20000 32 256
.......
cache_mem 256 MB
reload_into_ims on
half_closed_clients off
maximum_object_size 40960 KB
ipcache_size 8192
fqdncache_size 8192
refresh_pattern . 15 20% 10080
cache_replacement_policy LFUDA
store_avg_object_size 100 KB
store_objects_per_bucket 100
maximum_single_addr_tries 5

.....
pipeline_prefetch on
memory_pools off

problem i am facing with squid 2.5 is continous increase in swap memory
utilization
and at every 2 day squid exceed total swap limit and kernel has to kill the
squid process.

this is what i faced after making pipiline prefetch on in squid 2.5 and i
observed mem_node is going to take 42 % of total memory and it is going to
increse withing the time.

before pipeline prefetch on that no such problem was found.

but with squid 2.4 having pipeline prefetch on i am not finding this
problem....

can any one tell me what is reason behind it?

waiting for reply

-- atit
Received on Sat May 10 2003 - 03:48:37 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 17:16:31 MST