Re: [squid-users] squid exceeding total system swap limit

Hi Henrik,
    Sorry for my previous incomplete mail. After tweaking with
pipeline_prefetch option I found that this is not a problem with this TAG.
With or without "pipeline_prefetch" squid keep on growing beyond my all
Physical memory and Linux O.S kill that squid process due to (Out Of Memory)
reason.
    Firstly I'm giving you information about my development setup

[atit@cyberdev squid-2.5.STABLE1]$ ./src/squid -v
Squid Cache: Version 2.5.STABLE1
configure
options: --exec_prefix=/usr --bindir=/usr/sbin --libexecdir=/usr/lib/squid
--localstatedir=/var --sysconfdir=/etc/squid --datadir=/usr/lib/squid --enab
le-poll --enable-snmp --enable-removal-policies=heap,lru --enable-storeio=di
skd,ufs --enable-linux-netfilter --enable-gnuregex --with-pthreads --enable-
basic-auth-helpers=LDAP,PAM,MSNT --disable-ident-lookups --enable-underscore
s

 [atit@cyberdev squid-2.5.STABLE1]$ gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/egcs-2.91.66/specs
gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)

    After few days I thought to give a try to debug memory and I manually
added "Memory Debugging" options in include/autoconf.h
#define XMALLOC_TRACE 1
#define XMALLOC_DEBUG 1
#define XMALLOC_STATISTICS 1
    After recompiling and running squid with "-D -N -m" option I got
messages about memory leakage. You can find this debugging information from
the link
http://24online.elitecore.com/upgrades/squidxmalloc.log.gz

    After starting squid I given setting of proxy in my "Internet Explorer",
and in /etc/squid/allow_list, I added IP-Address of my workstation. Then I
opened site http://www.squid-cache.org and after finish of this request I
terminated Squid Cache by Pressing Ctrl+C. Above compressed file is output
after stopping Squid Cache.

[sumit@cyberdev squid-2.5.STABLE1]$ cat /etc/squid/squid.conf
acl all src 0.0.0.0/0.0.0.0

cache_peer 192.168.203.162 sibling 3128 3130
acl cache_peer1 src 192.168.203.162
http_access allow cache_peer1

cache_dir ufs /var/spool/squid 4000 16 256
log_mime_hdrs off
cache_store_log none
ftp_user anonymous@mydomain.com

#Defaults:
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl SSL_ports port 443 563
acl Safe_ports port 80 88 21 443 563 70 210 1025-65535
acl CONNECT method CONNECT

#Default configuration:
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

proxy_auth_realm Cyberoam proxy-caching web server

visible_hostname CACHE-Engine

httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
forwarded_for on
reload_into_ims on

snmp_access allow all

uri_whitespace allow

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# TAG: http_access

#IE tag to be added for squid version 2.4

acl allowlist src "/etc/squid/allow_list"

http_access allow allowlist

acl localmachine1 dst 192.168.203.204
http_access allow localmachine1

###################################################
# ACL(s) Entry Starts
###################################################

# ACL(s) for Security Policy

acl secpol_1 src "/etc/squid/securitypolicy/secpol_1"
acl secpol_2 src "/etc/squid/securitypolicy/secpol_2"
acl secpol_3 src "/etc/squid/securitypolicy/secpol_3"
acl secpol_4 src "/etc/squid/securitypolicy/secpol_4"
acl secpol_5 src "/etc/squid/securitypolicy/secpol_5"

# ACL(s) for WebCategory

acl webcat_1.id url_regex -i "/etc/squid/webcategory/webcat_1.id"
acl webcat_1.exl url_regex -i "/etc/squid/webcategory/webcat_1.exl"
acl webcat_2.id url_regex -i "/etc/squid/webcategory/webcat_2.id"
acl webcat_2.exl url_regex -i "/etc/squid/webcategory/webcat_2.exl"
acl webcat_3.id url_regex -i "/etc/squid/webcategory/webcat_3.id"
acl webcat_3.exl url_regex -i "/etc/squid/webcategory/webcat_3.exl"
acl webcat_4.id url_regex -i "/etc/squid/webcategory/webcat_4.id"
acl webcat_4.exl url_regex -i "/etc/squid/webcategory/webcat_4.exl"
acl webcat_5.id url_regex -i "/etc/squid/webcategory/webcat_5.id"
acl webcat_5.exl url_regex -i "/etc/squid/webcategory/webcat_5.exl"
acl webcat_6.id url_regex -i "/etc/squid/webcategory/webcat_6.id"
acl webcat_6.exl url_regex -i "/etc/squid/webcategory/webcat_6.exl"
acl webcat_7.id url_regex -i "/etc/squid/webcategory/webcat_7.id"
acl webcat_7.exl url_regex -i "/etc/squid/webcategory/webcat_7.exl"
acl webcat_8.id url_regex -i "/etc/squid/webcategory/webcat_8.id"
acl webcat_8.exl url_regex -i "/etc/squid/webcategory/webcat_8.exl"

# ACL(s) for Schedules

acl sch_1_1 time 00:00-23:59
acl sch_2_2 time D 10:00-18:00
acl sch_3_3 time DA 10:00-18:00
acl sch_4_4 time D 00:00-23:59
acl sch_5_5 time S 00:00-23:59
acl sch_5_6 time A 00:00-23:59
acl sch_6_7 time S 00:00-23:59
acl sch_7_8 time DA 15:00-17:00
acl sch_8_9 time 22:00-23:59
acl sch_8_10 time D 00:00-08:00
acl sch_8_11 time D 15:00-18:00

http_access deny all
miss_access allow all

err_html_text 192.168.203.204

> ----- Original Message -----
> From: "Henrik Nordstrom" <hno@squid-cache.org>
>
> > On Saturday 10 May 2003 11.48, atit jariwala wrote:
> >
> > > this is what i faced after making pipiline prefetch on in squid 2.5
> > > and i observed mem_node is going to take 42 % of total memory and
> > > it is going to increse withing the time.
> >
> > Sounds like there may be a bug in the pipeline prefetch function in
> > Squid-2.5.
> >
> > Please register a bug report, and include the folling information:
> >
> > * output of "squid -v"
> > * squid.conf settings
> > * cachemgr "General Runtime" (see below)
> > * cachemgr "Memory Utilization" (see below)
> >
> > Both cachemgr pages in three copies:
> > 1. shortly after start
> > 2. then after your Squid has been running for some time to allow for
> > cache_mem to be filled up. You are using a cache_mem of 256 MB which
> > will take some time to be used.
> > 3. and finally when Squid has been running for a longer time causing
> > shortage of memory, but before where it crashes.
Received on Thu May 22 2003 - 06:30:24 MDT