Re: [squid-users] Slow memory leak

On 9/26/2012 9:09 PM, tcr_at_raynersw.com wrote:
> Hi Eliezer,
>
> Thanks for the feedback. This memory leak causes real-world problems for me, as squid starts to do a lot of swapping when it exhausts physical RAM, and things slow down. Also, the sheer magnitude of the memory numbers is just ridiculous... squid easily grows to in excess of 10GB of resident memory as shown in top. I've got the servers restarting their squids periodically, but that's a pretty bad solution.
>
> Here is my squid.conf. Note two include files... squid_ns5_allowed_ips.conf and squid_blacklist_ips.conf . These are lists of IPs in an ACL. The allowed IPs one has lots of entries (almost 20,000) and that's the only thing I think is really unusual about my setup, so I'm wondering if that is exposing a leak somehwere.

So no cache_dir at all? just plain memory?

what the include files:
include /etc/squid/squid_ns5_allowed_ips.conf
include /etc/squid/squid_blacklist_ips.conf

contains?what lines?
you can have only one acl named:
acl src blacklist_ip "/etc/squid/bl.txt"
and use one line of
http_access deny blacklist_ip

but this amount of IP list such as 20k is a bad idea in any case there
is in squid.
it's not as much as squidguard blacklists which can contain 1 million
domains\ips but it seems like really bad idea to use this kind of BL in
squid config file.

What version of squid is this one?
it's seems to me like an old version of squid since it uses (squid) in
the ps output which in 3.2.1 will show (squid-1).

what are these black and allowed ips?
for clients? for destinations?
it matters a lot..

and may i ask why so many http_port? in your squid.conf output there is
nothing that gives in any way the need for such huge amount of http_port.

since you do have the list of IP's I would suggest to try first use
firewall based rules to allow or deny IPs instead of squid src or dst
acls just to take of the load of squid(this can make somethings slow but
will provide you the data about squid 20k acls that you are using)

If you are familiar with iptables one table or two additional tables
will give you what you need.

The next thing to try is external_acl or squidguard.

If you will have more data on the included files structure and acls It
will help a lot.

Regards,
Eliezer

-- 
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer <at> ngtech.co.il