Re: [squid-users] Squid choking on large ACL lists--high CPU usage

url_regex has a scalability problem when the lists grows large, especially if
most are word matches and not fixed matches. For each pattern it has to do a
full match against every URL seen.

If you want to block whole sites it is better to use the dst_domain or dst
ACL types. These scale a whole lot better.

If you want to block sites with a certain word or pattern in their domain
name, it is better to use dstdom_regex. It has a much smaller search scope
than url_regex (only the host.domain part, not the whole URL).

Regards
Henrik Nordström
Squid Hacker

On Sunday 04 November 2001 05.03, Adam Maynard wrote:
> acl [block_sites,unblock_sites,direct] url_regex -i "textfile"
>
> Squid is being used here to filter web content. I wasn't around when it was
> set up & I'm not all that familiar with it yet. I think the block list has
> about 1000 entries. We tried to load a much larger list using the same
> format (about 40x bigger i think) & CPU usage went to max. USE_BIN_TREE
> reconfigure helped a little but not enough. Any ideas?
> Thanks,
> Adam Maynard
>
> ----- Original Message -----
> From: "Henrik Nordstrom" <hno@squid-cache.org>
> To: "Adam Maynard" <ml@cirrusnetworks.com>; <squid-users@squid-cache.org>
> Sent: Saturday, November 03, 2001 7:24 PM
> Subject: Re: [squid-users] Squid choking on large ACL lists--high CPU usage
>
> > What kind of ACL lists are you using?
> >
> > And how large?
> >
> > Regards
> > Henrik Nordström
> > Squid Hacker
> >
> > On Saturday 03 November 2001 23.36, Adam Maynard wrote:
> > > Anybody know why using a large acl list would push squid's cpu usage
> > > through the roof? I don't remember exact version # or config info. I
>
> know
>
> > > gnuregex is enabled & it's running on linux 2.4.5. Any general insight?
> > >
> > > AM
Received on Sun Nov 04 2001 - 04:45:15 MST