Re: [squid-users] Optimized Squids

Amos Jeffries wrote:
> On Tue, 22 Jun 2010 16:05:04 -0500, Seann Clark
> <nombrandue_at_tsukinokage.net> wrote:
>
>> All,
>>
>> I have been playing with/tweaking/breaking my squid for a few months
>>
>
>
>> now, and I am looking for suggestions from the list on improving
>> performance. This is on a home system, which does not have a large user
>> base. I am running a Dual Xeon 2.0 Ghz system with 2 gigs ram, 120 Gigs
>> hard drive, in a Raid 5 configuration controlled by a 3ware RAID card.
>>
>
> Optimization Tip #1: get rid of the RAID-5.
> http://wiki.squid-cache.org/SquidFaq/RAID
>
>
I am planning on adding in a new disk to the unit, to move the cache off
the RAID volume, and keep the RAID volume just for the Operating system.
Would that have any impact on squid if its binary lives on the RAID-5
volume, but the caches are on separate physical drives?
>> I was using the stock Fedora 8 RPM for this, which was single threaded,
>> squid 2.6 Stable 22. I am also running this with diskd currently.
>>
>
> Optimization tip #2: run the fastest Squid release.
> Today 2.7 is current fastest official for production use.
> If you are willing to run semi-production code 2.HEAD daily snapshots
> have more speed experiments and some few high-performance testers behind
> it.
>
>
>> I have recently recompiled squid to the latest stable for version
>> 2.7 (Stable 9) with the async io flag passed to the configure command.
>> After a little updating of my configurations, just enough to get it to
>> work (I haven't changed any of the settings that are new to 2.7, so they
>>
>
>
>> are defaults right now), I have noticed a drastic improvement in speed
>> and even when the system is at a high load (3-5 on the system load,
>>
>
> So that covers #2. :)
>
>
> Optimization tip #3: use fastest disk IO method available.
> Fedora being a linux that would be AUFS.
>
> Specifically in 2.7, AUFS tuned via cache_dir directive for large objects
> (~1MB or more) alongside COSS directories for objects smaller than the AUFS
> holds.
> NP: these two caches types are happy to share one disk between them. An
> exception to the on-disk-per-cache_dir rule of thumb.
>
>
>
To get this a little clearer, as I am still learning what I can do with
the cache, I would configure the directives as such (example sizes):

cache_dir coss /var/spool/coss_cache 1024 1
cache_dir aufs /var/spool/bigcache 1024 100

>> reported by top) it runs fairly well. I am looking at moving my cache
>> directories off to a different disk, formatted with reiserFS, I am also
>> planning on running it using diskd as the storage scheme. The drive I am
>>
>
>
> see tip #3. AUFS makes use of multi-core threads, diskd process is
> single-threaded even if it runs additional to the Squid main thread.
>
> From my own experience, Squid has had no direct benefit from RieserFS over
> extN. The OS and other apps sometimes do gain, so it's up to you.
>
> You will need to tune the items which get journalled such as noatime etc
> to increase disk IO speeds either way.
>
>
>
>> using will be about 100 gigs of formatted space, and I plan to use all
>> that space for the cache.
>>
>
> Make sure you have enough RAM for the indexes. 2GB is borderline for your
> planned cache size.
> 4KB per object stored is what the index uses, with up to 2^31 objects per
> cache_dir = up to 8GB index. Reality varies with your users habits.
>
> NP: 2.7 in-memory handling of large objects is not great, so sacrificing
> any large cache_mem RAM cache for COSS small-object cache is probably a
> good trade-off in that release.
>
>
Expanded RAM in the system is planned, so it is a good point you
mentioned as it gives me a baseline to get for the system
>> I would like to know if this is a good plan, or should I change some
>>
>
>
>> things and how to change them, as well as any suggestions of
>> configuration settings with the cache, and 2.7 options that may make a
>> cache run even faster.
>>
>>
>
> On top of those basic parameters, your ACL configuration strongly
> influences speed. Read the wiki ACL pages on which are grouped as fast/slow
> (NP: 'fast' does not strictly mean speed there) to get the ordering
> optimal.
> Avoid any operations using regex like plague.
> Avoid helper operations as well since they often introduce large amounts
> of lag as well.
>
> Amos
>
I don't use regex in my configuration, and all my ACL's are set for
default access for snmp, cache access, and cachemanager access.

~Seann