Re: [squid-users] how many hits can squid can handle

Robin Stevens wrote:

> On Wed, Sep 12, 2001 at 06:25:06PM -0500, Joe Cooper wrote:
>
>>I'm afraid Squid doesn't scale quite that well. I just shipped a unit > specced as follows:
>>
>>Dual PIII 1GHz
>>1.5GB RAM
>>3 x 18GB 15k RPM disks
>>
>
> Is there much to be gained by going for 15krpm disks over 10krpm?
> A few months ago, the opinion of my colleagues was that the extra money
> would be better spent on more RAM or more disk spindles.

Interesting you should bring that up. The reason we shipped a 15K based
system is because one of our suppliers was offering them at only about
$10 more per disk than our usual 10k IBM's (which are real screamers for
Squid in their own right). So we passed that price on in a quote to a
client, and they went for it. Great results too. I don't think we're
seeing significantly higher throughput than from the 10k disks (maybe a
little bit), but the response time during mid-load and high-load periods
is astonishingly fast (lower overall latency than some of our IDE
systems at any load--but the large RAM capacity probably plays a part
too). I think we're still hitting some CPU bottlenecks in Squid when
you get into these kinds of req rates...so the disks can only do so
much. But it's very nice and definitely the fastest Squid box we've
shipped (not by much though...we've shipped several similar systems with
single PIII/1GHz and 2 or 3 10k disks).

I think our supplier has now upped the price on the 15k disks a bit, but
even so, they are less than $80 more than the 10k disks of the same
capacity. I expect the 15k based boxes will prove pretty popular.

>>So I'd say even a 1.4GHz Athlon would max at 250 reqs/sec. Though a
>>dual Athlon 1.4 might run up to 350 or more.
>>
>
> I have seen a single PIII-1000 running at 300 reqs/sec, albeit with an
> empty disk cache. Performance seemed to drop quite sharply by the time the
> cache disks filled, although upgrading to squid 2.4 and using aufs has
> restored some of that performance. I'm still waiting for termtime to push
> these boxes to the limit, and then to make a case for buying a couple more
> to replace our previous generation of cache machines entirely :-)

Sure. Empty Squids are blazing fast! It's when they've been filled a
few times over that they start to get a bit halting. ReiserFS seems to
be getting better at preventing the slowdown (the impact of
fragmentation was once worse, relatively speaking, on Reiser than ext2,
but it seems improved of late). Also keep in mind that my definition of
'acceptable' performance is probably more stringent than most folks. If
my hit times climb over the 0.04 second mark for any extended length of
time, I begin to worry...I like it to hover around 0.02 or below most of
the time. As a commercial vendor supporting ISPs, my clients expect
very fast performance /and/ bandwidth savings. Just one or the other
won't do.

But really...I suspect some of the other IO bottlenecks will start to
rear their head, even with more CPU horsepower. But it's fun to push it
a little. And I do enjoy seeing a client load of over 3500 unique hosts
on a box I built. (But it hasn't been easy--I'm still fighting with the
kernel 2.4 on this box, and having some issues with getting a dual Squid
system to be as simple to manage and monitor as a single Squid
machine--I've had to rewrite all of the management tools we use to
support multiple processes.)
                                   --
                      Joe Cooper <joe@swelltech.com>
                  Affordable Web Caching Proxy Appliances
                         http://www.swelltech.com
Received on Tue Sep 18 2001 - 00:54:34 MDT