Re: [SQU] on multiprocessor machines

I have written up a Squid on Linux article that many folks have found
helpful in making Squid perform up to their needs...it's located here:

http://www.swelltech.com/pengies/joe/squidtuneup/t1.html

It only covers async i/o on Linux and Squid version 2.2STABLE5+hno
(still the reigning stability and speed champion among Squid's). When
2.4 goes stable I'll update it to take diskd into account, as well as
cover the new goodness that is async i/o.

As for the original idea mentioned, running multiple Squid processes
will require either: a separate object store for each Squid process
(giving poor hit ratio--unless balanced with an L4 switch or LVS with
the new LBLC or Thomas Proell's hash based scheduler), or a storetree
aware Squid, like the butterfly enabled Squid linked off of my homepage.
  However, butterfly is threaded already, and I doubt much would be
gained by running multiple processes. Then again, I don't have any MP
machines here to test the theory on...but the butterfly guys (Sizif and
Nikita) do, so I assume they would have said something if it made a big
difference.

As I mentioned briefly on the devel list, Moez and I are experimenting a
bit with running a 'hits-only' server on top of a Squid+butterfly to
serve and store misses. The hits-only server is based on Moez's quick
little Abyss HTTP web server. It still has some issues preventing us
from figuring out whether we gain anything on performance right now, but
we are working on it. This is merely research stuff at the moment, and
I wouldn't even think of trying to run it on a production box.

Squid is getting faster and more scalable...it's just taking it's time
getting there. ;-)

Robert Collins wrote:

> With async io or diskd squid will use multiple processors much more effectively. IIRC Joe Cooper has some info on this..
>
> Rob
> ----- Original Message -----
> From: "Chemolli Francesco (USI)" <ChemolliF@GruppoCredit.it>
> To: "'Rainer Ginsberg'" <rainer.ginsberg@de.bosch.com>; <squid-users@ircache.net>
> Sent: Tuesday, December 05, 2000 7:32 PM
> Subject: RE: [SQU] on multiprocessor machines
>
>
>
>>> Hi all,
>>>
>>> so Squid doesn't scale very well on multiprocessor hardware. It's one
>>> large process which can basically use only one CPU at a time.
>>>
>>> I was thinking about how to remedy this problem: My idea is to give
>>> the machine as many IP addresses as it has processes. Then start the
>>> same amount of Squid processes where each of them listens on one of
>>> the IP addresses. Then you could apply load balancing, either via DNS
>>> (cheap but not perfect) or with dedicated load-balancing appliances
>>> (very expensive).
>>>
>>> The problem with this approach is that the Squid processes might not
>>> "agree" on using different CPUs but all processes clogging to one or
>>> two CPUs and thus reducing performance.
>>>
>>> Has anybody else thought about a solution for Squid on multiprocessor
>>> systems?
>>
>> Thought - but not implemented:
>>
>> start as many squid processes as there are processors on different
>> ports, then use some clustering solution (like Linux Virtual Server)
>> to transparently redirect ports. All on the same system.

                                   --
                      Joe Cooper <joe@swelltech.com>
                  Affordable Web Caching Proxy Appliances
                         http://www.swelltech.com

--
To unsubscribe, see http://www.squid-cache.org/mailing-lists.html