Re: [squid-users] H/W requirement for Squid to run in bigger scene like ISP

While more/faster is always better, that's not what I'm arguing - I'll
take the fastest CPU budget allows and as many cores as they'll let me
buy :) In particular, more CPU *does* help when you have dozens of
helper apps such as authenticators or url rewriters.

I think the point of the argument is that when spec'ing hardware to
run squid as the primary application, if you're faced with the need to
tradeoff on cpu, memory, or disk i/o capabilities, you'll get *far*
more Bang For The Buck by adding RAM and disk spindles - 2x500GB is
better than 1x1TB - than you would by putting more CPU cores in your
server instead.

-C

On Jul 15, 2008, at 4:51 AM, Michel wrote:

>> Hi,
>>
>> One thing to keep in mind is that in my experience, it makes sense to
>> not only get fast disks, but put as much RAM in the box you can
>> afford. Now *don't* give this all the squid via the mem_cache config;
>> let the OS use the spare memory for caching disk reads. This will
>> spee
>>
>> Additionally, don't RAID your disks beyond RAID 1, and only do that
>> if
>> you have to for reliability requirements. The more individual
>> spindles
>> attached to separate cache_dirs, the better. Amos is right that I/O
>> trumps CPU here every time.
>>
>> When we swapped out older squid boxes that couldn't take more than
>> 2GB
>> of RAM, or more than one disk, and put in 64-bit boxen with 32GB
>> and 3
>> cache-dirs (6 drives, paired into three RAID1 devices), we saw things
>> improve dramatically despite the fact that the CPUs were actually
>> slower. We went from topping out at 5K queries per minute to being
>> able to handle ~20K/minute without breaking a sweat. Pretty dramatic
>> IMHO.
>>
>> Hope this helps,
>>
>> -Chris
>>
>> On Jul 14, 2008, at 10:04 AM, Amos Jeffries wrote:
>>
>>> Anna Jonna Armannsdottir wrote:
>>>> On mán, 2008-07-14 at 13:01 +0200, Angelo Hongens wrote:
>>>>> All the servers I usually buy have either one or two quad core
>>>>> cpu's,
>>>>> so it's more the question: will 8 cores perform better than 4?
>>>>>
>>>>> If not, I would be wiser to buy a single Xeon X5460 or so, instead
>>>>> of
>>>>> 2 lower clocked cpu's, right?
>>>> In that case we are fine tuning the CPU power and if there are 8
>>>> cores in a Squid server, I would think that at least half of them
>>>> would
>>>> produce idle heat: An extra load for the cooling system. As You
>>>> point
>>>> out, the CPU speed is probably important for the part of Squid that
>>>> does
>>>> not use threading or separate process. But the real bottlenecks are
>>>> in the I/O, both RAM and DISK. So if I was buying HW now, I would
>>>> mostly be looking at I/O speed and very little at
>>>> CPU speed. SCSI disks with large buffers are preferable, and if
>>>> SCSI is not a viable choice, use the fastest SATA disks you can
>>>> find - Western
>>>> Digital Raptor used to be the fastest SATA disk, dot't know what is
>>>> the
>>>> fastest SATA disk now.
>>>
>>> This whole issue comes up every few weeks.
>>>
>>> The last consensus reached was dual-core on a squid dedicated
>>> machine. One for squid, one for everything else. With a few GB of
>>> RAM and fast SATA drives. aufs for Linux. diskd for BSD variants.
>>> With many spindles preferred over large disk space (2x 100GB instead
>>> of 1x 200GB).
>>>
>>> The old rule-of-thumb memory usage mentioned earlier (10MB/GB +
>>> something for 64-buts) still holds true. The more available the
>>> larger the in-memory cache can be, and that is still where squid
>>> gets its best cache speeds on general web traffic.
>>>
>>> Exact tunings are budget dependent.
>>>
>
> and the whole issue again and again is understated, at least you
> guys admit
> already
> dualcores ...
>
> I really do not understand the resistance, there *_IS_NO_* doubth
> that an 8core
> machine is faster than a twocore - and whatever software you put in
> there it
> *_IS_*
> faster, unless it's idle what eventually is the problem at all, are
> your
> machines
> idle so you don't see it ????
>
> of course the budget IS a point (8core is expensive) but a AM2
> quadcore is
> absolutely cheap today so there is again NO doubth what kind of CPU
> to buy,
> especially squid takes advantage of quads, especially of AMD quads
> what I
> believe
> cause of it's NUMA arquitecture
>
> in order to show it once I attach 3 images of Average load
> comparism, and
> please,
> before flaming this up try to understand what Unix load average
> really means and
> say, I graph it directly to the available cores so you can see when
> spikes
> are high
> and continuously near or above CPU than the machine can be seen as
> busy, so when
> spikes are low machine is free to breath (less IO wait-state) and to
> serve of
> course
>
> so then, less your load average more response time you get, and as I
> ever
> said: You
> can feel it and that is what the connected clients spell as "fast"
>
> so you can see a AM2 X2, a AM2 X4 and a Dual opteron dualcore
> machine, both AM2
> sockets are with constant 6-8MB through-going traffic, the opteron
> serves
> 16MB, all
> three have 3 SCSI-U320 250G disks on ZFS and a 16G for the OS, both
> AM2 are
> with 4G
> and the opteron with 16G of RAM, each of them with 5 ETHs with 4
> internal
> subnets
> and one external, transparent proxy running three squid instances
> and diskd,
> firewall and some scripts collecting stats and making rrdtool
> images, the AMS
> serve
> about 450 and the opteron 800 clients at peak hours, average of
> 250kbit/s of
> download limit each and not to forget, this is freebsd 7-stable amd64
>
> so my suggestion is get yourself an as-much-cores you can get and
> enjoy
>
> michel
>
> I send two more msgs with the next images attached, this is the 2x
> dualcore opteron
>
>
>
>
> ****************************************************
> Tecnologia Internet Matik http://info.matik.com.br
> Sistemas Wireless para o Provedor Banda Larga
> Hospedagem e Email personalizado - e claro, no Brasil.
> ****************************************************<wip_load-
> day-2xX2opt.png>
Received on Tue Jul 15 2008 - 16:32:48 MDT