Re: Benchmarking Squid

Alex Rousskov wrote:
> On Tue, 2008-04-29 at 14:17 +1200, Amos Jeffries wrote:
>> I've been giving the benchmarking a little more though over the last few
>> days.
>>
>> The proposal from way back was to build a library of benchmarking results
>> IIRC. We have some responses from people, but I have not seen anything
>> like organisation or presentation of it.
>>
>> We need to publish some basic benchmarking configurations. Adrian has
>> spoken before of his specific configs to stress certain parts of squid.
>> Things like that which we can say to people "do this and let us know the
>> results"
>>
>> When we have those benchmarking instructions ready we are going to need
>> some feedback on the results. What details are needed to make useful
>> graphs/tables etc of the results?
>> - which benchmarking config was used
>> - squid version
>> - OS + Kernel
>> - CPU speed
>> - RAM
>> - HDD speed + type?
>> - NIC speed
>> anything else?
>> and what metrics should things be measured in? req per sec per CPU-MHz by OS?
>
> FWIW, I am making [slow] progress towards the "standardized" setup which
> we can use to benchmark Squid and publish the results. I hope to start
> publishing in May.
>
> A part of that work is preparing a set of modern workloads that folks
> can use to submit more numbers. In my experience it is difficult to get
> reliable/meaningful numbers so I am not very optimistic about collecting
> quality public data (unfortunately), but I think we should still try.
>
> The metrics depend on the workload, but in general you need at least
> rate/throughput, response time, hit ratio (if caching), and errors. No
> single metric is meaningful. You can optimize one and "freeze" others at
> reasonable levels and workload often do that.
>
> HTH,
>
> Alex.
>
>

Thanks Alex. That was pretty much spot on what I was hoping to hear back.
 From what we've been hearing the two key rates will be req/sec (at any
low response time) and hit ratio.

Amos

-- 
Please use Squid 2.6.STABLE19 or 3.0.STABLE4