Re: [squid-users] What load can SQUID cope with in accelerator mode?

At 19:17 14-03-2001, Duane Wessels wrote:
>On Wed, 14 Mar 2001, Simon Morley wrote:
> > For example we have the following hardware:-
> >
> > Dual PIII 800Mhz with 1GB RAM and 8GB SCSI for the cache.
> > This is running Linux with a 2.2 kernel.

>I've done tests with polygraph and achieved 500+ requests per second
>with similar workload characteristics (90% DHR, 10K file size).

Would you guys please share some hints about how you accomplished that?

I have a very similar system (athlon-800, 1.25Gb ram, 4x4gb scsi disk). It
is running two apache webservers one with all the fancy stuff and one to
serve static files (mostly zip-files and a couple of png's). At peak I have
1500 concurrent http-demons of the later running and up to 30-60 seconds of
queue for getting a request served. My best guess is that it was
approximately 20 req/s at peak. In the static-serving http I have a little
perl-script running that translated the request to a physical file via
mod_rewrite.

Then I got the idea of putting a squid up as a accelerator, moving the http
demon to 127.0.0.1, using the same perlscript as redirector. At first
glance it worked like a charm. When I was the only user I got almost full
throughput when requesting a uncached file and it was even faster when the
file was cached. Then I onlined it, and cpuload, memory usage and such
dropped dramatically, the number of http-demons was reduced to about 300 -
fine. Then I tried it for my self, I had to wait about 3 minutes before
squid started to serve my request and when it did I got about 2kb/sec
(instead of expected 1mb/sec). When squid had run for about 10 minutes it
killed itself with a message of "FATAL: Too many queued redirector requests".

I believe that this not is the real cause of death considering that I had
started 30 and one did the job fine when running from apache. Have you any
idea of what could be blocking? disk io?

some facts: OS is FreeBSD 3.3 (highly tweaked, filedescriptors and such are
configured for high load)

squid: 2.3.STABLE4 ./configure --disable-ident-lookups --disable-wccp

Snip from the logfile which repeat every 10 minutes:
-----------------------------------------------------
2001/02/28 00:30:51| Validated 1375 Entries
2001/02/28 00:30:51| store_swap_size = 587134k
2001/02/28 00:30:51| WARNING: All redirector processes are busy.
2001/02/28 00:30:51| WARNING: 10 pending requests queued
2001/02/28 00:30:51| Consider increasing the number of redirector processes
our config file.
2001/02/28 00:30:52| storeLateRelease: released 0 objects
2001/02/28 00:40:51| WARNING: All redirector processes are busy.
2001/02/28 00:40:51| WARNING: 125 pending requests queued
2001/02/28 00:40:51| storeDirWriteCleanLogs: Starting...
2001/02/28 00:40:51| WARNING: Closing open FD 18
2001/02/28 00:40:51| Finished. Wrote 1839 entries.
2001/02/28 00:40:51| Took 0.1 seconds (32771.4 entries/sec).
FATAL: Too many queued redirector requests
Squid Cache (Version 2.3.STABLE4): Terminated abnormally.
----------------------------------------------

Havent really done anything to the config but:
icp_port 0
tcp_outgoing_address 127.0.0.1
cache_mem 128 MB
maximum_object_size 102400 KB
cache_dir ufs /cache 4000 8 8
quick_abort_min -1 KB
range_offset_limit -1 KB
httpd_accel_host 127.0.0.1
httpd_accel_port 80
forwarded_for off
store_avg_object_size 500 KB
client_db off
reload_into_ims on

Regards

-- 
Ole Møller olm@cybercity.dk, Sysadm CyberCity Internet