RE: [squid-users] squid + Intel STL2 and slow performance after few hours

An update here,

Kernel 2.4.2 and kernel 2.4.6 both don't work. I have tried to find out any
problems with Interl STL2 motherboard with 2 1GHz processors and 1GB RAM
but have not got any answers. Intel sent a doc to check the MPS version set
to 1.1 and not 1.4 which could give the problem. The BIOS does not have any
facility to change that version.

So as it stands, the server is next to useless, unless after some time I use
it as my personal web cache server. :(

Something to do with kernel as I have realised that when i stop squid and
have large 'memory cache' even untarring takes up all that cpu utilisation
even ftp download is slow.

I will keep trying but any input is highly appreciated. (anyone using SLT2
server board with linux+squid?)

Mitesh

-----Original Message-----
From: Mitesh P Choksi [mailto:mitesh.choksi@stcl.com]
Sent: 25 July 2001 12:37
To: 'Chemolli Francesco (USI)'; squid-users@squid-cache.org
Subject: RE: [squid-users] PLEASE HELP: Large cache deployment and squid

In case this gives some idea. As soon as i started the L4 switching to point
to squid server, the system usage, i.e. squid usage jumped to and hit 99%.
This was about 12 requests/second.

   procs memory swap io system
cpu
 r b w swpd free buff cache si so bi bo in cs us sy
id
 0 0 0 0 118180 51944 685940 0 0 0 48 119 135 0 5
95
 0 0 0 0 118176 51944 685940 0 0 0 0 101 145 0 0
100
 1 0 0 0 118176 51944 685940 0 0 0 2 111 134 0 3
97
----------- requests startes going in
 2 0 0 0 118044 51944 685960 0 0 4 0 247 38 0 82
18
 1 0 0 0 117848 51944 686044 0 0 5 18 221 36 0 77
23
 2 0 0 0 117644 51944 686132 0 0 16 45 241 38 0 80
20
 2 0 1 0 117628 51944 686140 0 0 0 6 228 3 0 99
1
 2 0 1 0 117284 51944 686244 0 0 1 22 219 43 0 80
20
----------- requests stopped to keep the server alive
 2 0 0 0 117256 51944 686264 0 0 0 58 193 34 0 84
16
 0 0 0 0 117244 51944 686264 0 0 0 6 119 126 0 11
89
 1 0 0 0 117228 51944 686272 0 0 0 26 115 139 0 5
95
 1 0 0 0 117216 51944 686280 0 0 0 18 116 142 0 4
96
 0 0 0 0 117188 51944 686292 0 0 0 26 117 135 0 5
95
 1 0 0 0 117172 51944 686292 0 0 0 13 124 140 0 5
94
 1 0 0 0 117168 51944 686296 0 0 0 10 125 131 0 7
93
 0 0 0 0 117168 51944 686296 0 0 0 1 111 135 0 2
98

May be the server board chipset is not supported, and that high IO is giving
a problem. When the memory cache is low, then no problems, it works like a
breeze.

Mitesh

-----Original Message-----
From: Mitesh P Choksi [mailto:mitesh.choksi@stcl.com]
Sent: 25 July 2001 12:31
To: 'Chemolli Francesco (USI)'; squid-users@squid-cache.org
Subject: RE: [squid-users] PLEASE HELP: Large cache deployment and squid

Hi Chemolli,

Thanks for the direction. I tried with --enable-time-hack and
with --disable-time-hack. but did not find any difference. The only thing I
see here is that as soon as the pagecache increases to 400+MB even a single
request can drive the squid CPU utilization to 5% or more.

Can you please send me the command line that you have used on teh system. I
have also read something about SMB and non-RAM limiting pagecache giving
some problem. Maybe that it's the kernel that will require tuning rather
than squid.

My question is how do I find out what is the problem. Is there anything in
squid that can let me know this.

Moreover, does gettimeofday() and poll() fire even if I have /dev/null in
the cache logs. If so than i can spare some CPU time by disabling logs.

Thanks for any input in advance.

Mitesh

-----Original Message-----
From: Chemolli Francesco (USI) [mailto:ChemolliF@GruppoCredit.it]
Sent: 25 July 2001 10:03
To: 'mitesh.choksi@stcl.com'; squid-users@squid-cache.org
Subject: RE: [squid-users] PLEASE HELP: Large cache deployment and squid

> PLEASE HELP
>
> I have been trying to post a questoin about very large cache
> to the list but
> have not succeeded.
>
> Anyways, I have the following situation, and don't know if
> anybody has faced
> it
> as well.
>
> 2 * 1GHz PIII
> Intel Server Motherboard (ServerWorks PCI compiled in the kernel.)
> 1GB RAM
> cache_mem = 2MB

How come this little cache_mem? You have plenty of RAM, use it...

> pagecache = 2 15 75
> SCSI HDD
> cache_dir = 3 * 13GB on different disks
> linux kernel = 2.4.2-2 smp (recompiled to remove all
> unnecessary stuff and )
> squid version = 2.4STABLE1 (from redhat rawhide) (had to
> install openssl as
> well)
>
> After 12 hours of use even 150+ request / minute (yes
> minutes) squid takes
> up 70%+ CPU.

I have half your CPU speed and manage 150+ req/sec (280 during peak-time).
I don't get those kinds of loads.

> It has also happened to me with 2.3STABLE4 (from default
> rh7.1 linux cd)
> kreiserfsd taking up 10% cpu at times

Check out Joe Cooper's performance tuneups (they are referenced
from somewhere on the Squid site).
The only case I had of such a CPU usage was when using a broken
--enable-time-hack.
I'd suggest that you rebuild squid from scratch on your system (from the
tarball)

> memory cache goes to 555MB+ that's what vmstat shows. squid
> takes up ~20MB
> as shown by top

the 500Mb part is not strange, since Linux allocates all unused RAM to that.

> Strace showed me the following after the 12 hour check (poll %time is
> surprising)
> 100.11 -2088636.745941 -440 2907882 poll

Squid is spinning like crazy in the select loop.
Might be an effect of --enable-time-hack.

> All documents I read says Squid does not require more
> processing power, yet
> I gave it 2 x 1GHz. Is that a problem?

That's much more than you'll need unless you do more than
400 req/s.

--
	/kinkie