Re: [squid-users] squid on 32-bit system with PAE and 8GB RAM

Hello,

althouh Amos explained much, I think there may be something to add...

> Gavin McCullagh wrote:
> >We've been monitoring the hit rates, cpu usage, etc. using munin. We
> >average about 13% byte hit rate. Iowait is now a big issue -- perhaps not
> >surprisingly. I had 4GB RAM in the server and PAE turned on. I upped this
> >to 8GB with the idea of expanding squid's RAM cache. Of course, I forgot
> >that the squid process can't address anything like that much RAM on a
> >32-bit system. I think the limit is about 3GB, right?

On 17.03.09 00:31, Amos Jeffries wrote:
> For 32-bit I think it is yes. You can rebuild squid as 64-bit or check
> the distro for a 64-bit build.

I think it was mentioned that 32-bit squid running on 64-bit system can use
nearly 4GB of RAM.

Note that it's always good for squid to leave some RAM for OS, disk cache
etc. Especially on system with that big cache. Object fetched from disk
aren't cached by squid, only by the OS.

> So your 600 GB disk cache is likely to use ~6GB of RAM for index +
> whatever cache_mem you allocate for RAM-cache + index for RAM-cache + OS
> and application memory.

That's not so good. But with that big cache, I'd increase
maximum_object_size which would lower the objects count (Not sure how much).

> >I have two questions. Whenever I up the cache_mem beyond about 2GB, I
> >notice squid terminates with signal 6 and restarts as the cache_mem fills.
> >I presume this is squid hitting the 3GB-odd limit? Could squid not behave
> >a little more politely in this situation -- either not attempting to
> >allocate the extra RAM, giving a warning or an error?
>
> cache.log should contain a FATAL: message and possibly a line or two
> beforehand about why and where the crash occured.
> Please can you post that info here.

However it's very probable that the squid's address space grew out of
possibilities. Recompile OS/SQUID or Lower cache_mem, and maybe even
cache_disk - for performance reasons people already mentioned they use 50%
of their cache partition size (can your single disk handle the traffic?)

> >My main question is, is there a sensible way for me to use the extra RAM?
> >I know the OS does disk caching with it but with a 600GB cache, I doubt
> >that'll be much help.
>
> RAM swapping (disk caching by the OS) is one major performance killer.
> Squid needs direct access to all its memory for fast index searches and
> in-transit processing.

OS disk caching will still help you read from the disk faster (caching of
directories content etc). If you leave some RAM for the OS, of course.

> > I thought of creating a 3-4GB ramdisk and using it
> >as a volatile cache for squid which gets re-created (either by squid -z or
> >by dd of an fs image) each time the machine reboots. The things is, I
> >don't know how squid addresses multiple caches. If one cache is _much_
> >faster but smaller than the other, can squid prioritise using it for the
> >most regularly hit data or does it simply treat each cache as equal? Are
> >there docs on these sorts of issues?
>
> No need that is already built into Squid. cache_mem defines the amount
> of RAM-cache Squid uses.

Although there are afaik some problems in older squid versions that
discouraged using too big memory_cache. And, again, memory cache is only for
objects fetched from network, not for those on disk.

> Squid allocates the disk space based on free space and attempts to
> spread the load evenly over all dirs to minimize disk access/seek times.
> cache_mem is used for the hottest objects to minimize delays even further.

-- 
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
How does cat play with mouse? cat /dev/mouse