Re: Raw vs. cooked file systems for cache?

On Thu, 7 Jan 1999, Oskar Pearson wrote:

> Hi
>
> > I've been doing some reading on caching proxies, and most people state that
> > you'll get better performance if you don't rely on the OS for your file
> > system management and instead use raw filesystem space for storing the
> > cache. Has anyone played with this for squid that is not reselling it (i.e.
> > don't consider their code to be proprietary or trade secret)?
>
> It has been discussed before on squid-dev. Stewart Foster (who did the
> majority of the async-io support in Squid) was going to be doing this. He
> eventually left connect.com.au, and someone there was going to develop a FS
> for Squid in his place. I believe that this was Kevin Littlejohn
> <darius@connect.com.au>.
>
> estern@packetstorm.on.ca had an alpha version that essentially got rid of
> the FS: I am not sure of progress there.

Ya, I did have it running. However, the method I was using tending to
fragment the store badly after a short period of time. I'm still pondering
the best way to do things.

What I'll probably end up doing is creating a "high performance" patch for
squid. It will give higher performance than stock squid at the expense of
either higher RAM requirements, "wasted" HD space, or both.

I'm currently of the opinion that the best way to go is to store objects
on a raw disk partition rather than a OS FS. This will eliminate the OS
disk cache from being used. Then, you can set squid's cache_mem higher. I
think squid will make better use of the memory than the OS's disk cache
can.

Cisco cacheengine reportedly uses a cyclic filesystem, which is a VERY
attractive design except that I imagine it is disasterous to your hit
rate.

The big problem I'm struggling with is: will the improvements of a custom
FS be worth the effort? If there is only a 5% benefit to be had, its not
really worth doing. Until that question is answered, I'm reluctant to
spend large amounts of time working on it.

/-----------------------------------------------------------------------/
/ Eric Stern - PacketStorm Technologies - (519) 826-9395 /
/ http://www.packetstorm.on.ca /
/ WebSpeed - a transparent web caching server - available now! /
/-----------------------------------------------------------------------/
Received on Thu Jan 07 1999 - 03:18:54 MST