Re: squid-2.5 and coss

On 27 Aug 2003, at 9:51, Kinkie <kinkie-dev@kinkie.it> wrote:

> Duane Wessels <wessels@squid-cache.org> writes:
>
> >> Here's a fun suggestion from our friend Robert - how about creating a
> >> _directory_ instead of a file for the COSS cachedir? Then we can place
> >> the store logfile in there, the coss storefile in there _and_ any other
> >> metadata. It'd make life a whole lot easier and mean you won't get bitten
> >> by the requirement to specify swaplog paths..
> >
> > The only reason not to do that is if we want to eventually support
> > using raw devices/partitions for COSS storage. I think it was one
> > of the original plans.
> >
> > However, to do that, the current code would need some enhancements
> > to make sure that I/Os occur on 512-byte boundaries and are always
> > a multiple of 512 in size.

 Isn't block io restricted to 4/8K size anyway? Its a matter of VM of OS.

> > That might get around some 2GB file-size limits in some cases too.
> >
> > Performance-wise, I suspect it doesn't matter. Using the filesystem
> > seems to give darn good throughput already.
>
> In fact, it's a common trend in current RDBMs: they're starting to favour
> storage areas on fileystems rather than on raw devices. Linux has better
> support for filesystem-based raw operations (O_DIRECT) than to raw devices
> proper.
>
> BTW: O_DIRECT would be a good approach to our FS I/O work but:
> - we need to be able to keep an object hot after reading it - the OS won't
> do it for us anymore
> - it only supports reading and writing on page boundaries, typically 4kb
> (we'd need to be able to pad)

 Afaik, original sense of using raw devices or O_DIRECT was to avoid OS
 stupidity in wasting ram on caching useless data and unmanaged fragmentation
 that would lead to random seeks being not optimisable.
 By using raw device or O_DIRECT it becomes responsibility of app to ensure
 optimal disk access pattern.
 Now that ram has become cheap and huge disk systems fast, need for apps
 handling additional complexity is hard to justify.

 COSS, being basically ring buffer, is very simple disk pattern and doesn't
 really benefit much from avoiding OS support, so, imo, it would work as well
 on plain FS as on any other approach. The only reason to go for raw disk
 or O_DIRECT would be if we want to keep ram usage to very minimum, ie for
 very lowend boxes.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Data AS
 Tel: 6501 731, Fax: 6501 725
 Pärnu mnt. 158, Tallinn
 11317 Estonia
Received on Wed Aug 27 2003 - 03:44:59 MDT