Re: store module abstractions

----- Original Message -----
From: "Kevin Littlejohn" <darius@bofh.net.au>
To: "Robert Collins" <robert.collins@itdomain.com.au>
Cc: <squid-dev@squid-cache.org>
Sent: Sunday, February 18, 2001 3:44 PM
Subject: Re: store module abstractions

<snip>
> > I'd like to propose that we turn the current 2 level abstraction of "store" &
> "fs" into a three level abstration: "store" &
> > "storefs" & "io".
> >
> > Then the majority of the duplicate code that implements the store logic for t
> he "local filesystem" store becomes a module, and the
> > i/o calls that are the real difference between diskd & async & ufs can be spl
> it out.
>
> Adrian, this is part of the stuff we talked about re: storeRebuild and
> similar, yeah? There is duplicate code there, really the bits that talk to
> the actual drive should have a clear interface, the rebuild and similar logic
> should be shared, and should present a single interface to squid itself.

Yes. That's how I'm suggesting it be broken up.

> Ah - except reading on in your post, you're suggesting a common interface
> _below_ the fs module. I'd be wary of that - some fs'es (reiser, fi) may
> implement themselves in kernel on some platforms, hence have a different more
> efficient way of operating, I'd imagine. Loosing that would be unfortunate.

I'm not suggesting we lose anything:
We create a new interface below the current "fs" module to handle writing and reading to and from the drive.
If a new drive interface is needed, that is analogous to existing the interface, it becomes an i/o module.
If a new interface such as reiserraw crops up (which has little in common with the ufs style store fs modules) it becomes a new "fs
level" module.
And as a new "fs level" module, it can bolt in it's own i/o if it needs to, or leverage the existing i/o modules, or even define a
new low level i/o interface (should that be needed).

I envisage the follow modules to start with:
storefs - coss, ufs, null, sfs, reiserraw
storeio - ufsio, aufsio, diskd

with coss utilising ufsio/aufsio/diskd on appropriate platforms
and ufs utilising ufsio/aufsio/diskd on appropriate platforms
and null using no storeio module
and sfs using ?? (what is special about sfs - it's a kernel fs module using the standard io interface, but tuned for squids read
write pattern & allowing metadata to reside in inodes yes? I'm guessing it can be accessed via ufs/aufs& diskd easily enough. I also
think it needs it's own storefs level module because rebuilds etc will be very different for it.)
and reiserraw using no i/o module (as I understand it reiserraw has no inmemory index, no rebuild, no clean/dirty state..)

> I _think_ that's what you're referring to, I'd be very interested in taking
> this on myself (as it leads directly to being able to make sfs work). If you
> do branch it out, can you let me know if/where I can help? I'm more than
> happy to bog into asyncio/ufs/diskd code - I've been there once long ago with
> Stew, it's fun code to work with :)
>
> KevinL
>

Cool. Well given the plethora of new modules it's introducing, I'd like to make it depend on the generic module work I'm doing..
(but I haven't heard any feedback on that yet ...)

Rob
Received on Sat Feb 17 2001 - 22:01:57 MST