Re: Squid memory footprint (was: MemPools rewrite)

On Thu, Nov 02, 2000, Andres Kroonmaa wrote:
>
> I wonder how can you eliminate StoreEntry? IMHO it contains crucial
> information that allows squid to skip disk accesses. Moving parts
> of this data into squidfs doesn't seem to change much in ram usage.
> Moving this crucial information onto disks implies enormous performance
> penalty, doesn't it?

You are assuming the FS can't handle object reference / lock counts and
freshness information itself. :-)

You can easily move the StoreEntry into the FS, making the FS operations
return a StoreEntry and not a StoreIOState. This would reduce a *lot*
of overhead from the storage manager, and psuh it into the FS code.
This works out to be very very good when something like reiserfs_raw is
used, since a lot of the directory code can disappear. It can also
lead to less copies and listwalks in the storage manager.

(Think of it this way: new_StoreEntry() is your create(),
storeAppend() is your write(), storeClientCopy() is your read,
storeRelease() is your unlink(), storeGet() is your open()..
then with a little semantic twisting in the FS layer, you could
do away completely with a lot of the code in store_client.c ,
store_swapin.c and store_swapout.c, since the FS can implement it
if it needs to (and a cache fs can do it in kernel space .. )

I'm currently working on another project to get a working IO model I can
stress test going. (I tried to get there with commloops, but I need
to rewrite the storage manager to get the performance boosts I think I
can get with comm_kqueue / comm_devpoll .. ) Once I've finished, I
will get back onto the storage manager code rework.

Not Enough Hours In The Day Syndrome(tm) :)

Adrian

-- 
Adrian Chadd			"God: Damn! I left pot everywhere!
<adrian@creative.net.au>	  Now I'll have to create Republicans!"
				    - Bill Hicks