Re: [squid-users] FreeBSD :Diskd ? (WAS: Compile Crash)

On Thu, Dec 20, 2001, Dave Raven wrote:
> Okay, so are you saying that it would be better to just run soft-updates,
> noasync?
> I'd have too look into it myself, but I'll take your word on that.
> Of course, this leaves the question of whether to run squid in async and
> your disks in noasync?
> Wise/Unwise?
>
> Also, what does diskd do exactly that might benifit squid?

*puts on his FreeBSD committer (albeit currently inactive) hat*

diskd was written because, quite simply, FreeBSD's current threading
system doesn't support non-blocking disk io. This means, that if you
tried to use the aufs store_dir type under squid, each disk operation
would stop the squid process - exactly like ufs does. diskd implements
disk IO by pushing it out to an external process to handle - squid
asks the diskd process to perform some IO, and the disk process
returns the IO when its completed.

softupdates. Softupdates is, quite simply, a mechanism whereby
the data written to disk is done in such a way that the state of
the disk is always consistent. A disk mounted normally with
no options (soft-updates or async) will be much, much slower
since all metadata operations (create/delete inodes, create/update/delete
directories) will be _synchrnonous_ (when you create a directory,
the thing is created and flushed to disk there and then.)

async removes this restriction and lets the OS update things
in the buffer cache before synching (every few seconds or so)
to disk. This is obviously faster, but if you lose power at
any stage the disk state is totally and utterly unknown - you
can have half-written directories, inode/block tables, whatnot,
fsck will generally complain loudly and more often then not
you'll lose your filesystem totally.

softupdates, as stated above, orders the writes to disk in such a way
that if at any time you lose power and have uncommitted changes to
the disk, the disk state is still "consistent". Ie, if you create
a file, softupdates will queue the disk blocks to be written in
such a way that you'll have the following happen:
(Rough here - its been a while since I _looked_ at the code, but
you'll get an idea of what its trying to do.)

* update the block bitmap, pointing out which blocks will be initially
  allocated
* update the inode bitmap, pointing out which inode will be allocated
* update the inode, creating the file (but not linking into a directory)
* update the directory, linking the file into the current directory
* sometime in the future, write the data blocks

The whole point of this exercise is background fsck - if, at any point
in time in the above, you lose power, you'll end up with allocated
blocks which really aren't, an allocated inode which isn't, an inode
which is allocated but not linked into anything, or a created file
(with possibly no data in it.) The filesystem will be consistent, but
some space will be wasted. A background task can then run over the disks
reclaiming free space. This code is real - its in FreeBSD-current,
its working, and it'll be included in FreeBSD-5.0 whenever its released.
Unlike squid, FreeBSD-current _is not_ anywhere near production level,
so please, don't be tempted by this cool feature (and many others!)
since you'll most likely end up with a panic'ed box after a random interval.

Once upon a time, generally in freebsd-3 and maybe the first couple of
freebsd-4.x releases, there were softupdate problems. Nowdays, softupdates
is pretty much a stable thing, and is included in the kernel by default.

Now, as a squid and freebsd person, I run my caches with:

* freebsd and squid (heh)
* diskd cache_dir lines
* all disks mounted with noatime (don't waste time/disk IO updating the
  access times of files)
* all disks with softupdates (stable now)
* sysctl -w vfs.vmiodirenable=1 - improves disk directory IO, sacrifices
  a little RAM, is actually stable now
* various other little tweaks to the sysv, TCP, IP and socket parameters

I haven't played with dirpref just yet, but from what I hear it will
improve directory operations by a substantial amount. Play with it.

hopefully this lengthy rant has clarified a few things.

adrian
Received on Thu Dec 20 2001 - 15:53:07 MST