Re: doublecheck

----- Original Message -----
From: "Duane Wessels" <wessels@squid-cache.org>
To: <squid-dev@squid-cache.org>
Sent: Tuesday, November 14, 2000 5:50 PM
Subject: doublecheck

> Folks,
>
> I'm sorry for falling behind on email. I should have kept up with
> these doublecheck changes.
>
> I really don't like the code that got just got added.
>
> when I originally wrote the doublecheck feature, it was to be used
> for offline debugging, never on a live production cache. Thats
> why -S option does nothing except abort() if it finds an anomoly.
> It was an undocumented "feature" that I had used a couple of times.
> Probably it should have been removed.
>
> The code that got committed makes my caches really really slow
> during a dirty rebuild.

Yes it is slower... but faster than a complete rebuild from disk I think.

> It is totallly unacceptable to call stat() for every file during
> rebuild, no matter if clean or dirty. The rebuild process needs
> to be made faster, not slower!

I agree.

> The swapin metadata MD5 and URL checks make sure that we don't swap
> in bogus data. I see these stat() calls as superfluous.

So if we have URL listed as cached, when we try to serve the hit, we
findout it is a miss (because of bogus data?)
Then I suggest that we #ifdef the -S switch to a --enable option, and make
the -S fix the problems it finds rather than
just report them.

> I think what you want to do belongs more with the "validation"
> procedure. Better to start a background task that calls stat at
> a low rate if necessary, after the swap.state files have been fully
> read.

I'll look at that. Are there any gotchas I should know about
locking/removing store items once the swap.state is in memory? Perhaps a
reference in the code to see the best way of doing it?

> Duane W.
>
>
>
>
Received on Tue Nov 14 2000 - 00:49:23 MST