Re: possible squid bugs [was: cacheability engine]

Hi Henrik,

I didn't see that, so to make sure we're on the same page:

You're saying that StoreEntry would only hold last-validated and a
fresh-until value, so that where e is a StoreEntry for a given object,

squid-curtime = local clock time at now
e->last-validated = squid-curtime at last origin server fetch/validation
e->fresh-until = squid-curtime when object becomes stale

All other values come straight from parsed object headers.

To integrate what I was saying then, you do this when you fetch or validate
an object from the origin server:

(from RFC2616 12.2.3):
initial_age = max( max(0, squid_curtime - http_date), http_age)

* Expires-based freshness calculation:
  fresh-until = squid-curtime + ( http_expires - http_date ) - initial_age

* Cache-Control-based freshness calculation:
  fresh-until = squid-curtime + http_maxage - initial_age

for both subsequent requests do:
  is-fresh = ( squid-curtime < e->fresh_until )

Each response serviced from the cache needs to have an Age header attached:

* Age header calculation:
  http_age = initial_age + ( squid-curtime - last-validated )

And those without a fresh-until, but with Last-Modified (and no nasty
Cache-Controls) can do:

* Heuristic freshness calculation:
  obj-pct = ( squid-curtime - last-validated) / ( http_date - http_lastmod )
  is-fresh = ( config-pct > obj-pct )

Heuristic calculation could be calculated ahead of time, but that means that
if the admin changes their config, old objects still get served with the old
pct. Don't know it would be worth the savings to try and work around
this; however, it does mean that you need to look up and parse the object
headers on each heuristic refresh check. I don't know enough about the code
to say how expensive that is.

Looks like it might be worthwhile to store initial_age as well.

Cheers,

On Tue, Mar 28, 2000 at 12:54:31AM +0200, Henrik Nordstrom wrote:
> Duane Wessels wrote:
>
> > I played around with the 2.4 code and removed 'expires' from StoreEntry
> > and replaced 'lastmod' with 'lastvalid'. This makes the StoreEntry
> > smaller, but also makes it incompatible with current caches. Anyway,
> > it seems to work okay. I think we can stop storing lastmodified
> > time in StoreEntry and store lastvalid time instead.
>
>
> What about my earlier proposal to change the storeentry to only contain
> last-validated and fresh-until timestamps, and get rid of all other
> timestamps (including last-modified).
>
> last-validated is used for Age and Date headers. It is the date header
> from the last response seen for the object, or time the request was sent
> if no date header is present (or invalid date header).
>
> fresh-until is the result of the refresh check, and recalculated on
> cache hits.
>
> This makes the timestamps non-ambigous, solves the issue where Squid is
> giving overly stale Date headers, uses less memory and fixes the refresh
> issue with cache digests.
>
> Unless you change the freshness calculation to be on storage/refresh you
> must store lastmodified to be able to calculate the refresh pattern on
> ICP requests.
>
> /Henrik

-- 
Mark Nottingham, Senior Developer
Akamai Technologies (San Mateo, CA)