Re: squid-prefetching status

On Sat, 14 May 2005, Nick Lewycky wrote:

> That's a very good plan. Does anyone have recent logs publicly
> available? I have some IRCache logs for the day of May 31, 2004 -- but
> when I tried the first 5,000 entries, I found that 87% of the prefetches
> weren't fetched later in the log. I think this is mostly because the
> pages changed after that date and also because of filtering effects from
> client caching.

The ircache logs is a second level cache log. Not directly suitable for
prefetching simulation.

There is/was other logs available. You need a log which shows end-user
requests with only client caching, and including referer information so
you can reconstruct the content & relationships somewhat. Using these logs
as "real life" without simulated content is a bit hard as the available
content has changed a lot.

I suspect your best is to use the available leaf cache logs (not the
ircache logs) to build a view of the content model seen at the proxy, then
use this as input to build simulated content where you can run repeated
measurements both with and without prefetching. Next step if you are happy
with the results would be real-life testing with the proxy instrumented to
log a little more information than usual.. (referer, partial hit info,
time since prefetch if prefetched, etc as you may find useful)

> What I'd really like to have is a way to look at the page load times
> instead of running through individual URLs.

page load times is a bit hard to measure at the proxy as it is dependent
on a number of factors not easily seen from proxy logs or even proxy
traffic..

Regards
Henrik
Received on Sun May 15 2005 - 01:20:39 MDT