Re: squid-prefetching status

Jon Kay wrote:
> Nick Lewycky wrote:
>
>>Finally, does anyone have suggestions for how to test for performance
>>improvement due to prefetching?
>
> A good way to test how your algorithms are working is to get a nice, long
> actual Squid workload -eg, URLs fetched, and compare how long it takes
> to execute the whole thing with and without prefetching.

That's a very good plan. Does anyone have recent logs publicly
available? I have some IRCache logs for the day of May 31, 2004 -- but
when I tried the first 5,000 entries, I found that 87% of the prefetches
weren't fetched later in the log. I think this is mostly because the
pages changed after that date and also because of filtering effects from
client caching.

What I'd really like to have is a way to look at the page load times
instead of running through individual URLs.

> Note that you generally have to prefetch a LOT of stuff to get much
> improvement,
> because web cache fetch popularity follows zipf's law and decays slowly.

I hadn't heard of Zipf's law. It's interesting, thank you for
introducing me to it! Just to make certain I understand what you're
saying ... you're noting that I need a lot of log data to test with
because most fetches enter the working set where prefetching won't help
and so I need a large number of cache misses?

> Good luck with your work.

Thank you!

Nick Lewycky
Received on Sat May 14 2005 - 19:02:53 MDT