Absolutely right.
Check the imdb terms of service for example. Web mirror tools,
browser "accelerators" and prefetch agents are anti-social
and using them just encourages admins to write code to
block them or slow them down, its an arms race nobody wins.
If you run a web mirror tool, it is very hard to ensure you
it will not get stuck in an infinite or near infinite
recursion of content that you will never read. All that
bandwidth costs the site money and if any is dynamic, costs
other users of the site time as the site slows down for
everyone.
Sorry to rain on your parade but this is a pet hate of mine.
-Justin
On Tue, Jan 18, 2000 at 11:52:47AM -0000, Dave J Woolley wrote:
> > From: Ben Mckellar [SMTP:Ben@hotlinesupport.com]
> >
> > Am i able to tell squid to fetch entire sites an store it in its cache?
> >
> Not with squid alone.
>
> > e.g so it would fetch the entire site over nitem and in the morning i
> > could
> > surf it through cache?
> >
> Many commercial sites consider this to be unauthorised
> access (i.e. hacking). Please get written permission
> before attempting it.
Received on Tue Jan 18 2000 - 11:26:17 MST
This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:50:28 MST