Re: prefetching using cache-server statistics

I don't know if it is generic software or not. Check out:

        http://xxx.lanl.gov/RobotsBeware.html

which has links to other info.

Ed

On Sat, 3 Aug 1996, Wojtek Sylwestrzak wrote:

> Are these any generic servers or software or just custom built traps ?
> Are they distinguishable in any way ?
>
> --w
>
> >
> > Beware, there are robot traps built into some web servers, that if they
> > detected a robot trying to download all of their pages, they will stop
> > serving pages to the robot (some are designed to even cause the robot
> > harm). This is especially bad if an organization's proxy server triggers
> > the trap, and so prevents the entire organization for accessing that
> > site's pages (this happened at PARC a couple of months ago, and we had to
> > write to that site to be taken off their list of robots).
> >
> > Ed
> >
>
Received on Sat Aug 03 1996 - 13:38:12 MDT