Re: Accessing wwwcache.hensa using squid

As Martin kicked a bit of this thread over to squid-users, it
seems relevant to add my 2 penn'orth as well.

P
------- Forwarded Message

Received: from neptune.pegasus.cranfield.ac.uk by xdm096.pegasus.cranfield.ac.u
k with SMTP
        id AA18330; Thu, 24 Oct 1996 12:29:14 +0100
Received: from gizmo.lut.ac.uk (majordom@gizmo.lut.ac.uk [158.125.96.46]) by
neptune.pegasus.cranfield.ac.uk (8.7.5/8.7.3) with ESMTP id MAA00014 for
<P.Lister@cranfield.ac.uk>; Thu, 24 Oct 1996 12:28:27 +0100 (BST)
Received: (majordom@localhost) by gizmo.lut.ac.uk (8.8.2/8.6.9) id MAA13268
for cybercache-outgoing; Thu, 24 Oct 1996 12:21:21 +0100 (BST)
Received: from calisto.ccc.cranfield.ac.uk (calisto.ccc.cranfield.ac.uk
[138.250.1.172]) by gizmo.lut.ac.uk (8.8.2/8.6.9) with ESMTP id MAA13263 for
<cybercache@mrrl.lut.ac.uk>; Thu, 24 Oct 1996 12:21:18 +0100 (BST)
From: P.Lister@cranfield.ac.uk
Received: from panama (actually host panama.pegasus.cranfield.ac.uk)
          by calisto with SMTP (PP); Thu, 24 Oct 1996 12:19:33 +0100
Received: from localhost by panama (5.65/4.7) id AA10208;
          Thu, 24 Oct 1996 12:19:03 +0100
Message-Id: <9610241119.AA10208@panama>
X-Mailer: exmh version 1.6.7 5/3/96
To: "M.L.Bowman" <mlb@sesame.hensa.ac.uk>
Cc: wwwcache-users@mailbase.ac.uk, p.lister@cranfield.ac.uk,
        cybercache@mrrl.lut.ac.uk
Subject: Re: Accessing wwwcache.hensa using squid
In-Reply-To: Your message of "Wed, 23 Oct 96 11:30:27 BST."
<199610231030.LAA00018@unix.hensa.ac.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Thu, 24 Oct 96 12:19:03 +0100
X-Mts: smtp
Sender: owner-cybercache@mrrl.lut.ac.uk
Precedence: bulk

> At the moment Netscape is coping well with the load put on it,
> about 2.8 million requests per day. If you know of a Squid server
> successfully handling this kind of load we would be interested to hear
> about it.

Maggie,

Though I appreciate that the support costs of updating software which
is developing constantly should never be underestimated (I'm only just
updating to Squid, because, hey, Harvest 1.4 has been working fine for
months), I think you miss the point. Please accept the following as
the constructive criticism it is intended to be. Believe me, I write
in sorrow, not in anger.

I believe that the reason that Squid user want HENSA use to use Squid
is NOT because it's better at handling lots of requests from browsers,
but because it's better at handling ICP *between* co-operating sites
caches, particularly using UDP and multicast and all that cool
stuff. The recent debate about optimal cache topologies applies
equally to local as well as international links. My observation is
that Squid actually handles requests quite fast - if the load is so
high that even Squid can't cope, then maybe there's a deeper problem,
huh? So, no, I can't show you a Squid server that can handle the load
you describe, but maybe if you joined the cybercache party we could
all handle those requests more efficiently.

The trouble is, HENSA seems to treat Squid users as eccentrics haven't
yet Seen the Light of the Gospel According to Netscape. No personal
slight intended; if HENSA staff assure me that you don't think that
way then I'll accept that - as long as you accept that this is
impression that I as a cybercachie receive. Describing Netscape v2 as
"the next generation of proxy technology"; is the kind of hype I
expect to see on www.netscape.com, *not* on
<http://www.hensa.ac.uk/wwwcache/tfaq.html>, the FAQ of a service
which most non-experts would assume is providing the academic
community with neutral advice. Hensa describes itself as "The UK
Academic National Web Cache at HENSA Unix" (note "The" - no others are
acknowledged). The only reference to Squid is "How can I configure
Squid to use you as a parent cache?", which hardly represents a
balanced discussion of their relative technical merits. We get advice
like "How should I configure my browser?" which says a great deal
about the HENSA attitude - please would you change the first line of
this FAQ section to read "If your local site runs a cache, you'll find
this much more efficient; contact your local Computer Centre for
details"?

Consider the DNS - almost every JANET site runs a local server, run in
co-operating with the top levels of ac.uk and uk. You want more
resilience? Run a mutual secondary with another site. Is your site
DNS overloaded? add a local secondary or delegate a domain to
department. But handle traffic problems *locally*. Do we have a single
large national DNS server? A single national mail hub? A single
national NNTP server? No, of course not, the idea would be
laughable. I suggest that the same is true of web caches.

Don't misunderstand me; there is a definite role for national caches
like Hensa as centres for connection to other cache systems, and as
facilitators to helping JANET sites to run their own caches. Maybe
it's even appropriate for some smaller sites without resources for
caching to use HENSA direct. How many of your 2.8 million requests are
from browsers at sites which already have local caches? How many are
from sites which probably could use a cache but haven't got round to
it yet? How many could be potentially be handled by an ICP multicast
with ac.uk? I reckon that there's far more interesting work to do in
co-ordinating JANET site caches than in just looking after one big
one.

My suggestion to HENSA is - run one of your boxes as a Squid cache,
available only to registered JANET site caches, and encourage sites to
run site caches. Use the details of the registered site caches to
make an *intelligent* Netscape automatic proxy configurator so that
browsers use their local site caches. Study logs to see: (1) which
sites are configuring their browsers to use HENSA even though there's
a local site cache available - persuade them to use it; (2) which
sites are noticeable by their absence from any cache logs - persuade
them to set up local caches. Where sites can't or won't run local
caches, encourage them to register a local "www-cache.camford.ac.uk"
alias pointing back at HENSA or (another co-operating site) so that a
local cache could potentially still be set up without needing to
reconfigure thousands of browsers.

Remember also, not only do us Squid sites wish to use HENSA's cache as
a parent, but I think larger Squid sites would be happy for *HENSA* to
neighbour *our* caches, creating a large JANET wide cache to minimises
the number of outgoing requests as well as reducing the incoming
traffic to any single cache.

I'm beginning to sound like a "Make $$$ fast" spam, but can anyone
point out any fundamental flaws in what I've just said? As far as I
can see, everybody wins, HENSA included.

Peter Lister Email: p.lister@cranfield.ac.uk
Computer Centre, Cranfield University Voice: +44 1234 754200 ext 2828
Cranfield, Bedfordshire MK43 0AL UK Fax: +44 1234 751814
- ------------------------------------------------------------------------
GLENDOWER I can call spirits from the vasty deep.
HOTSPUR Why, so can I, or so can any man;
          But will they come when you do call for them? (Henry IV, pt 1)
- ------------------------------------------------------------------------

------- End of Forwarded Message

-- 
Peter Lister                             Email: p.lister@cranfield.ac.uk
Computer Centre, Cranfield University    Voice: +44 1234 754200 ext 2828
Cranfield, Bedfordshire MK43 0AL UK        Fax: +44 1234 751814
------------------------------------------------------------------------
GLENDOWER I can call spirits from the vasty deep.
HOTSPUR   Why, so can I, or so can any man;
          But will they come when you do call for them? (Henry IV, pt 1)
------------------------------------------------------------------------