ICP multiplexer?

Hi all,

I've been wondering how to distribute the immense load that hits our
central campus proxy cache server while still presenting to neighbor
universities' proxy servers the illusion of a single large cache. Here's
what I think I would have to do to spread the load:

 - large intitutes run their own squids each.
 - all squids on campus are neighbors to each other (with a few exceptions
   depending on link speeds)
 - THERE IS NO SINGLE CENTRAL PARENT (I'll explain why later)

To participate meaningfully in a regional or national caching strategy,
the same still holds, except

 - THERE HAS TO BE A SINGLE CENTRAL PARENT

a patent contradiction!

The problem is that Internet use is doubling so fast that hardware
requirements just cannot be met quickly enough. Spreading the load onto
separate machines is the only feasible alternative that also promises to
scale properly.

On the other hand, I want to offer the information cached locally to other
campuses because we're all competing for the same badly crowded resource.
 I do NOT want them to send 10 ICP packets instead of one, though!

I therefore propose the following solution to this dilemma:

Implement an ICP protocol multiplexer that will, for each ICP request
received, send it to all its children and return a positive answer if one
of them does. The follow-up HTTP requests would already be handled
properly by existing squid software, by querying the children again and
retrieving the info from the one that answers, but of course querying
again is something that would ideally be avoided in a true ICP multiplexer
implementation.

The multiplexer would also be configured to channel ICP requests from
local caches to regional caches, for easier maintenance of regional
cooperation agreements.

A multiplexer could easily be run on several independent machines since it
doesn't actually contain any expensive state, allowing easy backup-service
implementations in a squid/harvest environment without having to implement
rotating DNS (a no-no in our campus LAN, I hear).

Adding this to Squid would probably be straightforward; a single config
line would need to be added to the squid.config file (ICP_children,
perhaps).

Just a thought...

Andreas

-- 
Andreas Strotmann       / ~~~~~~ \________________A.Strotmann@Uni-Koeln.DE
Universitaet zu Koeln  /| University of Cologne   \
Regionales Rechenzentrum| Regional Computer Center \
Robert-Koch-Str. 10    /|    Tel: +49-221-478-5524 |\   Home: -221-4200663
D-50931  Koeln        __|__  FAX: +49-221-478-5590 |__________~~~~~~~~~~~~