Re: Integrating FTP-Mirrors

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 4 Nov 1998, Andreas Heiduk wrote:

> Hello!
>
> I want integrate your local FTP Mirror into a Squid hierarchy. Whenever Squid
> sees a "ftp:"-URL, it should fetch it from the local mirror if possible.
>
> I think I could do it with a large redirector database. Another choice would
> be to install a dedicated "FTP-Mirror-Cache" which interacts via ICP with the
> other Squids.

Hello Andreas,

I've been toying with similar ideas myself. I currently have squirm, a
redirector, setup to redirect accesses to common ftp sites to closer
mirror sites.

Given the possibility for stale data / changing mirror information,
I've added rules to allow the redirection to be bypassed. So, for
instance, the following rules redirect accesses to
'updates.redhat.com' to a mirror at sunsite, and accesses to
'-updates.redhat.com' to redhat directly.

regexi ^ftp://(updates\.redhat\.com/.*$) ftp://sunsite.doc.ic.ac.uk/Mirrors/\1
regexi ^ftp://-(updates\.redhat\.com/.*$) ftp://\1

The main problems here are:

1. Communications with mirror site maintainers
2. Administration of a large and volatile redirection list.
3. Lack of automatic fail-over.

The size of the user base is probably also a consideration. Mirror
sites lag behind primary sites, and unfortunately there is no
mechanism I'm aware of to run an 'if-modified-since' / 'if-exists'
type check from a redirector.

Writing a redirector which alternates between two or more mirrors
gives you some failover capability, but at the cost of keeping
multiple copies of the same data in the cache.

Some musings,

Mark

+-------------------------------------------------------------------------+
Mark Cooke The views expressed above are mine and are not
Systems Programmer necessarily representative of university policy
University Of Birmingham URL: http://www.sr.bham.ac.uk/~mpc/
+-------------------------------------------------------------------------+

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
Charset: noconv

iQA/AwUBNkCfVXlbx4ZElNuLEQJ+ZgCff3r9OODMsw5PS7My+njXK7Yb3c8AmwXb
U0sZd7ZAdh5uB1SSrdLlu0EO
=R/KP
-----END PGP SIGNATURE-----
Received on Wed Nov 04 1998 - 11:32:44 MST