Re: ideas

Regarding the SMP question..

Do you remember the discussion we had earlier about breaking Squid up in
various separate components/processes:

* Connection manager(s)
* Store index manager
* Disk I/O threads/daemons
* DNS cache manager
* System monitoring/control manager
* What else there is to separate from the rest..

That design plays quite well with SMP configurations. Normally there
will be one connection manager/CPU, one store index manager, one DNS
cache manager and probably at least one disk I/O thread/daemon per
spindle.

For asymetric multi processing you would probably use one connection
manager / CPU, more than one index manager, and some kind of close
peering between the index managers.

To dig a bit deeper in the design I intended:

a) All connections, both client connections and outgoing connections to
other servers are managed withing the connection manager. If a request
is received by a connection manager, then it is this connection manager
who contacts the next hop in the request forwarding chain.
b) Data received from the origin servers are mirrored to the storage
where applicable. There are also some other types of feedback going from
the connection managers to the store index manager like "make sure this
object is no longer cached", or "this object is still up to date".
c) The store/index manager provides two interfaces:
   i1) Hints in the form of a cache digest or something similar, quickly
telling for sure that an object is NOT in the cache. This is to provide
a very fast path for most cache misses.
   i2) A request interface where the connection manager(s) request a
stored object. Used when (i1) indicates that the object might be in the
cache.
d) The connection managers should be self supporting. If any of the
other components have failed then bypass that component.
e) The system monitoring/control manager only monitors the other
components and makes sure everything is up and running. It is also
responsible to bring down all components in a controlled manner when
asked to.

As I said earlier a design like this requires that the request
forwarding is rewritten to be centered around a request or request
forwarding and NOT a StoreEntry. Store entries will only exist in the
store index manager and/or disk I/O.

Connection sharing will probably only be possible within a connection
manager, and there might be a little higher latency on cache hits, but
that is fairly small price to pay for the added scalability and
realibility gained from having a design like this.

/Henrik

Adrian Chadd wrote:
>
> Hey,
>
> I'm throwing my ideas up on my website. Its just a text file right now,
> and its very rough but I think it outlines the basic ideas I have with
> the network and http client/server code. I will add some storage manager
> comments to this text file next week.
>
> Comments are welcome, as I'd like to finish finalising an outline and
> start implementing rather soon. :-P
>
> Adrian
>
> --
> Adrian Chadd Build a man a fire, and he's warm for the
> <adrian@creative.net.au> rest of the evening. Set a man on fire and
> he's warm for the rest of his life.
Received on Sun Jun 11 2000 - 13:52:35 MDT