Re: some new branches

I agree with you. I have a strong opinion that the store clients should NOT know about the server side mechanics either though.
Capabilities available when servicing a given request - maybe.

So my goal is to remove the server side logic from the client side, and put all that into the server side (http.c mainly). Then
allow the client side to ask the 'mediation layer'? for results of those operations as meta data and as binary offsets in the
responses.

So we end up with
server protocol >--mediation layer-->client provider
on a non cacheable response (store optimised out)

server procotol >--mediation layer-->client provider
                                      \
                                        \-> store
on a cachable response

and like wise for partial hits, revalidated hits and the like.

My reasoning is that the upstream protocols of http/ftp/wais/as db/gopher shouldn't have to fake http metadata and encoding to
provide data to things on the client side (which are internal such as the as database in squid or the client_side protocol for
sending http).
FTP for example has very different range mechanics than http. But it is capable of making range requests, if a client has requested
that, and with appropriate care even making partial range requests.

At the moment the . is half in client_side, and half in the server side, and spread around to handle issues with gopher response
formatting, and _all_ the data passes through the store. There is no clear 'intermediary layer' because of that. The store acts as a
buffer AND as a storage mechanism.

Idea: split out or at least rename the store in and store out routines to make this separation clear (if it is separated) and if
not, lets separate it out.

Example:
storeDoAppend() always calls storeSwapOut() when data is presented to it. It doesn't act as T junction intermediary at all. (It does
if you consider the store, the _persistent store_ - but the store maanger does more than just persistent storage today).

Rob
ps coupla comments down below as well for clarity.

----- Original Message -----
From: "Henrik Nordstrom" <hno@hem.passagen.se>
To: "Robert Collins" <robert.collins@itdomain.com.au>
Cc: "Adrian Chadd" <adrian@creative.net.au>; <squid-dev@squid-cache.org>
Sent: Wednesday, February 14, 2001 8:04 PM
Subject: Re: some new branches

> I have a strong opinion that the store should not know about HTTP, or at
> most have a very limited knowledge. (some abstract knowledge will be
> required for ranges, ETag support and similar things)
>
> This is why I put the store on a T connection
>
> Cache miss:
>
> server >---.---> client
> \
> \-> store

I think the . is in the wrong position. It should be where the branch to the store occurs, not after it (possibly my ascii layouts
broken).

>
> Cache hit:
>
> store >---.---> client

This is bad - why does the http intermediary need to be involved again? the client is a http client protocol provider - it takes
requests and reponses and formats etc appropriately - whats in the store should have the intermediarys work already done.

ie store >---> http client.

> Partial cache hit (including revalidations which updates headers):
>
> server >---.---> client
> / \
> store >-/ \-> store

problem: the server in this diagram needs information contained in the ".". Why shouldn't the server be given a generic request to
satisfy, with knowledge from the store as to locally stored ranges & variations and then the logic is processed without going back
to the store to the . to the client to decide that, no we need another upstream requests?

>
> Yes, there is a need for a intermediary layer that collects what is
> available and constructs the appropriate upstream query and which splits
> the data flow. This layer is the . in the graphs above. But not even
> this layer theoretically needs to be very specific for HTTP. But calling
> this layer for the store is very inappropriate as it stores nothing. It
> is only logics.

So the store handles storage logics? Cool.

In theory then, we can replace the . intermediary with a (say) realmedia streaming protocol intermediary, a new server side protocol
and a new client side protocol and have a rtsp proxy? This sort of generic framework is good.

So what we have is a set of intermediaries with a common API, a set of protocol clients that use that API, and a set of protocol
servers that potentially have a intermediary specific API, but better if it's standard so that in theory, we can join the bits
together in an arbitrary fashion:

obvious:
FTP server side, http intermediary, http client (external data sink)
http server side, http intermediary, http client (external data sink)

speculation:
HTTP server side, http intermediary, http client, internal HTCP data sink ?
or
HTTP server side, htcp intermediary, htcp client ?

Rob
Received on Wed Feb 14 2001 - 03:32:42 MST