Re: further to store & message flow discussion

Robert Collins wrote:

> As I see it the "intermediary" is visible from the server
> protocol side, and visible from the client protocol side,
> and the store is not directly visible to either the server
> protocol side or the client protocol side.

Exactly.

> If possible the intermediary layer should be protocol neutral
> itself, dealing in generalised messages with metadata, not http
> headers and the like.

Exactly.

> So what do we call it? Message handler? Protocol arbitrator
> (because it sits between the protocols of http response, http
> request, store reads, store writes, ...) ?

A name? Good question.

> As I see it most of the external store API calls need to be
> deprecated for general use and centralised into the "name to be
> determined" portion of squid.

Exacly.

> This raises another question: for internally generated objects,
> say gopher responses and error messages, does the intermediary
> buffer them until the client reads and sends the message, or does
> the intermediary _appear_ to buffer them, but actually loop them
> through the store as currently happens. (I have no preference here,
> just raising the issue).

Well.. the internal objects should be "asyncronous" just like any
protocol. Very few of the internal objects actually requires buffering.

> So what we should end up with is a straightforward implementation
> of http server rules in client_side, http client rules in http.c,
> and the logic to tie an incoming response to a request and flow
> the data through, copying to store if needed in the intermediary.

That is the vision yes.

> Design points:
> * From upstream the intermediary recieves clearly marked data &
> metadata, and the recieved data need not be contiguous.

Yes, but separated in protocol meta data (headers and internal
information) and data ranges..

> * For sending downstream, the intermediary handles message merging,
> providing client_side the requested data & metadata, or an error
> condition.

Yes.

> * The intermediary is needed to satisfy every request because it
> is the interface through which client_side reaches the store

and the upstream protocols.

> * The intermediary is not http-specific. It can deal with an
> incoming gopher response as accurately and effectively as an incoming
> http response. Corollary: server side protocols must be able to split
> recieved data into metadata and message data.

Here I am not so sure on how things should be done, but probably yes. It
is a question of where and when HTTP gatewaying of the replies takes
place. As long as Squid is a pure HTTP proxy the gatewaying should take
place in the protocol implementation, but if we consider mixing then it
might be a interesting idea to move the gatewaying closer to the client
and store native protocol objects..

> * EOF indicators can be sent in-line on the last buffer, rather than
> afterwards. This saves a whole round of 0 byte writes through
> content chains, and means that the terminating 0 on chunked
> transfers in client_side can be part of the final network packet,
> rather than a single 5 byte network write (hardly efficient).

Minor issue compared to the rest, but is a good suggestion.

/Henrik
Received on Sun Feb 18 2001 - 13:46:40 MST