Re: [RFC] Have-Digest and duplicate transfer suppression

On 11/08/11 16:58, Alex Rousskov wrote:
> On 08/10/2011 09:29 PM, Amos Jeffries wrote:
>> On Thu, 11 Aug 2011 15:12:56 +1200, Amos Jeffries wrote:
>>> On Thu, 11 Aug 2011 11:09:38 +1200, Robert Collins wrote:
>>>> (But for clarity - I'm fine with what you proposed, I just wanted to
>>>> consider whether the standards would let us do it more directly, which
>>>> they -nearly- do AFAICT).
>>>>
>>>> -Rob
>>>
>>> Same. I don't mind this type of extension ...BUT...
>>>
>>> I think fixing bug 2112 (lack of If-None-Match support) and bug 2617
>>> (wrong ETag validation handling) should be done first before any
>>> extensions are tried. That will allow you to see who much of a problem
>>> (or not) the potential failure cases actually are in practice.
>>>
>>> Amos
>>
>> Want-Digest: and Digest: validation mechanism from
>> http://tools.ietf.org/html/rfc3230 covers the remainder of the proposal.
>> So no custom extensions needed to meet all the requirements.
>
> We did reuse a few ideas from that part of the larger Jeff Mogul's work
> I mentioned earlier, but I believe RFC 3230 Digest and Want-Digest
> headers differ from what is being discussed here:
>
> - Their digests are for instances while our digests are for entities.

I fail to see where entity MD5 can be better than instance MD5.

Especially given that child cache is fed from the parent cache which can
supply Digest: on all responses in advance of a Want-Digest:.

I would think there is a good case for Squid universally supplying MD5
or SHA or both on all HIT/200. It can be (already is?) stored in the
meta data for fast validation.

> - Their headers are end-to-end while ours are hop-by-hop.

By my reading there is no end-to-end requirement.
Leaking the Digest: reply header when used as per the spec is no loss
and some potential benefit.
Leaking a Digest: request header

> - AFAICT, their Digest header is meant mostly for responses, while our
> If-None-Match or Have-Digest header is used in requests.

Yes the most obvious use is in responses. However I do not see anything
actually mentioning a request/reply direction in the RFC.
  IMO the case for Have-Digest is a good case for Digest: request
header. (and a few bytes less to salve the bloat complaints). Used as a
(strong) conditional request validator it makes the first hop which can
satisfy it the "end".
  ie if we implement this in Squid, how is the parent proxy to now that
its own parent in turn wont assist with the optimization when it can't?

There is no guarantee that any recipient parent will act on it to
produce 304 instead of 200. So worst-case is the status quo.

If you insist on this being hop-by-hop you are always free to add it to
the Connection: header to be stripped at the next hop.

>
> Want-Digest or a similar support advertisement is wasteful in the common
> case, but is also useful to prevent sending If-None-Match or Have-Digest
> requests to servers that do not understand them. This is something we
> may want to add.

So what you want the child to be sending is:
  If-None-Match: FOO
  Digest: md5=BLAH

or
  Digest: md5=BLAH

Response from the parent would be 304 or 200 + new object. Noting that
in response to a Digest validated request (strong validator) we can make
304 can send new Expires, Cache-Control, Date, and Vary values to update
the child object headers. Without a body.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.14
   Beta testers wanted for 3.2.0.10