Magic Markers and MD5

Hi,

Browsing through the TODO list, I noted the Magic Marker item.
I'm putting another release of my push caching code through real live
testing (it still slows down when I'm using a very heavily loaded cache as
the receiver). This kind of setup makes cache-cache transfers very important
since one of my goals is to build a "cache array" that can handle
failure better.

If we use a Magic Marker sequence of bytes, how do we know that the
sequence will not occur in real live data? How about adding a
Content-Length to objects that lack one? Or, compute the MD5 checksum
and send a Content-MD5 header? However, we will only be able to tamper
with the headers on HITs; MISSes that cut through the parent proxy
won't have a chance. Nevertheless MD5 is useful for dupe detection,
which I plan to add into the push system.

Anyone already working on this, or with some ideas? I foresee using
the "usual" RSA md5 routines, and with the Init, Update, and Final
functions called from httpReadReply. We store the intermediate context
in HttpStateData, and the final output in MemObject. We can also
put the MD5 checksum in store.log where it can be useful for some log
analyzers.

Cheers,

-- 
miguel a.l. paraz  <map@iphil.net>                              +63-2-893-0850
iphil communications, makati city, philippines          <http://www.iphil.net>