Re: Caching responses with "Vary" responses

> On 07/19/2012 08:46 PM, Amos Jeffries wrote:
> > On 19/07/2012 9:50 p.m., Alexander Komyagin wrote:
> >> On Wed, 2012-07-18 at 11:10 -0600, Alex Rousskov wrote:
> >>> On 07/17/2012 09:28 AM, Alexander Komyagin wrote:
> >
> >>>> It seems that on the first HIT we have successfully loaded data
> from
> >>>> the
> >>>> swap store, but then we save the data to shared memory. So the
> next
> >>>> request takes data from shm. And it looks like Squid doesn't save
> >>>> mem_obj->vary_headers to shm!!
> >>> Shared memory cache is meant to be just like a disk cache in this
> >>> aspect. It does not know anything about mem_obj->vary_headers and
> just
> >>> packs/unpacks the STORE_META_STD TLV (see StoreMapSlot
> >>> src/ipc/StoreMap.h) part of StoreEntry.
> >>>
> >>> I do not see vary_headers used by any src/fs/* code. Do you know
> how the
> >>> disk cache gets them? Perhaps there is a higher level call to pack
> and
> >>> unpack Vary request headers that the MemStore is missing. We can
> add
> >>> special field/code to StoreMapSlot but I would prefer to see how
> it is
> >>> done for the disk storage first, in hope that we can reuse the
> same
> >>> code.
> >>>
> >> For the disc cache, STORE_META_VARY_HEADERS is used (see
> >> store_swapmeta.cc). Appropriate tlv is added to the list.
> >>
> >> But for shm I don't see any SwapMeta work! There are some calls
> that are
> >> writing reply headers and body to the internal mem_hdr object and
> then I
> >> can see this object data copied to shm!
>
>
> The current SwapMeta work is in populating StoreMapSlot with
> STORE_META_STD TLV. Clearly, that is not enough to support Vary.
>

So why we can't use for shm the same saving procedure as for Swap Cache?
It's much easier to have only one store object packing procedure for all
different types of memory store. How can we do that?

>
> >> In the meantime, those who are in need can use my small dirty hack
> >> (squidv3-vary-headers-shm-hack.patch) that packs
> mem_obj->vary_headers
> >> as an additional reply header, so it can be flawlessly saved to shm
> and
> >> recovered afterwards.
> >
> > Isn't vary_headers just a copy of Vary: reply header anyway? so it
> is
> > already in the response headers once.
>
>
> AFAIK, you need to store the matching _request_ headers to tell
> whether
> the new request headers are the same.

Yeah, my first though was indeed to reuse stored headers. But
vary_headers are the calculated function of reply headers and the
original request (httpMakeVaryMark() function). So, as Alex pointed out,
we really need them.

In the patch I've previously attached, Squid just adds the calculated
vary_headers value as an additional special header to the reply, so it
can be saved in shm (without interfering any Squid reply processing
logic!).
Then, when the mem_obj is recovered from memory, it's vary_headers field
is populated from the corresponding header.
Of course, that header is removed during reply preparation for the
client, so clients are completely unaware of such a black magic ;)

>
> HTH,
>
>
> Alex.
>

-- 
Best wishes,
Alexander Komyagin