Re: Caching responses with "Vary" responses

On 07/23/2012 02:31 AM, Alexander Komyagin wrote:
>
>> On 07/19/2012 08:46 PM, Amos Jeffries wrote:
>>> On 19/07/2012 9:50 p.m., Alexander Komyagin wrote:
>>>> On Wed, 2012-07-18 at 11:10 -0600, Alex Rousskov wrote:
>>>>> On 07/17/2012 09:28 AM, Alexander Komyagin wrote:
>>>
>>>>>> It seems that on the first HIT we have successfully loaded data
>> from
>>>>>> the
>>>>>> swap store, but then we save the data to shared memory. So the
>> next
>>>>>> request takes data from shm. And it looks like Squid doesn't save
>>>>>> mem_obj->vary_headers to shm!!
>>>>> Shared memory cache is meant to be just like a disk cache in this
>>>>> aspect. It does not know anything about mem_obj->vary_headers and
>> just
>>>>> packs/unpacks the STORE_META_STD TLV (see StoreMapSlot
>>>>> src/ipc/StoreMap.h) part of StoreEntry.
>>>>>
>>>>> I do not see vary_headers used by any src/fs/* code. Do you know
>> how the
>>>>> disk cache gets them? Perhaps there is a higher level call to pack
>> and
>>>>> unpack Vary request headers that the MemStore is missing. We can
>> add
>>>>> special field/code to StoreMapSlot but I would prefer to see how
>> it is
>>>>> done for the disk storage first, in hope that we can reuse the
>> same
>>>>> code.
>>>>>
>>>> For the disc cache, STORE_META_VARY_HEADERS is used (see
>>>> store_swapmeta.cc). Appropriate tlv is added to the list.
>>>>
>>>> But for shm I don't see any SwapMeta work! There are some calls
>> that are
>>>> writing reply headers and body to the internal mem_hdr object and
>> then I
>>>> can see this object data copied to shm!
>>
>>
>> The current SwapMeta work is in populating StoreMapSlot with
>> STORE_META_STD TLV. Clearly, that is not enough to support Vary.
>>
>
> So why we can't use for shm the same saving procedure as for Swap Cache?
> It's much easier to have only one store object packing procedure for all
> different types of memory store. How can we do that?

Current shared memory cache requires a synchronous "saving procedure"
(one function call must save everything). Disk store API uses an
asynchronous one. Correctly supporting asynchronous "saving procedure"
using shared memory is desirable but difficult, as discussed on the
"object size restrictions" squid-dev thread.

However, we may be able to use the same meta-data part of the "saving
procedure" for both disk and shared memory cache if that part of the
procedure is synchronous. I do not remember why I did not, but this
needs to be re-investigated in case I missed that opportunity.

HTH,

Alex.
Received on Mon Jul 23 2012 - 17:51:22 MDT