Re: [RFC] cache architecture

On 25.01.2012 08:21, Alex Rousskov wrote:
> On 01/24/2012 01:51 AM, Amos Jeffries wrote:
>> On 24/01/2012 6:16 p.m., Alex Rousskov wrote:
>>> On 01/23/2012 07:24 PM, Amos Jeffries wrote:

>> Admin can already configure
>> several cache_dir and "cache_mem 0". The problem is just that todays
>> Squid do some horrible things when configured that way as side
>> effects
>> of our current design assumption.
>
> The assumption is no longer there, but there is still code that uses
> it.
> This is one of the reasons why I consider finishing deleting
> store_table
> very important.

Sure.

>>>> 3) possibly multiple cache_mem. A traditional non-shared
>>>> cache_mem, a
>>>> shared memory space, and an in-transit unstructured space.
>>> In-transit space is not a cache so we should not mix it and
>>> cache_mem in
>>> an "ideal design" blueprint. Collapsed forwarding requires caching
>>> and
>>> has to go through cache_mem, not in-transit space.
>>
>> So proposals for collapsables which are too large for cache_mem?
>
> If I understand what collapsable requests are,

Sorry. Any request which might be shared/collapsed into another. Along
the same line of terminology broadness as "cacheable".
Not necessarily done, but possible.

> cache admission,
> allocation, or eviction policy can treat collapsables specially. This
> is
> a lower-level issue, IMO, than the blueprint you are discussing
> though.
>
>> or when "cache_mem 0"?
>
> Collapsables can go through disk. If disk caching is also disabled,
> we
> cannot collapse requests because the admin told us not to cache at
> all.
> If the issue is important for some, it would be possible to implement
> a
> caching policy that reserves space for collapsables and caches
> nothing
> else, of course.

It's becoming clear that collapsing needs to run primarily out of the
in-transit space. If request B happens and earlier request A is
collapsable, in-transit, and the response intro bytes still available
the two might be collapsed. Otherwise we have to skip collapsing and
find those initial bytes by current means as a separate request. Cache
HIT/MISS/IMS source of the response data being irrelevant to all this
comparison.

So. Dropping this side discussion for now. We can pick it up again when
planning to add collapsed forwarding.

>
>>> Please keep in mind that any non-shared cache would
>>> violate HTTP in an SMP case.
>>
>> You have yet to convince me that the behavious *is* a violation. Yes
>> the
>> objects coming back are not identical to the pattern of a
>> traditional
>> Squid. But the new pattern is still within HTTP semantics IMO, in
>> the
>> same way that two proxies on anycast dont violate HTTP. The cases
>> presented so far have been about side effects of already bad
>> behaviour
>> getting worse, or bad testing assumptions.
>
> If "MUST purge" is just a MAY because things will usually work even
> if
> the cache does not purge, then this is a question for HTTP WG because
> we
> would not be able to reach an agreement on this issue here; IMHO, a
> burden of proof that ignoring MUST is OK should be on those who want
> to
> ignore it. I do not see a point in those MUSTs if we can violate them
> at
> will by deploying a proxy with two out-of-sync caches.

Agreed on the burden of proof.

It comes down to viewpoint and reality of distributed pathways. Like
Henrik said this is a micro-scale replica of the Internet-wide
distributed cache.
  I see the "MUST purge" as applying to the *pathway*/chain which is
handling the request or reply triggering that MUST. Pathways which never
receive that request/reply CANNOT obey it implicitly.
  Place two separate caches behind one domain name on round-robin IP
selection and the same problem you point at occurs.

The difference viewpoint of syncing vertically (to the origin), versus
horizontally (to sibling caches). All my reading of the spec leads me to
conclude that the mandatory syncing is vertical rather than horizontal
by design.

Yes, we should take reasonable measures to propigate the purge info
around the known caches and make them all behave the same on future
requests. But cannot really call it a true violation when that is
impossible to do so.

We also need to enumerate how many of these cases are specifically
"MUST purge" versus "MUST update". The update case is a lot more lenient
to sync issues than purges are.
  For example any response which has must-revalidate or
proxy-revalidate, or any request which has max-age=0 will simply skip
past and IMS auto-syncs the pathway *with origin*. For the other cases
we often have those huge heuristic estimation holes in the spec to walk
through.

NOTE that we do have sideband means of purging when necessary to
actually purge: IPC packets doing HTCP CLR equivalent between the
workers. Or plain old HTCP CLR between unique HTCP port for each worker.

>>>> 8) cache_dir scan should account for externally added files.
>>>> Regardless
>>>> of CLEAN/DIRTY algorithm being used.
>>>> by this I mean check for and handle (accept or erase)
>>>> cache_dir
>>>> entries not accounted for by the swap.state or equivalent meta
>>>> data.
>>>> + allows reporting what action was taken about the extra files. Be
>>>> it
>>>> erase or import and any related errors.
>>> I think this should be left to individual Stores. Each may have
>>> their
>>> own way of adding entries. For example, with Rock Store, you can
>>> add
>>> entries even at runtime, but you need to update the shared maps
>>> appropriately.
>>
>> How does rock recover from a third-party insertion of a record at
>> the
>> correct place in the backing DB followed by a shutdown?
>> erase the slot? overwrite with something later? load the object
>> details
>> during restart and use it?
>
> Since there is no swap.state, it is the latter: load the object
> details
> during restart and use it.
>
>> For now it is perfectly possible to inject entries into UFS and COSS
>> (at
>> least) provided one knows the storage structure and is willing to
>> cope
>> with a DIRTY restart.
>
> For UFS, you need a DIRTY DIRTY restart (no swap.state at all) if you
> only add a file. You do not need a DIRTY restart if you also update
> swap.state* logs, of course.
>
> For COSS, you may not need a DIRTY restart because, *IIRC*, Squid
> COSS
> does not use swap.state although it does create it.
>
> For Rock, I think you may safely import files without a restart.
>

Thank you.

Amos
Received on Wed Jan 25 2012 - 02:04:07 MST