Re: [squid-users] Wondering Why This Isn't Caching

On Friday 13 December 2013 11:51:05 Martin Sperl wrote:
> Here the back-ported patch to 3.4.1, which we will be testing
internally -
> until it gets incorporated into the 3.4 release tree...
 
> Initial tests show that it works as expected - the vary-response-
header is
> now handled correctly...
 
> Thanks,
> Martin
>
> Here the modified patch based on
> http://bugs.squid-cache.org/show_bug.cgi?id=3806 which applies
against the
> clean tar ball of 3.4.1:
 
> === modified file 'src/store.cc'
> --- src/store.cc 2013-02-16 02:26:31 +0000
> +++ src/store.cc 2013-03-25 10:51:54 +0000
> @@ -750,7 +750,7 @@
> StoreEntry *pe = storeCreateEntry(mem_obj->url,
> mem_obj->log_url, request->flags, request->method);
 /* We are allowed to
> do this typecast */
> HttpReply *rep = new HttpReply;
> - rep->setHeaders(Http::scOkay, "Internal marker object",
> "x-squid-internal/vary", -1, -1, squid_curtime + 100000);
 +
> rep->setHeaders(Http::scOkay, "Internal marker object",
> "x-squid-internal/vary", 0, -1, squid_curtime + 100000); vary =
> mem_obj->getReply()->header.getList(HDR_VARY);
> if (vary.size()) {
>
> Various fixes making Vary caching work better.
> More work is needed to re-enable shared memory caching of Vary
responses.
>
> bag5s r12741: Do not start storing the vary marker object until its
key
> becomes public.
 bag5s r12742: Log failed (due to "Vary object loop" or
> "URL mismatch") hits as TCP_MISSes. bag5s r12743: Refuse to cache
> Vary-controlled objects in shared memory (for now).
> === modified file 'src/MemStore.cc'
> --- src/MemStore.cc 2012-10-16 00:18:09 +0000
> +++ src/MemStore.cc 2013-11-08 22:02:09 +0000
> @@ -293,40 +293,46 @@ MemStore::considerKeeping(StoreEntry
&e)
> }
>
> assert(e.mem_obj);
>
> const int64_t loadedSize = e.mem_obj->endOffset();
> const int64_t expectedSize = e.mem_obj->expectedReplySize();
>
> // objects of unknown size are not allowed into memory cache, for
now
> if (expectedSize < 0) {
> debugs(20, 5, HERE << "Unknown expected size: " << e);
> return;
> }
>
> // since we copy everything at once, we can only keep fully loaded
> entries
 if (loadedSize != expectedSize) {
> debugs(20, 7, HERE << "partially loaded: " << loadedSize << "
!= "
> <<
 expectedSize);
> return;
> }
>
> + if (e.mem_obj->vary_headers) {
> + // XXX: We must store/load SerialisedMetaData to cache Vary in
RAM
> + debugs(20, 5, "Vary not yet supported: " <<
> e.mem_obj->vary_headers); + return;
> + }
> +
> keep(e); // may still fail
> }
>
> bool
> MemStore::willFit(int64_t need) const
> {
> return need <= static_cast<int64_t>(Ipc::Mem::PageSize());
> }
>
> /// allocates map slot and calls copyToShm to store the entry in
shared
> memory
 void
> MemStore::keep(StoreEntry &e)
> {
> if (!map) {
> debugs(20, 5, HERE << "No map to mem-cache " << e);
> return;
> }
>
> sfileno index = 0;
> Ipc::StoreMapSlot *slot = map-
>openForWriting(reinterpret_cast<const
> cache_key *>(e.key), index);
 
> === modified file 'src/client_side_reply.cc'
> --- src/client_side_reply.cc 2013-10-01 23:21:17 +0000
> +++ src/client_side_reply.cc 2013-11-08 22:02:09 +0000
> @@ -470,71 +470,73 @@ clientReplyContext::cacheHit(StoreIOBuff
> /* the store couldn't get enough data from the file for us to id
> the
 * object
> */
> /* treat as a miss */
> http->logType = LOG_TCP_MISS;
> processMiss();
> return;
> }
>
> assert(!EBIT_TEST(e->flags, ENTRY_ABORTED));
> /* update size of the request */
> reqsize = result.length + reqofs;
>
> /*
> * Got the headers, now grok them
> */
> assert(http->logType == LOG_TCP_HIT);
>
> if (strcmp(e->mem_obj->url, http->request->storeId()) != 0) {
> debugs(33, DBG_IMPORTANT, "clientProcessHit: URL mismatch, '"
<<
> e->mem_obj->url << "' != '" << http->request->storeId() << "'");
 +
> http->logType = LOG_TCP_MISS; // we lack a more precise LOG_*_MISS
code
> processMiss();
> return;
> }
>
> switch (varyEvaluateMatch(e, r)) {
>
> case VARY_NONE:
> /* No variance detected. Continue as normal */
> break;
>
> case VARY_MATCH:
> /* This is the correct entity for this request. Continue */
> debugs(88, 2, "clientProcessHit: Vary MATCH!");
> break;
>
> case VARY_OTHER:
> /* This is not the correct entity for this request. We need
> * to requery the cache.
> */
> removeClientStoreReference(&sc, http);
> e = NULL;
> /* Note: varyEvalyateMatch updates the request with vary
> information
 * so we only get here once. (it also takes care of cancelling
> loops) */
> debugs(88, 2, "clientProcessHit: Vary detected!");
> clientGetMoreData(ourNode, http);
> return;
>
> case VARY_CANCEL:
> /* varyEvaluateMatch found a object loop. Process as miss */
> debugs(88, DBG_IMPORTANT, "clientProcessHit: Vary object
loop!");
> + http->logType = LOG_TCP_MISS; // we lack a more precise
LOG_*_MISS
> code
 processMiss();
> return;
> }
>
> if (r->method == Http::METHOD_PURGE) {
> removeClientStoreReference(&sc, http);
> e = NULL;
> purgeRequest();
> return;
> }
>
> if (e->checkNegativeHit()
> && !r->flags.noCacheHack()
> ) {
> http->logType = LOG_TCP_NEGATIVE_HIT;
> sendMoreData(result);
> } else if (!http->flags.internal && refreshCheckHTTP(e, r)) {
> debugs(88, 5, "clientCacheHit: in refreshCheck() block");
> /*
> * We hold a stale copy; it needs to be validated
>
> --- src/store.cc.caching 2013-12-08 17:20:54.000000000 -0800
> +++ src/store.cc 2013-12-13 02:29:00.101025671 -0800
> @@ -795,12 +795,14 @@
> }
>
> #endif
> - pe->replaceHttpReply(rep);
> + pe->replaceHttpReply(rep,false); // no write until key is
> public
 
> pe->timestampsSet();
>
> pe->makePublic();
>
> + pe->startWriting(); // after makePublic()
> +
> pe->complete();
>
> pe->unlock();
>
> -----Original Message-----
> From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
> Sent: Donnerstag, 12. Dezember 2013 21:03
> To: squid-users_at_squid-cache.org
> Subject: RE: [squid-users] Wondering Why This Isn't Caching
>
> On 2013-12-12 23:57, Martin Sperl wrote:
>
> > Ok - I have just tried the latest version from trunk and it seems to
> > do what it is expected to do now!
> >
> > So the question is: will this patch also go into 3.4Beta or do we
have
> > to wait for 3.5 to get this resolved?
>
>
> Well yes and no. 3.4 beta is done with. I am okay with it going into the
> 3.4 stables' though.
>
> Amos
>
> This message and the information contained herein is proprietary
and
> confidential and subject to the Amdocs policy statement,
 you may review at
> http://www.amdocs.com/email_disclaimer.asp

I applied this patch to my 3.4.1 installation. This fixed a problem where
websites powered by cloudflare were not being cached by squid. Now
they are cached. thank you very much!
Received on Sat Dec 14 2013 - 23:56:44 MST