RE: [squid-users] RE: Squid Restarting Issue

Hey, no patches have been done to squid. We're using that version.

Unfortunately doesn't seem to be any core dumps for that date lying around. Seems like its not configured correctly to do core dumps - changing that now.

Note - we have 6 instances of squid handling heavy load under a load balancer. This has only happened once on one instance in the last couple of months that I'm aware of - only on the project for a few months - so it's unlikely to happen again anytime soon to get a new core dump.

Note - this happened 21 times in a row on only one instance of squid - so it's unlikely that a specific URL came through to cause it - we haven't seen any URLs that weren't in all 6 instances of squid. I'm wondering why did it not continue to happen? Is there any kind of further resilience in squid that would stop the restarts from continuing on? I'm just very concerned that in the future this may happen continuously?

Is there any routine maintenance we could suggest for our client that would mitigate this happening again, like restarts of the server every few weeks? Manually clearing the cache/etc?

Thanks,
Justin

-----Original Message-----
From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
Sent: Tuesday, August 02, 2011 5:23 PM
To: squid-users_at_squid-cache.org
Subject: Re: [squid-users] RE: Squid Restarting Issue

On 02/08/11 18:25, Justin Lawler wrote:
> Hi,
>
> We have an instance of squid - running version 3.0.15 (I know it's fairly old at this stage - but it'll be very difficult to upgrade unless there's a really good reason).
>
> We're getting the below error in squid - this is happening 10-20 times, where squid throws the error& restarts. It eventually seemed to recover itself.
>
> I've found very old bugs on this on version 3.0-Pre3-20050510, but found nothing since then. Is there a bug logged for this issue - could only find something logged on 3.0pre3 - (bug 879) but it says that it was fixed for final release version. Is this a known issue? Or is there any workarounds we can use for now?
>
> Any help much appreciated.
>
> Regards,
> Justin
>

You are the first to hit it again since then.
Is your squid patched with old patches written prior to 3.0 perhapse?

There should be a core dump sitting around from the crash which you can
use to figure out what the real problem behind the crash is. You need to
know that to find a workaround or fix.

Amos

>
> 2011/08/01 04:29:46| WARNING: swapfile header inconsistent with available data
> 2011/08/01 04:29:46| could not parse headers from on disk structure!
> 2011/08/01 04:29:46| memCopy: could not find start of [348,4444) in memory.
> 2011/08/01 04:29:46| mem_hdr::debugDump: lowest offset: 0 highest offset + 1: 163862.
> 2011/08/01 04:29:46| mem_hdr::debugDump: Current available data is: [0,348) - .
> 2011/08/01 04:29:46| storeDirWriteCleanLogs: Starting...
> 2011/08/01 04:29:46| WARNING: Closing open FD 14
> 2011/08/01 04:29:46| Finished. Wrote 43655 entries.
> 2011/08/01 04:29:46| Took 0.11 seconds (386399.24 entries/sec).
> FATAL: Squid has attempted to read data from memory that is not present. This is an indication of of
> (pre-3.0) code that hasn't been updated to deal with sparse objects in memory. Squid should
> coredump.allowing to review the cause. Immediately preceeding this message is a dump of the available data in the
> format [start,end). The [ means from the value, the ) means up to the value. I.e. [1,5) means that there are 4
> bytes of data, at offsets 1,2,3,4.
>
> Squid Cache (Version 3.0.STABLE15): Terminated abnormally.
> CPU Usage: 4.322 seconds = 2.452 user + 1.871 sys
> Maximum Resident Size: 0 KB
> Page faults with physical i/o: 0
> 2011/08/01 04:29:49| Starting Squid Cache version 3.0.STABLE15 for sparc-sun-solaris2.10...
> This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
> you may review at http://www.amdocs.com/email_disclaimer.asp
>

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.14
   Beta testers wanted for 3.2.0.10