Re: [squid-users] Valgrind results on 3.2.1

Hi Marcus,

The 5GB is on a squid on a different server... one not running under valgrind. I've restarted that process since my last e-mail so it's using much less mem now, but be assured it will eventually creep back up to 5GB and beyond.

I get the 5GB number from top and ps... they show roughly that number for both virtual and resident. It's a real problem... the machines will start swapping like crazy when they exhaust physical RAM and the servers slow down badly. Left for long enough, they'll eventually crash with a "malloc() returned null" exception or something along those lines. It's a real leak.

Amos identified some leaks from my valgrind output and when those make it into a daily build I'll try again and report my findings.

Thanks
-Ty

On Sep 27, 2012, at 5:57 PM, Marcus Kool <marcus.kool_at_urlfilterdb.com> wrote:

>
>
> On 09/27/2012 03:39 PM, tcr_at_raynersw.com wrote:
>> Some new stuff has been logged by valgrind overnight- it's at the bottom of this e-mail. Looks like mostly dupes but I see some that might be new.
>>
>> However I still don't see any accounting for the amount of leakage I'm seeing. Right now, squid on one of my servers is using a whopping 5GB of memory, yet when I do mgr:mem on it, it only tells me that the total of all allocations is about 600 MB.
>
> How do you get the information of "5 GB" ?
>
> Note that Valgrind uses various techniques for detecting programming errors and
> one of them is to not actually release memory when a program want to release it,
> but instead administers it as unreleased to detect bugs like
> "accessing released memory". See the Valgrind documentation for more information.
>
>> How can I find out what all those allocations are coming from, since mgr:mem doesn't seem to have any accounting of them?
>>
>> Thank you
>> -Ty
>>
>>
>>
>> ==26647== Invalid read of size 8
>> ==26647== at 0x6CE3D44: __strspn_sse42 (in /lib64/libc-2.12.so)
>> ==26647== by 0x535F79: strListGetItem (HttpHeaderTools.cc:259)
>> ==26647== by 0x51FA07: httpMakeVaryMark (http.cc:538)
>> ==26647== by 0x4BB11A: varyEvaluateMatch (client_side.cc:3812)
>> ==26647== by 0x4D44B2: clientReplyContext::cacheHit(StoreIOBuffer) (client_side_reply.cc:483)
>> ==26647== by 0x57743C: store_client::callback(long, bool) (store_client.cc:165)
>> ==26647== by 0x5778A4: store_client::scheduleMemRead() (store_client.cc:454)
>> ==26647== by 0x577CEA: store_client::doCopy(StoreEntry*) (store_client.cc:383)
>> ==26647== by 0x577FA5: storeClientCopy2(StoreEntry*, store_client*) (store_client.cc:337)
>> ==26647== by 0x4D323B: clientReplyContext::doGetMoreData() (client_side_reply.cc:1751)
>> ==26647== by 0x4D5614: ClientHttpRequest::httpStart() (client_side_request.cc:1358)
>> ==26647== by 0x4D770A: ClientHttpRequest::doCallouts() (client_side_request.cc:1604)
>> ==26647== Address 0x1cb6eba8 is 0 bytes after a block of size 40 alloc'd
>> ==26647== at 0x4C25A28: calloc (vg_replace_malloc.c:467)
>> ==26647== by 0x65B441: xcalloc (xalloc.cc:75)
>> ==26647== by 0x657B99: MemPoolMalloc::allocate() (MemPoolMalloc.cc:62)
>> ==26647== by 0x56D116: String::allocBuffer(unsigned long) (String.cc:56)
>> ==26647== by 0x56D3A5: String::allocAndFill(char const*, int) (String.cc:131)
>> ==26647== by 0x56D458: String::operator=(char const*) (String.cc:85)
>> ==26647== by 0x530A4C: HttpHeaderEntry::HttpHeaderEntry(http_hdr_type, char const*, char const*) (HttpHeader.cc:1503)
>> ==26647== by 0x53269C: HttpHeader::putStr(http_hdr_type, char const*) (HttpHeader.cc:1155)
>> ==26647== by 0x536475: httpHeaderPutStrf (HttpHeaderTools.cc:126)
>> ==26647== by 0x4D10B9: clientReplyContext::buildReplyHeader() (client_side_reply.cc:1426)
>> ==26647== by 0x4D1DDA: clientReplyContext::sendMoreData(StoreIOBuffer) (client_side_reply.cc:2117)
>> ==26647== by 0x57743C: store_client::callback(long, bool) (store_client.cc:165)
>> ==26647==
>> ==26647== Invalid read of size 8
>> ==26647== at 0x6CE3D2D: __strspn_sse42 (in /lib64/libc-2.12.so)
>> ==26647== by 0x535F79: strListGetItem (HttpHeaderTools.cc:259)
>> ==26647== by 0x51FA07: httpMakeVaryMark (http.cc:538)
>> ==26647== by 0x4BB11A: varyEvaluateMatch (client_side.cc:3812)
>> ==26647== by 0x4D44B2: clientReplyContext::cacheHit(StoreIOBuffer) (client_side_reply.cc:483)
>> ==26647== by 0x57743C: store_client::callback(long, bool) (store_client.cc:165)
>> ==26647== by 0x5778A4: store_client::scheduleMemRead() (store_client.cc:454)
>> ==26647== by 0x577CEA: store_client::doCopy(StoreEntry*) (store_client.cc:383)
>> ==26647== by 0x577FA5: storeClientCopy2(StoreEntry*, store_client*) (store_client.cc:337)
>> ==26647== by 0x4D323B: clientReplyContext::doGetMoreData() (client_side_reply.cc:1751)
>> ==26647== by 0x4D5614: ClientHttpRequest::httpStart() (client_side_request.cc:1358)
>> ==26647== by 0x4D770A: ClientHttpRequest::doCallouts() (client_side_request.cc:1604)
>> ==26647== Address 0x19f09418 is 0 bytes after a block of size 40 alloc'd
>> ==26647== at 0x4C25A28: calloc (vg_replace_malloc.c:467)
>> ==26647== by 0x65B441: xcalloc (xalloc.cc:75)
>> ==26647== by 0x657B99: MemPoolMalloc::allocate() (MemPoolMalloc.cc:62)
>> ==26647== by 0x56D116: String::allocBuffer(unsigned long) (String.cc:56)
>> ==26647== by 0x56D3A5: String::allocAndFill(char const*, int) (String.cc:131)
>> ==26647== by 0x56D458: String::operator=(char const*) (String.cc:85)
>> ==26647== by 0x530A4C: HttpHeaderEntry::HttpHeaderEntry(http_hdr_type, char const*, char const*) (HttpHeader.cc:1503)
>> ==26647== by 0x530BC9: HttpHeaderEntry::clone() const (HttpHeader.cc:1614)
>> ==26647== by 0x532DDF: HttpHeader::append(HttpHeader const*) (HttpHeader.cc:484)
>> ==26647== by 0x53A45F: HttpReply::clone() const (HttpReply.cc:615)
>> ==26647== by 0x4D1A1B: clientReplyContext::cloneReply() (client_side_reply.cc:1515)
>> ==26647== by 0x4D1DDA: clientReplyContext::sendMoreData(StoreIOBuffer) (client_side_reply.cc:2117)
>> ==26647==
>> ==26647== Invalid read of size 8
>> ==26647== at 0x6CE3D2D: __strspn_sse42 (in /lib64/libc-2.12.so)
>> ==26647== by 0x535F79: strListGetItem (HttpHeaderTools.cc:259)
>> ==26647== by 0x5705E4: StoreEntry::hasOneOfEtags(String const&, bool) const (store.cc:1981)
>> ==26647== by 0x570741: StoreEntry::hasIfNoneMatchEtag(HttpRequest const&) const (store.cc:1966)
>> ==26647== by 0x4D34EA: clientReplyContext::processConditional(StoreIOBuffer&) (client_side_reply.cc:711)
>> ==26647== by 0x4D47B0: clientReplyContext::cacheHit(StoreIOBuffer) (client_side_reply.cc:571)
>> ==26647== by 0x57743C: store_client::callback(long, bool) (store_client.cc:165)
>> ==26647== by 0x5778A4: store_client::scheduleMemRead() (store_client.cc:454)
>> ==26647== by 0x577CEA: store_client::doCopy(StoreEntry*) (store_client.cc:383)
>> ==26647== by 0x577FA5: storeClientCopy2(StoreEntry*, store_client*) (store_client.cc:337)
>> ==26647== by 0x4D323B: clientReplyContext::doGetMoreData() (client_side_reply.cc:1751)
>> ==26647== by 0x4D45D5: clientReplyContext::cacheHit(StoreIOBuffer) (client_side_reply.cc:504)
>> ==26647== Address 0x104c8748 is 0 bytes after a block of size 40 alloc'd
>> ==26647== at 0x4C25A28: calloc (vg_replace_malloc.c:467)
>> ==26647== by 0x65B441: xcalloc (xalloc.cc:75)
>> ==26647== by 0x657B99: MemPoolMalloc::allocate() (MemPoolMalloc.cc:62)
>> ==26647== by 0x56D116: String::allocBuffer(unsigned long) (String.cc:56)
>> ==26647== by 0x56D3A5: String::allocAndFill(char const*, int) (String.cc:131)
>> ==26647== by 0x56D458: String::operator=(char const*) (String.cc:85)
>> ==26647== by 0x530A4C: HttpHeaderEntry::HttpHeaderEntry(http_hdr_type, char const*, char const*) (HttpHeader.cc:1503)
>> ==26647== by 0x530BC9: HttpHeaderEntry::clone() const (HttpHeader.cc:1614)
>> ==26647== by 0x51FE06: HttpStateData::httpBuildRequestHeader(HttpRequest*, StoreEntry*, RefCount<AccessLogEntry> const&, HttpHeader*, _http_state_flags) (http.cc:1954)
>> ==26647== by 0x520D71: HttpStateData::buildRequestPrefix(MemBuf*) (http.cc:2007)
>> ==26647== by 0x5259B9: HttpStateData::sendRequest() (http.cc:2112)
>> ==26647== by 0x525E08: HttpStateData::start() (http.cc:2165)
>> ==26647==
>>
>>
>>
>>
Received on Fri Sep 28 2012 - 01:09:11 MDT