Re: [squid-users] Intermittent SquidKerbAuth 'Cannot allocate memory'

On 07/01/2011 12:37, "Nick Cairncross" <Nick.Cairncross_at_condenast.co.uk>
wrote:

>From time to time my users experience constant unsatisfiable prompts from
>squid. Cache.log reports:
>
>2011/01/07 12:04:53| authenticateNegotiateHandleReply: Error validating
>user via Negotiate. Error returned 'BH gss_acquire_cred() failed:
>Unspecified GSS failure. Minor code may provide more information. Cannot
>allocate memory'
>2011/01/07 12:04:53| authenticateNegotiateHandleReply: Error validating
>user via Negotiate. Error returned 'BH gss_acquire_cred() failed:
>Unspecified GSS failure. Minor code may provide more information. Cannot
>allocate memory'
>
>Quickest fix is to 'service squid restart' but I'd like to get to the
>bottom of it as how/why this occurs. Squidkerbauth helper can't allocate
>memory, freezes and refuses to process requests. Has anyone else come
>across this sort of thing before? Memory leak..? Any suggestions for
>further debugging welcome.

Just wanted to post back with my findings so far - still working on
this... With the help from the list users I found the cause of my problem:
A memory leak from squid_kerb_auth when using the KRB5RCACHETYPE="none"
variable
(http://wiki.squid-cache.org/ConfigExamples/Authenticate/Kerberos#Squid_Con
figuration_File). With this variable set and producing a blob via
squid_kerb_auth_test and running this against valgrind on squid_kerb_auth
I receive the following memory leak:

==28959== 68 bytes in 1 blocks are definitely lost in loss record 55 of 68
==28959== at 0x4022903: malloc (vg_replace_malloc.c:195)
==28959== by 0x40CA6F0: krb5_rc_resolve_full (in
/usr/lib/libkrb5.so.3.3)
==28959== by 0x40C7954: krb5_get_server_rcache (in
/usr/lib/libkrb5.so.3.3)
==28959== by 0x4047DA0: krb5_gss_acquire_cred (in
/usr/lib/libgssapi_krb5.so.2.2)
==28959== by 0x40533CD: ??? (in /usr/lib/libgssapi_krb5.so.2.2)
==28959== by 0x403C912: gss_add_cred (in /usr/lib/libgssapi_krb5.so.2.2)
==28959== by 0x403CEB5: gss_acquire_cred (in
/usr/lib/libgssapi_krb5.so.2.2)
==28959== by 0x8049A1C: main (squid_kerb_auth.c:493)

If I unset KRB5RCACHETYPE and re-run the same test I don't receive the leak

==28967== 68 bytes in 1 blocks are still reachable in loss record 60 of 74
==28967== at 0x4022903: malloc (vg_replace_malloc.c:195)
==28967== by 0x40CA6F0: krb5_rc_resolve_full (in
/usr/lib/libkrb5.so.3.3)
==28967== by 0x40C7954: krb5_get_server_rcache (in
/usr/lib/libkrb5.so.3.3)
==28967== by 0x40C1BB1: krb5_rd_req (in /usr/lib/libkrb5.so.3.3)
==28967== by 0x40459C1: krb5_gss_accept_sec_context (in
/usr/lib/libgssapi_krb5.so.2.2)
==28967== by 0x40532C2: ??? (in /usr/lib/libgssapi_krb5.so.2.2)
==28967== by 0x403C318: gss_accept_sec_context (in
/usr/lib/libgssapi_krb5.so.2.2)
==28967== by 0x4058FE0: spnego_gss_accept_sec_context (in
/usr/lib/libgssapi_krb5.so.2.2)
==28967== by 0x403C318: gss_accept_sec_context (in
/usr/lib/libgssapi_krb5.so.2.2)
==28967== by 0x8049AA4: main (squid_kerb_auth.c:500)

I believe the leak relates to this MIT list post:
http://mailman.mit.edu/pipermail/krbdev/2009-November/008248.html.

Unfortunately, I'm using RHEL 5.5 32bit and yum updated to the most recent
RH supported libraries, and the version being used is prior to a fix
(v1.6.1). In the case of my gssapi libraries from rpm -q -i -f
/usr/lib/libgssapi_krb5.so.2 gives

Name : krb5-libs Relocations: (not relocatable)
Version : 1.6.1 Vendor: Red Hat, Inc.
Release : 55.el5 Build Date: Tue 30 Nov 2010
07:33:33 PM GMT
Install Date: Thu 20 Jan 2011 11:35:09 AM GMT Build Host:
x86-006.build.bos.redhat.com
Group : System Environment/Libraries Source RPM:
krb5-1.6.1-55.el5.src.rpm
Size : 1432349 License: MIT, freely
distributable.

So, choices are: Attempt to patch, unset KRB5RCACHETYPE and see how much
load increases or enlist the hep of RH to see what can be done.

Out of interest: Can anyone give a recommendation as to how to work
out/get a counter going on the amount of Kerberos authreqs in, say, a 5
min period? A clumsy way is to use cachemgr ad note the difference in
number of negotiate auth requests after refreshing the page 5 mins later...

Cheers,

The information contained in this e-mail is of a confidential nature and is intended only for the addressee. If you are not the intended addressee, any disclosure, copying or distribution by you is prohibited and may be unlawful. Disclosure to any party other than the addressee, whether inadvertent or otherwise, is not intended to waive privilege or confidentiality. Internet communications are not secure and therefore Conde Nast does not accept legal responsibility for the contents of this message. Any views or opinions expressed are those of the author.

The Conde Nast Publications Ltd (No. 226900), Vogue House, Hanover Square, London W1S 1JU
Received on Thu Jan 20 2011 - 16:07:09 MST