Re: [squid-users] Squid 3.3 is very aggressive with memory

Interesting! Where abouts in the codebase is the code relating to the
username cache? I'd like to take a look at this.

Thanks,

Nathan.

On Wed, Dec 18, 2013 at 5:33 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On 18/12/2013 6:54 p.m., Alex Rousskov wrote:
>> On 12/16/2013 10:24 PM, Nathan Hoad wrote:
>>
>>> While running under this configuration, I've confirmed that memory
>>> usage does go up when active, and stays at that level when inactive,
>>> allowing some time for timeouts and whatnot. I'm currently switching
>>> between the two instances every fifteen minutes.
>>>
>>> Here is a link to the memory graph for the entire running time of the
>>> second process, at 1 minute intervals:
>>> http://getoffmalawn.com/static/mem-graph.png. The graph shows memory
>>> use steadily increasing during activity, but remaining reasonably
>>> stable during inactivity.
>>
>> I agree that this looks like a memory leak, but (in general) it could
>> also be some kind of memory pooling or cache entry accumulation.
>>
>>
>>> Where shall we go from here?
>>
>>
>> I recommend the following next steps:
>>
>> 1. Set "memory_pools off".
>>
>> 2. Disable all caching with "cache deny all".
>>
>> Do you see as similar memory growth pattern after the above two steps?
>>
>> * If yes: Time for valgrind or ALL,9 debugging. I can help you make that
>> choice if needed. You can actually do those things now, without doing
>> steps 1 and 2 first, but valgrind and log analysis take time so if we
>> can avoid it by eliminating false positives and/or simplifying the
>> setup, we should do that first...
>>
>> * If no: Try re-enabling caching, but using smaller memory [and disk?]
>> cache sizes so that a cache gets and stays _full_ way before you run out
>> of RAM. This will eliminate cache index growth as a suspect. If your
>> disk cache is full already or uses Rock store, then this applies to
>> memory cache only. Do you see as similar memory growth pattern after
>> re-enabling caching? Are your caches full?
>
>
> The other possibility is username cache in 3.3, since the credentials
> used to be tied to the HTTP request details they came from. We had a
> memory "leak" bug a while back where the credentials reference to that
> request would hold up releasing of the entire transaction state memory.
>
> Amos
Received on Sun Dec 22 2013 - 07:39:55 MST