Re: [squid-users] 100% CPU Load problem with squid 3.3.8

OK so now there is another thing That I have tested:
/etc/pam.d/common-session
dosn't have the limit module as a default so the admin will set it as he
wants and to prevent a problem..

adding this line:
session required pam_limits.so

to the common-session file forces the ulimits on a PAM session startup
and end..
this forces the bash(which is a pam) session to use the limits that are
set by the admin in the limits.conf...
It's not such a good idea to allow a users such a thing but this is the
admin choice.

Eliezer

On 09/14/2013 06:07 PM, Pavel Kazlenka wrote:
> On 09/14/2013 03:44 PM, Eliezer Croitoru wrote:
>> SORRY typo:
>> http://www.linuxtopia.org/online_books/linux_administrators_security_guide/16_Linux_Limiting_and_Monitoring_Users.html#PAM
>>
>>
>> the above can clarify more about the ulimit stuff.
>>
>> The basic solution is to define the soft limit in the init script.
>
> I don't think that this is a good solution. It could work as a temporary
> hook. See my thoughts below.
>> I would go to make sure that the hard and the soft limits are the
>> problem..
>> like
>> ulimit -Sa >>/tmp/ulimit_test
>> ulimit -Ha >>/tmp/ulimit_test
>>
>> this will make sure that the limits problem are in the soft and hard.
>>
>> It's a basic linux issue which is not related to squid but more to the
>> distribution and how you define ulimits
>>
>> I assume the limit is on the bash level rather then on the OS level.
>> http://www.linuxtopia.org/online_books/linux_administrators_security_guide/16_Linux_Limiting_and_Monitoring_Users.html#Bash
>>
>>
>> hope it helps clarify the issue.
>>
>> There could be an option that will be added to the init.d script to
>> specify the ulimit soft and hard by a config file or variable.
>>
>> I hope to post a new script for centos in the comming weeks.
>>
>> Eliezer
>>
>> On 09/14/2013 03:33 PM, Eliezer Croitoru wrote:
>>> as stated before the mentioned solution was to insert the ulimit into
>>> the init script to make sure the limit is absolute!
>>>
>>> there might be a chance for this to solve or help solve and find the
>>> issue:
>>> On 09/14/2013 12:05 PM, Mohsen Dehghani wrote:
>>>> Oh , no...it is 1024
>>>> thanks for the help
>>>> Now I added 'ulimit -n 65000' in squid init file and the problem is
>>>> resolved. But some questions:
>>>>
>>>> 1-why is it 1024 While I've set 65535 FD at compile time and squid user
>>>> which is "proxy" has this much limitation in limit.conf file?
> This is an interesting question. I guess we need someone like
> package/distribution maintainer here, because I don't know why limits.d
> doesn't work.
>>>> 2-is it ok to increase FD limit in this way?
> No, that's not a good idea. You will have the problem each time you will
> try to update your squid using package manager (as you .
> You have to set limits in /etc/security/sysctl.d/squid.conf file (or
> file with another name). Of course, you have to find out why this
> doesn't work at the moment.
>>>> 3-Apearantly according to "# cat /proc/sys/fs/file-max" my os FD
>>>> limit is
>>>> 400577. Can I increase squid FD to it
> Not really good idea to, as thus you allow squid to use all the
> available system-wide file descriptors. This value doesn't seem to be
> too high though so you can increase both system file descriptors
> (sys.fs.file-max) and squid's one.
>>>> 4-What is best FD limit for about 150Mbps bandwidth and 18000 RPM
> 18000 rpm means you need 18000 descriptors available. I guess it will
> not be hard to find out appropriate value watching squid log and
> increasing nofile system limit each time you encounter warning in
> squid's log.
>>>>
>>>> This could be ugly troubleshooting practice, but you can try to
>>>> modify your
>>>> init script (or upstart job, not sure how exactly squid is being
>>>> started in
>>>> ubuntu). The idea is to add 'ulimit -n > /tmp/squid.descriptors' and
>>>> see if
>>>> the number is really 65k.
>>>>
>>>> On 09/14/2013 09:41 AM, Mohsen Dehghani wrote:
>>>>>> I don't see any logic here. Are you sure your squid is started not by
>>>> root?
>>>>>> Is replacing 'root' by 'squid' or '*' solves issue as well?
>>>>> When I manually start service by root, there is no file descriptor
>>>>> warning and squid works as normal.
>>>>> But when the system boots up and starts the service automatically,
>>>>> squid runs out of FD.
>>>>>
>>>>> I've tested different the following settings without any luck. Every
>>>>> time that the box reboots, I have to login and restart service
>>>>> manually.
>>>>>
>>>>> root soft nofile 65000
>>>>> root hard nofile 65000
>>>>> proxy soft nofile 65000
>>>>> proxy hard nofile 65000
>>>>> squid soft nofile 65000
>>>>> squid hard nofile 65000
>>>>> * soft nofile 65000
>>>>> * hard nofile 65000
>
> The values seems fine. What exactly the name of file you put them into?
>>>>>
>>>>> It seems these settings only works if the user logins to system.
>>>>> My squid user is "proxy"(I configured it at the time of compile).
>>>>>
>>>>> Maybe some useful info:
>>>>> OS:Ubuntu 12.04
>>>>>
>>>>> # ulimit -n
>>>>> 65000
>>>>>
>>>>> # squidclient mgr:info | grep 'file descri'
>>>>> Maximum number of file descriptors: 65536
>>>>> Available number of file descriptors: 65527
>>>>> Reserved number of file descriptors: 100
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>
Received on Sat Sep 14 2013 - 22:11:42 MDT