Amos: Thanks... this helps a lot.
I just have one question regarding... this:
client_persistent_connections off
server_persistent_connections off
If I enable this, my webservers starts having high load. The httpds in
them never closes any connection, and soon I run out of childs in the
webservers.. and my terminal servers too. I have aprox 70 users per
terminal server, if I set the persistent connections, everything goes
to hell.
Besides that, I run out of file descriptors.
Do you think that if I enable this back, the traffic generated by the
ntlm will be reduced? Do I need both options on?
Maybe I can activathe them back, increase the filedescriptors a lot,
and see if I cant force the persistent connection on the webservers to
close instead of keep alive...
What do you think?
Tnxs in advance.
Amos Jeffries <squid3_at_treenet.co.nz> escribió:
> Guido Marino Lorenzutti wrote:
>> Hi people: I have a squid using ntlm to authenticate the users. I
>> also use a external acl but im running out of ideas to make it run
>> faster.
>>
>> If I disable the ntlm everything works very well, and the cache
>> hits increses a lot.
>>
>> I found that the squids asks a lot for username and passwords to
>> the winbind, and the winbind asks everytime to my pdc. This
>> generates a lot of traffic between them and a high load on the pdc.
>>
>> Every hit on any page, squid ask to the winbind for the username
>> and password. Is this the expected behavior? Is there any way to
>> reduce (caching maybe?) this? I didn't find a solution in the
>> winbind, to stop asking the credentials to the pdc.
>>
>> I have a terminal server enviroment, so where you see 69 clients
>> the are in fact more than 500 users.
>>
>> This is my output of squidclient mgr:info
>>
>> Squid Object Cache: Version 2.6.STABLE5
>> Start Time: Fri, 05 Feb 2010 07:21:21 GMT
>> Current Time: Sat, 20 Feb 2010 03:01:08 GMT
>> Connection information for squid:
>> Number of clients accessing cache: 69
>> Number of HTTP requests received: 11790881
>> Number of ICP messages received: 0
>> Number of ICP messages sent: 0
>> Number of queued ICP replies: 0
>> Number of HTCP messages received: 0
>> Number of HTCP messages sent: 0
>> Request failure ratio: 0.00
>> Average HTTP requests per minute since start: 552.5
>> Average ICP messages per minute since start: 0.0
>> Select loop called: 154266917 times, 8.300 ms avg
>> Cache information for squid:
>> Request Hit Ratios: 5min: 50.5%, 60min: 18.3%
>> Byte Hit Ratios: 5min: 14.1%, 60min: 26.3%
>> Request Memory Hit Ratios: 5min: 0.0%, 60min: 10.0%
>> Request Disk Hit Ratios: 5min: 19.7%, 60min: 21.9%
>> Storage Swap size: 7833612 KB
>> Storage Mem size: 409452 KB
>> Mean Object Size: 19.26 KB
>> Requests given to unlinkd: 0
>> Median Service Times (seconds) 5 min 60 min:
>> HTTP Requests (All): 0.00919 0.03066
>> Cache Misses: 0.35832 0.44492
>> Cache Hits: 0.01164 0.01847
>> Near Hits: 0.33943 0.37825
>> Not-Modified Replies: 0.00286 0.00405
>> DNS Lookups: 0.09117 0.10906
>> ICP Queries: 0.00000 0.00000
>> Resource usage for squid:
>> UP Time: 1280387.834 seconds
>> CPU Time: 5238.387 seconds
>> CPU Usage: 0.41%
>> CPU Usage, 5 minute avg: 0.07%
>> CPU Usage, 60 minute avg: 0.05%
>> Process Data Segment Size via sbrk(): 561092 KB
>> Maximum Resident Size: 0 KB
>> Page faults with physical i/o: 4
>> Memory usage for squid via mallinfo():
>> Total space in arena: 561092 KB
>> Ordinary blocks: 555876 KB 13964 blks
>> Small blocks: 0 KB 0 blks
>> Holding blocks: 1744 KB 4 blks
>> Free Small blocks: 0 KB
>> Free Ordinary blocks: 5215 KB
>> Total in use: 557620 KB 99%
>> Total free: 5215 KB 1%
>> Total size: 562836 KB
>> Memory accounted for:
>> Total accounted: 511637 KB
>> memPoolAlloc calls: 1443436295
>> memPoolFree calls: 1441310223
>> File descriptor usage for squid:
>> Maximum number of file descriptors: 1024
>> Largest file desc currently in use: 268
>> Number of file desc currently in use: 261
>> Files queued for open: 0
>> Available number of file descriptors: 763
>> Reserved number of file descriptors: 100
>> Store Disk files open: 2
>> IO loop method: epoll
>> Internal Data Structures:
>> 407686 StoreEntries
>> 34175 StoreEntries with MemObjects
>> 34170 Hot Object Cache Items
>> 406635 on-disk objects
>>
>>
>> This is the output of squidclient mgr:ntlmauthenticator
>>
>> (warning: the avg service time is with NO users, when everyone is
>> using it the avg service time peeks the 1000 msec. YES 1K msec).
>>
>> NTLM Authenticator Statistics:
>> program: /usr/bin/ntlm_auth
>> number running: 200 of 200
>> requests sent: 2500498
>> replies received: 2500498
>> queue length: 0
>> avg service time: 19.24 msec
>>
>> # FD PID # Requests Flags Time Offset Request
>> 1 12 17113 168619 0.046 0 (none)
>> 2 13 17114 62644 0.055 0 (none)
>> 3 14 17118 31007 0.076 0 (none)
>> 4 15 17120 15188 0.094 0 (none)
>> 5 16 17121 5759 0.093 0 (none)
>> 6 17 17122 2845 0.071 0 (none)
>> 7 18 17124 1572 0.524 0 (none)
>> 8 19 17125 891 0.533 0 (none)
>> 9 21 17130 486 0.584 0 (none)
>> 10 22 17131 302 0.647 0 (none)
>> 11 23 17132 194 0.741 0 (none)
>> 12 24 17135 127 0.818 0 (none)
>> 13 25 17137 84 0.756 0 (none)
>> 14 26 17138 56 0.898 0 (none)
>> 15 27 17143 46 0.954 0 (none)
>> 16 28 17149 36 1.002 0 (none)
>> 17 29 17155 24 1.125 0 (none)
>> 18 30 17161 22 1.094 0 (none)
>> 19 31 17162 16 1.252 0 (none)
>> 20 32 17165 10 5.137 0 (none)
>> 21 33 17167 8 4.807 0 (none)
>> 22 34 17168 4 1.470 0 (none)
>> 23 35 17169 4 1.522 0 (none)
>> 24 36 17170 2 1.185 0 (none)
>> 25 37 17171 2 0.613 0 (none)
>> 26 38 17172 2 0.839 0 (none)
>> 27 39 17173 0 0.000 0 (none)
>>
>>
>> Any ideas in how to improve this scenario?
>>
>> This is the squid.conf
>>
>> hierarchy_stoplist cgi-bin ?
>> acl QUERY urlpath_regex cgi-bin \?
>> http_port 3128
>> #debug_options ALL,1 33,2
>> log_fqdn off
>> cache_store_log none
>> useragent_log none
>> cache_log /var/log/squid/cache_log.log
>> access_log /var/log/squid/access.log
>> error_directory /usr/share/squid/errors/Spanish
>> emulate_httpd_log on
>>
>> offline_mode off
>> strip_query_terms on
>> httpd_suppress_version_string on
>>
>> auth_param ntlm program /usr/bin/ntlm_auth
>> --helper-protocol=squid-2.5-ntlmssp --domain=MYDOMAIN
>> auth_param ntlm children 200
>> auth_param ntlm keep_alive on
>> authenticate_ttl 60 seconds
>> authenticate_ip_ttl 2 minutes
>> authenticate_cache_garbage_interval 10 seconds
>
> Seems a bit extreme to be running the garbage collection 10 seconds.
> It happens as needed on top of this.
>
> The defaults are measured in hours and user browsing times are
> usually longer than minutes.
>
>> auth_param basic program /usr/bin/ntlm_auth
>> --helper-protocol=squid-2.5-basic --domain=MYDOMAIN
>> auth_param basic children 10
>> auth_param basic realm Squid proxy-caching web server
>> auth_param basic credentialsttl 2 hours
>>
>> external_acl_type ldap_group ttl=1200 children=25 %LOGIN
>> /usr/lib/squid/squid_ldap_group -b "GROUPDN" -f "MYFILTER" -h
>> LDAPSERVER -v3 -S -P
>>
>> negative_ttl 5 minutes
>
> This is not really a good idea.
> It will extend the period of outage for every service failure and
> may hose the network access to a website for 5 minutes following a
> single client page error.
>
>> positive_dns_ttl 5 hours
>> negative_dns_ttl 1 minutes
>
> Please don't play with DNS TTLs unless you know 100% how they will
> affect things.
>
>> half_closed_clients off
>> connect_timeout 3 seconds
>> cache_dir aufs /var/spool/squid 9000 16 256
>> cache_swap_low 85
>> cache_swap_high 95
>> maximum_object_size 81920 KB
>> maximum_object_size_in_memory 300 KB
>> cache_mem 400 MB
>> fqdncache_size 6144
>> cache_replacement_policy lfuda
>> memory_replacement_policy lru
>> pipeline_prefetch off
>> client_persistent_connections off
>> server_persistent_connections off
>
> Persistent connections are REQUIRED for NTLM and related
> connection-based auth to be used efficiently.
>
> NTLM auth against the proxy requires persistent client connections,
> pass-thru to web servers requires both and the connection pinning
> feature as well.
>
>> visible_hostname myproxy.mydomain
>>
>> hierarchy_stoplist cgi-bin ?
>> acl QUERY urlpath_regex cgi-bin \?
>> no_cache deny QUERY
>>
>> acl all src all
>> acl lan_10_8 src 10.8.0.0/255.255.0.0
>
> acl lan_10_8 src 10.8.0.0/16
>
>>
>> acl webservers dst 10.8.50.220/255.255.255.255
>> 10.8.50.221/255.255.255.255 10.8.50.222/255.255.255.255
>> 10.8.50.223/255.255.255.255
>
> acl webservers dst 10.8.50.220 10.8.50.221 10.8.50.222 10.8.50.223
>
>
>>
>> acl nomsnurl dstdomain "/etc/squid/nomsn"
>>
>> acl manager proto cache_object
>> acl localhost src 127.0.0.1/255.255.255.255
>
> acl localhost src 127.0.0.1
>
>> acl SSL_ports port 443 563 1863 6667 4430
>> acl Safe_ports port 80 # http
>> acl Safe_ports port 443 563 # https, snews
>>
>> acl auth proxy_auth REQUIRED
>> acl noinet external ldap_group noinet
>> acl linuxadmin external ldap_group linuxadmin
>> acl nomsn external ldap_group nomsn
>> acl dummy src 0.0.0.0/0.0.0.0
>
> acl dummy src all
>
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE8 or 3.0.STABLE24
> Current Beta Squid 3.1.0.16
>
Received on Sat Feb 20 2010 - 05:06:00 MST
This archive was generated by hypermail 2.2.0 : Sat Feb 20 2010 - 12:00:05 MST