On 27/07/2013 11:10 p.m., Ahmad wrote:
> hi ,
> i have 2 squid machine and both of them has the same load of users and BW
>
> but of them has 32 G ram and the other has 48 G ram
>
> here is a comparision between them :
>
> 1-server of 32 G ram
> squidclient -p 2222 mgr:info | grep 'file des'
> Maximum number of file descriptors: 65536
> Largest file desc currently in use: 52462
> Number of file desc currently in use: 39277
> Available number of file descriptors: 26258
> Reserved number of file descriptors: 100
>
>
>
> 2-server of 48 G ram
>
> squidclient -p 2222 mgr:info | grep 'file des'
> Maximum number of file descriptors: 65536
> Largest file desc currently in use: 52497
> Number of file desc currently in use: 50191
> Available number of file descriptors: 15345
> Reserved number of file descriptors: 100
>
>
>
> note that server 1 has larger number of server 2 relative to Number of
> file desc currently in use ,
>
> also , i note that server 1 is making better than server 2 in bw saving .
>
> my question is , does this relative to ram ???
Yes and no.
It is relative to RAM, but not close enough to be an indicator or Squid.
FD are used for all of listening sockets, files access, client/server
sockets, UDP sockets, IDS sockets. Each of these has very different
memory usage requirements and some have quite a bit of internal variance
relative to the transaction type underway as well.
> another issue , how do i know that i need to increase my rams ???
If you are close to filling it and reducing cache_mem is not an option.
>
> also i want my squid.conf to be checked for any thing could be enhance my
> server .
Which particular version of Squid?
> here is squid.conf file and it is identical to both servers .
>
> =========================================================================
> here is squid.conf file :
> include /etc/squid3/vc_squid_3.conf
What is in that file may affect performance. Since you have not provided
it I am going to have to assume it does not.
> ####################################
> ############################################################
> acl localnetz src 10.11.0.0/16
> http_access deny localnetz
> #################################################################
> ###############Redirection of private ips to webpage#############
> #################################################################
> acl localnetx src 10.12.0.0/16
> acl localnety dst 192.168.70.2/32
> http_access allow localnetx localnety
> http_access deny localnetx
> ############filtering without squidguard###########
> acl blockkeywords dstdomain "/etc/squid3/newsquid-porn.acl"
> http_access deny blockkeywords
> ##########################################################
> acl pornreg dstdom_regex "/etc/squid3/squid-regex.acl"
> http_access deny pornreg
> ############################################################
> ####################################################
> ####################################################
> #acl NO-CACHE-SITES dstdomain "/etc/squid3/not-to-cache-sites.txt"
> #no_cache deny NO-CACHE-SITES
> ################################################################
> #
> # Recommended minimum configuration:
> #
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32 ::1
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
> #################################################################################
> # Example rule allowing access from your local networks.
> # Adapt to list your (internal) IP networks from where browsing
> # should be allowed
> acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
> acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
> acl localnet src fc00::/7 # RFC 4193 local private network range
> acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged)
> machines
> acl mysubnet src 185.6.16.0/22 176.58.64.0/20 188.161.104.0/21
> 213.244.82.129/32
> http_access allow mysubnet
> ###################################################################################
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> ######################################################
> #
> # Recommended minimum Access Permission configuration:
> #
Sigh. This block from here ...
> # Only allow cachemgr access from localhost
> http_access allow manager localhost
> http_access deny manager
> ################################################
> # Deny requests to certain unsafe ports
> http_access deny !Safe_ports
> ###################################################################
> # Deny CONNECT to other than secure SSL ports
> http_access deny CONNECT !SSL_ports
> #####################################################################
> # We strongly recommend the following be uncommented to protect innocent
> # web applications running on the proxy server who think the only
> # one who can access services on "localhost" is a local user
> #http_access deny to_localhost
... to here is the default security settings for Squid. Protecting you
against quite a large number of malicious traffic and designed
explicitly to do so at the highest possible speed.
By placing your own http_access rules above it you are making Squid run
the processing for all those local rules *before* it detects things such
as an attempt to send Spam email through your proxy. Any one or more of
your local rules may even be *allowing* such an attack to take place
accidentally.
==> such as any one of the machines in "mysubnet" have free access to
do so if they become infected with a spambot.
Ideally for any forward-proxy or interception-proxy like yours local
rules should all be placed underneath the disclaimer here:
> ##################################################################
> #
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> #
> ###################################################################
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
> http_access allow localhost
> ##############################################################
> # And finally deny all other access to this proxy
> http_access deny all
> ###########################################################
> # Squid normally listens to port 3128
> http_port 1xxxxx
> http_port xxxxxx
> http_port xxxxx tproxy
> ###########################################################
> # We recommend you to use at least the following line.
> hierarchy_stoplist cgi-bin ?
Recommendation has changed. You can remove that line. The setting
appears to be useless in your configuration anyway.
> ###############################################################
> # Uncomment and adjust the following to add a disk cache directory.
> cache_dir aufs /cache1 90000 32 256
> cache_dir aufs /cache2 90000 32 256
> cache_dir aufs /cache3 90000 32 256
> ###########################################################
> # Leave coredumps in the first cache dir
> coredump_dir /var/spool/squid
> ###############################################################
> # Add any of your own refresh_pattern entries above these.
> ########### Refresh_pattern Config:
> #####################################################################################################################
> #for the music:
> refresh_pattern -i
> \.(mp2|mp3|mid|midi|mp[234]|wav|ram|ra|rm|au|3gp|m4r|m4a)(\?.*|$) 5259487
> 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache
> ignore-private
>
> #for the movies:
> refresh_pattern -i
> \.(mpg|mpeg|mp4|m4v|mov|avi|asf|wmv|wma|dat|flv|swf)(\?.*|$) 5259487 999%
> 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache
> ignore-private
>
> #for pictures:
> refresh_pattern -i
> \.(jpeg|jpg|jpe|jp2|gif|tiff?|pcx|png|bmp|pic|ico)(\?.*|$) 5259487 999%
> 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache
> ignore-private
>
> #for MS docs:
> refresh_pattern -i
> \.(chm|dll|doc|docx|xls|xlsx|ppt|pptx|pps|ppsx|mdb|mdbx)(\?.*|$) 5259487
> 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache
> ignore-private
I *really* hope you know what you are doing here. Ignoring the "private"
cache control on documents and email archives will cause them to
potentially be delivered to multiple people. Some of who may not be
wanting or supposed to receive them. With "dll" you are potentially even
sending one persons login credentials to everyone.
> #for various other docs:
> refresh_pattern -i \.(txt|conf|cfm|psd|wmf|emf|vsd|pdf|rtf|odt)(\?.*|$)
> 5259487 999% 5259487 override-expire ignore-reload reload-into-ims
> ignore-no-cache ignore-private
>
> #for the well-known compressed/excutable files:
> refresh_pattern -i
> \.(class|jar|exe|gz|bz|bz2|tar|tgz|zip|gzip|arj|ace|bin|cab|msi|rar)(\?.*|$)
> 5259487 999% 5259487 override-expire ignore-reload reload-into-ims
> ignore-no-cache ignore-private
>
> #for various client-side Web docs:
> refresh_pattern -i \.(htm|html|mhtml|css|js)(\?.*|$) 1440 90% 86400
> override-expire ignore-reload reload-into-ims
FMI: Which softwares or sites traffic is causing you to add
"ignore-private" to most of those patterns? That setting in particular
is a serious breach of the HTTP protocol. The software must be badly
broken to explicitly send "Cache-Control:private" on non-private
information, and you will definitely be causing major problems to
security systems like Captchas.
> #####################################################################################################################
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
> ###############################################################
> cache_mem 1000 MB
> ########### WCCP2 Config#############
> wccp2_router xxxxxx
> wccp_version 2
> wccp2_forwarding_method 2
> wccp2_return_method 2
> wccp2_assignment_method 2
> wccp2_service dynamic 60
> wccp2_service_info 60 protocol=tcp flags=src_ip_hash priority=250 ports=80
> wccp2_service dynamic 70
> wccp2_service_info 70 protocol=tcp flags=dst_ip_hash,ports_source
> priority=250 ports=80
> ###################################################################
> dns_nameservers xxxxx xxxxx 8.8.8.8
> cache_effective_user proxy
> cache_effective_group proxy
> visible_hostname squid
visible_hostname is supposed to be a FQDN and unique per-machine. If you
have configured each machines hostname settings correctly and registered
the name in DNS you do not need the above setting at all. Squid will
auto-detect the one appropriate to the machine it is running on.
> ############################################################
> ########### Performance Related Config:
> relaxed_header_parser on
Well, it is related to performance. But the OFF setting is what makes
reject or ignore traffic with syntax errors. The rejection happens fast,
so avoiding time wasted processing those transactions. But it may be
super-annoying for any of your clients using slightly broken software
which "works" through other proxies.
In general there are far too many broken software out there and this
needs to be left at its default setting (ON or remove from the config
completely), but if you are able to turn it OFF that would be wonderful
news.
> vary_ignore_expire on
If you have a Squid-3.1 or later that can be removed. Those versions do
not send HTTP/1.0 to servers.
> ##########################################
> memory_replacement_policy heap GDSF
> cache_replacement_policy heap LFUDA
> maximum_object_size_in_memory 64 KB
You have 1000 MB of cache_mem available. It may be possible to tune this
higher for faster HIT on larger objects. But you need to tune that
yourself with testing of how it affects the traffic and HIT rates.
> ###########################################
> ipcache_size 2048
> ipcache_low 96
> ipcache_high 99
> memory_pools off
YMMV on this one. The pools prevent Squid having to cycle through the
system malloc systems on every byte of memory allocated or deallocated,
and there are some things with very small sizes being
allocated/deallocated constantly in Squid. There is some overhead inside
Squid CPU usage instead.
If you must disable this you will be worth building Squid against a
highly optimized allocator library. The system default one is usually
not very great for high performance. Tcalloc was the best last time I
had to go through and rate them (a few years ago now so things may have
changed).
> pipeline_prefetch on
> ############################################
> httpd_suppress_version_string on
> server_persistent_connections on
> client_persistent_connections on
> pconn_timeout 2 minutes
> persistent_request_timeout 1 minute
> ###########################################
> forwarded_for on
> max_filedescriptors 65536
> max_open_disk_fds 65536
YMMV, but I personally do not think it is a good idea to have the total
and disk I/O FD limits the same. There are multiple network uses which
will need to be using FD so you risk the disks grabbing them _all_ if a
large disk garbage collection is running when you reached the max. That
would block opening new client and server connections completely until
the disk I/O was completed.
> relaxed_header_parser on
You already configured that to ON.
> reload_into_ims on
> client_lifetime 15 minutes
> read_timeout 5 minutes
> request_timeout 1 minutes
> ie_refresh on
> ignore_expect_100 on
This will be a drag on anyone using HTTP Expect functionality. The best
cure there is to upgrade to Squid-3.2 and drop this setting entirely.
Second-best is to leave it OFF, which is the default in Squid-3.1. Yes
clients will get 417 responses out of Squid but they are expected to
deal with those properly and being nice by ignoring Expect headers will
only result in traffic hanging while all involved FD are locked up
unusable for undefined length of time when the client hits a timeout
(which it may never do).
> vary_ignore_expire on
> ###############################
> ################################
> httpd_suppress_version_string on
> server_persistent_connections on
> client_persistent_connections on
> pconn_timeout 2 minutes
> persistent_request_timeout 1 minute
> #shutdown_lifetime 20 seconds
> #############################
> cache_swap_low 96
> cache_swap_high 99
> ###############################
>
HTH
Amos
Received on Sat Jul 27 2013 - 14:40:03 MDT
This archive was generated by hypermail 2.2.0 : Sun Jul 28 2013 - 12:00:05 MDT