Re: [squid-users] Large ACLs and TCP_OUTGOING_ADDRESS

Thank you very much.
Those stats look much better than the low peak ones. Though still not
Very close to the theoretical limits Adrian published for 2.7.

Some very marginal increases may be gained from re-ordering your
http_access lines that check for WindowsUpdate. Doing the src check
before the dstdomain check (left-to-right) will save a few cycles per
request.
so: http_access Allow windowsupdate ispros
becomes: http_access Allow ispros windowsupdate

cache_store_log can be set to 'none' for less time logging debug info
you generally don't need.

You may want to experiment with the collapsed_forwarding feature. It's
designed to reduce server-side network lags so should increase the
internal speeds but depends on higher hit ratios for best effect, which
at >40% you have.

That's all I can see right now that might provide any improvement at all.

Amos

Nyamul Hassan wrote:
> Thank you Amos for your valuable input on this. Please find attached a
> snapshot of peak hour traffic.
>
> I'm also attaching the following graphs:
>
> 1. Cache Hit Rate
> 2. Client Request Rate
> 3. CPU IOWait
> 4. Service Timers
>
> I'm also attaching a copy of my cache configuration. Looking at it, can
> you suggest me if I can get any better performance than it is? I think
> the IOWait is way too high, and I am using regular commodity SATA HDDs.
>
> Any input would be greatly appreciated.
>
> Regards
> HASSAN
>
>
>
>
>
> ----- Original Message ----- From: "Amos Jeffries" <squid3_at_treenet.co.nz>
> To: "Nyamul Hassan" <mnhassan_at_usa.net>
> Cc: "Squid Users" <squid-users_at_squid-cache.org>
> Sent: Monday, November 17, 2008 07:01
> Subject: Re: [squid-users] Large ACLs and TCP_OUTGOING_ADDRESS
>
>
>>> Hi,
>>>
>>> I run squid in an ISP scenario. We have got two identically configured
>>> squid caches being load balanced among 4,000 users over a 50 Mbps link.
>>> The
>>> system runs quite well, although not without the occassional hiccups.
>>> But,
>>> there is a complain from users about not being able to access some
>>> websites
>>> because of same external IP. For this, we configured the squid.conf to
>>> have
>>> ACLs for different user blocks of /24 and have them mapped through
>>> different
>>> external IPs on each of these boxes.
>>>
>>> However, not all /24 blocks have the same number of users, and I also
>>> have
>>> lots of real IPs still lying unused. I thought about creating different
>>> ACLs for every 5 or 8 users, and then map them to different external
>>> IPs.
>>> But, having them distributed in 8 IPs in each group would mean at least
>>> 500
>>> separate ACLs and their corresponding TCP_OUTGOING_ADDRESS directives.
>>>
>>> My question is, will this affect the performance of squid? Can squid
>>> handle
>>> this?
>>
>> Depends on the ACL type. Squid should be able to handle many easily. of
>> the ACl you need; src is the fastest, next best is dstdomain, then
>> dst. So
>> for a marginal boost when combining on one line, put then in that order.
>>
>> Just look for shortcuts as you go.
>>
>>>
>>> My servers are each running on Core 2 Duo 2.33 GHz, 8 GB of RAM, 5 HDDs
>>> (1x80GB IDE for OS, 4x160GB SATA for cache), total 256GB Cache Store
>>> (64GB
>>> on each HDD). One of the server's stats are (taken at a very low user
>>> count
>>> time):
>>
>> Thank you. We are trying to collect rough capacity info for Squid
>> whenever
>> the opportunity comes up. Are you able to provide such stats around peak
>> load for our wiki?
>> The info we collect can be seen at
>> http://wiki.squid-cache.org/KnowledgeBase/Benchmarks
>>
>> Amos
>>
>>
>>
> Cache Manager menu
>
> Squid Object Cache: Version 2.7.STABLE4
>
> Connection information for squid:
> Number of clients accessing cache: 2133
> Number of HTTP requests received: 6213380
> Number of ICP messages received: 1441542
> Number of ICP messages sent: 1441550
> Number of queued ICP replies: 0
> Request failure ratio: 0.00
> Average HTTP requests per minute since start: 11488.3
> Average ICP messages per minute since start: 5330.7
> Select loop called: 78705022 times, 0.412 ms avg
> Cache information for squid:
> Request Hit Ratios: 5min: 41.7%, 60min: 43.8%
> Byte Hit Ratios: 5min: 17.5%, 60min: 16.9%
> Request Memory Hit Ratios: 5min: 16.2%, 60min: 14.4%
> Request Disk Hit Ratios: 5min: 44.2%, 60min: 43.6%
> Storage Swap size: 241613712 KB
> Storage Mem size: 4194392 KB
> Mean Object Size: 35.25 KB
> Requests given to unlinkd: 0
> Median Service Times (seconds) 5 min 60 min:
> HTTP Requests (All): 0.55240 0.55240
> Cache Misses: 0.72387 0.68577
> Cache Hits: 0.02899 0.02451
> Near Hits: 0.64968 0.64968
> Not-Modified Replies: 0.00000 0.00000
> DNS Lookups: 0.00000 0.00000
> ICP Queries: 0.00033 0.00035
> Resource usage for squid:
> UP Time: 32450.582 seconds
> CPU Time: 5725.342 seconds
> CPU Usage: 17.64%
> CPU Usage, 5 minute avg: 23.55%
> CPU Usage, 60 minute avg: 23.66%
> Process Data Segment Size via sbrk(): 775752 KB
> Maximum Resident Size: 0 KB
> Page faults with physical i/o: 2
> Memory usage for squid via mallinfo():
> Total space in arena: 1937988 KB
> Ordinary blocks: 1934155 KB 34179 blks
> Small blocks: 0 KB 0 blks
> Holding blocks: 35360 KB 8 blks
> Free Small blocks: 0 KB
> Free Ordinary blocks: 3832 KB
> Total in use: 1969515 KB 100%
> Total free: 3832 KB 0%
> Total size: 1973348 KB
> Memory accounted for:
> Total accounted: 5661786 KB
> memPoolAlloc calls: 882142632
> memPoolFree calls: 850766245
> File descriptor usage for squid:
> Maximum number of file descriptors: 65536
> Largest file desc currently in use: 8068
> Number of file desc currently in use: 7035
> Files queued for open: 4
> Available number of file descriptors: 58497
> Reserved number of file descriptors: 100
> Store Disk files open: 289
> IO loop method: epoll
> Internal Data Structures:
> 6867535 StoreEntries
> 432110 StoreEntries with MemObjects
> 430724 Hot Object Cache Items
> 6854443 on-disk objects
>
> Generated Mon, 17 Nov 2008 15:36:52 GMT, by cachemgr.cgi/2.7.STABLE4
> Cache Manager menu
>
> authenticate_cache_garbage_interval 3600 seconds
> authenticate_ttl 3600 seconds
> authenticate_ip_ttl 0 seconds
> authenticate_ip_shortcircuit_ttl 0 seconds
> acl all src 0.0.0.0/0.0.0.0
> acl manager proto cache_object
> acl localhost src 116.193.170.25
> acl localhost src 127.0.0.1
> acl ispros_proxies src 116.193.170.24/255.255.255.254
> acl proxy01 src 116.193.170.24
> acl to_localhost dst 127.0.0.0/255.0.0.0
> acl SSL_ports port 443
> acl Safe_ports port 80
> acl Safe_ports port 1025-65535
> acl Safe_ports port 443
> acl Safe_ports port 21
> acl Safe_ports port 70
> acl Safe_ports port 210
> acl Safe_ports port 280
> acl Safe_ports port 488
> acl Safe_ports port 591
> acl Safe_ports port 777
> acl CONNECT method CONNECT
> acl windowsupdate dstdomain download.windowsupdate.com
> acl windowsupdate dstdomain www.download.windowsupdate.com
> acl windowsupdate dstdomain wustat.windows.com
> acl windowsupdate dstdomain c.microsoft.com
> acl windowsupdate dstdomain .update.microsoft.com
> acl windowsupdate dstdomain windowsupdate.microsoft.com
> acl windowsupdate dstdomain crl.microsoft.com
> acl windowsupdate dstdomain redir.metaservices.microsoft.com
> acl windowsupdate dstdomain images.metaservices.microsoft.com
> acl wuCONNECT dstdomain www.update.microsoft.com
> acl ...........
> ...
> ...
> ...
> acl ...........
> acl apache rep_header Server ^Apache
> http_access Allow manager localhost
> http_access Allow manager proxy01
> http_access Deny manager
> http_access Deny !Safe_ports
> http_access Deny CONNECT !SSL_ports
> http_access Allow CONNECT wuCONNECT ispros
> http_access Allow windowsupdate ispros
> http_access Allow CONNECT wuCONNECT ggnn_real
> http_access Allow windowsupdate ggnn_real
> http_access Allow CONNECT wuCONNECT ggnn_pk64
> http_access Allow windowsupdate ggnn_pk64
> http_access Allow CONNECT wuCONNECT ggnn_pk128
> http_access Allow windowsupdate ggnn_pk128
> http_access Allow CONNECT wuCONNECT ggnn_pk256
> http_access Allow windowsupdate ggnn_pk256
> http_access Allow CONNECT wuCONNECT ggnn_pk512
> http_access Allow windowsupdate ggnn_pk512
> http_access Allow CONNECT wuCONNECT ggnn_pknight
> http_access Allow windowsupdate ggnn_pknight
> http_access Allow ...
> ...
> ...
> ...
> http_access Allow ...
> http_access Allow localhost
> http_access Deny all
> http_reply_access Allow all
> icp_access Allow ispros_proxies
> ident_lookup_access Deny all
> reply_body_max_size 0 Allow all
> follow_x_forwarded_for Deny all
> acl_uses_indirect_client on
> delay_pool_uses_indirect_client on
> log_uses_indirect_client on
> ssl_unclean_shutdown off
> sslproxy_version 1
> http_port 0.0.0.0:3128 transparent protocol=http
> tcp_outgoing_address ...
> ...
> ...
> ...
> tcp_outgoing_address ...
> zph_mode off
> zph_local 0
> zph_sibling 0
> zph_parent 0
> zph_option 136
> cache_peer ... Sibling 3128 3130 proxy-only
> dead_peer_timeout 10 seconds
> hierarchy_stoplist cgi-bin
> hierarchy_stoplist ?
> cache_mem 4294967296 bytes
> maximum_object_size_in_memory 65536 bytes
> memory_replacement_policy lru
> cache_replacement_policy lru
> cache_dir aufs /cachestore/cache1 65536 16 256
> cache_dir aufs /cachestore/cache2 65536 16 256
> cache_dir aufs /cachestore/cache3 65536 16 256
> cache_dir aufs /cachestore/cache4 65536 16 256
> store_dir_select_algorithm least-load
> max_open_disk_fds 0
> minimum_object_size 0 bytes
> maximum_object_size 1073741824 bytes
> cache_swap_low 90
> cache_swap_high 95
> update_headers on
> access_log /var/log/squid/access.log squid
> logfile_daemon /usr/lib/squid/logfile-daemon
> cache_log /var/log/squid/cache.log
> cache_store_log /var/log/squid/store.log
> logfile_rotate 10
> emulate_httpd_log off
> log_ip_on_direct on
> mime_table /etc/squid/mime.conf
> log_mime_hdrs off
> pid_filename /var/run/squid.pid
> debug_options ALL,1
> log_fqdn off
> client_netmask 255.255.255.255
> strip_query_terms on
> buffered_logs off
> netdb_filename /var/log/squid/netdb.state
> ftp_user Squid@
> ftp_list_width 32
> ftp_passive on
> ftp_sanitycheck on
> ftp_telnet_protocol on
> diskd_program /usr/lib/squid/diskd-daemon
> unlinkd_program /usr/lib/squid/unlinkd
> storeurl_rewrite_children 5
> storeurl_rewrite_concurrency 0
> url_rewrite_children 5
> url_rewrite_concurrency 0
> url_rewrite_host_header on
> redirector_bypass off
> location_rewrite_children 5
> location_rewrite_concurrency 0
> max_stale 604800 seconds
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
> quick_abort_min 16 KB
> quick_abort_max 16 KB
> quick_abort_pct 95
> read_ahead_gap 16384 bytes
> negative_ttl 300 seconds
> positive_dns_ttl 21600 seconds
> negative_dns_ttl 60 seconds
> range_offset_limit 0 bytes
> minimum_expiry_time 60 seconds
> store_avg_object_size 13 KB
> store_objects_per_bucket 20
> request_header_max_size 20480 bytes
> reply_header_max_size 20480 bytes
> request_body_max_size 0 bytes
> via on
> cache_vary on
> broken_vary_encoding Allow apache
> collapsed_forwarding off
> refresh_stale_hit 0 seconds
> ie_refresh off
> vary_ignore_expire off
> request_entities off
> relaxed_header_parser on
> server_http11 off
> ignore_expect_100 off
> forward_timeout 240 seconds
> connect_timeout 60 seconds
> peer_connect_timeout 30 seconds
> read_timeout 900 seconds
> request_timeout 300 seconds
> persistent_request_timeout 120 seconds
> client_lifetime 86400 seconds
> half_closed_clients on
> pconn_timeout 60 seconds
> ident_timeout 10 seconds
> shutdown_lifetime 30 seconds
> cache_mgr ...
> mail_from ...
> mail_program mail
> cache_effective_user squid
> cache_effective_group squid
> httpd_suppress_version_string off
> visible_hostname ...
> umask 23
> announce_period 31536000 seconds
> announce_host tracker.ircache.net
> announce_port 3131
> httpd_accel_no_pmtu_disc off
> delay_pools 0
> delay_initial_bucket_level 50
> wccp_router 0.0.0.0
> wccp_version 4
> wccp2_rebuild_wait on
> wccp2_forwarding_method 1
> wccp2_return_method 1
> wccp2_assignment_method 1
> wccp2_service standard 0
> wccp2_weight 10000
> wccp_address 0.0.0.0
> wccp2_address 0.0.0.0
> client_persistent_connections on
> server_persistent_connections off
> persistent_connection_after_error off
> detect_broken_pconn off
> digest_generation on
> digest_bits_per_entry 5
> digest_rebuild_period 3600 seconds
> digest_rewrite_period 3600 seconds
> digest_swapout_chunk_size 4096 bytes
> digest_rebuild_chunk_percentage 10
> snmp_port 3401
> snmp_access Allow snmp_local localhost
> snmp_access Deny all
> snmp_incoming_address 0.0.0.0
> snmp_outgoing_address 255.255.255.255
> icp_port 3130
> log_icp_queries on
> udp_incoming_address 0.0.0.0
> udp_outgoing_address 255.255.255.255
> icp_hit_stale off
> minimum_direct_hops 4
> minimum_direct_rtt 400
> netdb_low 900
> netdb_high 1000
> netdb_ping_period 300 seconds
> query_icmp off
> test_reachability off
> icp_query_timeout 0
> maximum_icp_query_timeout 2000
> minimum_icp_query_timeout 5
> mcast_icp_query_timeout 2000
> icon_directory /usr/share/icons
> global_internal_static on
> short_icon_urls off
> error_directory /usr/share/errors/English
> err_html_text nonhierarchical_direct on
> prefer_direct off
> ignore_ims_on_miss off
> max_filedescriptors 65536
> tcp_recv_bufsize 0 bytes
> incoming_rate 30
> check_hostnames on
> allow_underscore on
> dns_retransmit_interval 5 seconds
> dns_timeout 120 seconds
> dns_defnames off
> hosts_file /etc/hosts
> dns_testnames netscape.com
> dns_testnames internic.net
> dns_testnames nlanr.net
> dns_testnames microsoft.com
> ignore_unknown_nameservers on
> ipcache_size 1024
> ipcache_low 90
> ipcache_high 95
> fqdncache_size 1024
> memory_pools on
> memory_pools_limit 5242880 bytes
> forwarded_for on
> cachemgr_passwd disable shutdown offline_toggle
> cachemgr_passwd XXXXXXXXXX all
> client_db on
> reload_into_ims off
> maximum_single_addr_tries 1
> retry_on_error off
> as_whois_server whois.ra.net
> offline_mode off
> uri_whitespace strip
> coredump_dir /var/cache
> balance_on_multiple_ip on
> pipeline_prefetch off
> high_response_time_warning 0
> high_page_fault_warning 0
> high_memory_warning 0 bytes
> sleep_after_fork 0
> zero_buffers on
> windows_ipaddrchangemonitor on
>
> Generated Mon, 17 Nov 2008 15:48:58 GMT, by cachemgr.cgi/2.7.STABLE4
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>

-- 
Please be using
   Current Stable Squid 2.7.STABLE5 or 3.0.STABLE10
   Current Beta Squid 3.1.0.2