Re: [squid-users] Allow MSN messenger

On 09/02/11 08:44, David Touzeau wrote:
>
>
>
> Le mardi 08 février 2011 à 13:35 -0500, Chad Naugle a écrit :
>> Which all looks ok, but is there an "http_access" that allows anything
>> other than the "CONNECT" method, such as:
>>
>> http_access allow MSN_hosts
>> http_access allow MSN_domains
>> http_access allow MSN_net
>>
>> Not to mention any other sites / hosts / ports (Such as port 80) before
>> the "http_access deny all", because whenever stacking ACL's there is an
>> implied "AND" operator, so each line only works like this:
>>
>> "Method is CONNECT" AND "Ports" AND "Destination is<list>"
>>
>> Otherwise DENY ALL is the likely culprit.
>>
>>
>>>>> David Touzeau<david_at_touzeau.eu> 2/8/2011 1:22 PM>>>
>>
>> Dear i Use squid 3.1.10 and i would like to allow MSN messenger pass
>> trough squid
>>
>> According wikis i did this :
>>
>> # Permit MSN
>> acl MSN_ports port 1863 443 1503
>> acl MSN_domains
>> dstdomain .microsoft.com .hotmail.com .live.com .msft.net .msn.com
>> .passport.com
>> acl MSN_hosts dstdomain messenger.hotmail.com
>> acl MSN_nets dst 207.46.111.0/255.255.255.0
>> acl MSN_methods method CONNECT
>>
>>
>> http_access allow MSN_methods MSN_ports MSN_hosts
>> http_access allow MSN_methods MSN_ports MSN_domains
>> http_access allow MSN_methods MSN_ports MSN_net
>>
>> But MSN still did want to connect with these errors:
>>
>> 192.168.82.173 - - [08/Feb/2011:10:48:38 -04-30] "POST
>> http://www.sqm.microsoft.com/sqm/messenger/sqmserver.dll HTTP/1.1" 403
>> 1662 TCP_MISS:DIRECT
>> 192.168.82.173 - - [08/Feb/2011:10:48:39 -04-30] "POST
>> http://www.sqm.microsoft.com/sqm/messenger/sqmserver.dll HTTP/1.1" 403
>> 1662 TCP_MISS:DIRECT
>> 192.168.82.173 - - [08/Feb/2011:10:48:39 -04-30] "POST
>> http://www.sqm.microsoft.com/sqm/messenger/sqmserver.dll HTTP/1.1" 403
>> 1662 TCP_MISS:DIRECT
>>
>> Where i'm wrong ??
>>
>> Best regards
>>
>>
>>
>
> This is the entire content of the squid.conf if you see something wrong,
> let me know....
>
>
> acl localhost src 127.0.0.1/32
> acl to_localhost dst 127.0.0.1/32
> acl manager proto cache_object
> auth_param basic credentialsttl 2 hour
> authenticate_ttl 1 hour
> authenticate_ip_ttl 60 seconds

Setting credentials timeouts but not otherwise configuring or using auth
at all. The above auth bits can all be dropped until needed.

> #--------- TWEEKS PERFORMANCES
> # http://blog.last.fm/2007/08/30/squid-optimization-guide
> memory_pools off

Hmm, this is optimization only on 64-bit machines with broken default
alloc implementations and some fairly rare people who suffer under
strange memory leaks problems we fail to replicate.

What it does is cause Squid to call out to the OS for every individual
piece of memory used, instead of allocating larger whole swap-page sized
chunks.

May be worth experimenting and doing your own measurements.

> quick_abort_min 0 KB
> quick_abort_max 0 KB
> log_icp_queries off
> client_db off
> buffered_logs on
> half_closed_clients off
>
> #--------- squidGard
>
> #IS C-ICAP enabled = 1
> redirect_program /usr/bin/squidGuard
> redirect_children 20

Style fix:
   That directive was renamed "url_rewrite_program" and
"url_rewrite_children" some time ago.

>
> #--------- SQUID PARENTS (feature not enabled)
>
> #--------- acls
> acl blockedsites url_regex "/etc/squid3/squid-block.acl"
> acl CONNECT method CONNECT
> acl purge method PURGE
> acl FTP proto FTP
> acl multimedia_rep rep_mime_type -i ^video/x-ms-asf$
> acl multimedia_rep rep_mime_type -i ^application/vnd.ms.wms-hdr.asfv1$
> acl multimedia_rep rep_mime_type -i ^application/x-mms-framed$
> acl multimedia_rep rep_mime_type -i ^image/
> acl multimedia_rep rep_mime_type -i ^video
> acl multimedia_rep rep_mime_type -i ^audio
> acl multimedia_rep rep_mime_type -i ^application/x-dvi$
> acl multimedia_rep rep_mime_type -i ^application/x-isoview
> acl multimedia_browsers browser -i ^Windows-Media-Player.* -i
> ^.*player.*

The regex library Squid uses places an implicit ".*" at the beginning
and end of the pattern unless you manually add ^ and $ anchors. So you
can drop the trailing .*

Also, -i (case ignored) ^.*player contains ^Windows-Media-Player. You
can drop the ^Windows-Media-Player one completely for a doubling of
speed on that ACL test.

> acl bigfiles_types urlpath_regex -i \.deb$
> acl bigfiles_types urlpath_regex -i \.rpm$
> acl bigfiles_types urlpath_regex -i \.iso$
> acl bigfiles_types urlpath_regex -i \.tar\.gz$
> acl bigfiles_types urlpath_regex -i \.gz$
> acl bigfiles_types urlpath_regex -i \.bz$
> acl bigfiles_types urlpath_regex -i \.tar$
> acl bigfiles_types urlpath_regex -i \.cue$
> acl bigfiles_types urlpath_regex -i \.nrg$
> acl bigfiles_types urlpath_regex -i \.crf$
> acl bigfiles_types urlpath_regex -i \.bwi$
> acl bigfiles_types urlpath_regex -i \.bwt$
> acl bigfiles_types urlpath_regex -i \.lcd$
> acl bigfiles_types urlpath_regex -i \.ccd$
> acl bigfiles_types urlpath_regex -i \.mdf$
> acl bigfiles_types urlpath_regex -i \.mds$
> acl bigfiles_types urlpath_regex -i \.vcd$
> acl bigfiles_types urlpath_regex -i \.cif$
> acl bigfiles_types urlpath_regex -i \.vdi$
> acl bigfiles_types urlpath_regex -i \.img$

Optimization:
   each test has to be run individually. compacting those down to one
pattern will allow the library to do some small optimizations and test
faster.
    acl bigfiles_types urlpath_regex -i \.(lcd|ccd|mdf|mds)$

Also, these days it is very popular to use query string parameters.
Which will break the ACL which expect filename to be last in the URL.

This pattern will catch a file extension even if it is followed by
"?garbage" or inserted in the query string itself:
  acl bigfiles_types urlpath_regex -i \.(lcd|ccd|mdf|mds)((\?|&).*)?$

>
> acl office_network src 192.168.82.0/24
>
>
> #--------- MAIN RULES...
> always_direct allow FTP
> # --------- SAFE ports
> acl Safe_ports port 1443 #FortiPartner
> acl Safe_ports port 80 #http
> acl Safe_ports port 443 #https
> acl Safe_ports port 21 #ftp
> acl Safe_ports port 1863 #MSN
> acl Safe_ports port 20 #ftp-data#
> acl SSL_ports port 9000 #Artica
> acl SSL_ports port 443 #HTTPS
> acl SSL_ports port 563 #https, snews
> acl SSL_ports port 6667 #tchat
> acl SSL_ports port 4343 #FortiGate
>
> # Permit MSN
> acl MSN_ports port 1863 443 1503
> acl MSN_domains
> dstdomain .microsoft.com .hotmail.com .live.com .msft.net .msn.com .passport.com
> acl MSN_hosts dstdomain messenger.hotmail.com
> acl MSN_nets dst 207.46.111.0/255.255.255.0
> acl MSN_methods method CONNECT
>
> acl MULTIMEDIA rep_mime_type -i ^(audio\/x-mpegurl|audio\/mpeg|video
> \/flv|video\/x-flv|application\/x-shockwave-flash|audio\/ogg|video\/ogg|
> application\/ogg)
> $

Bit simpler: (just a manual compaction on my part)

  -i
^(audio\/(x-)?mpeg(url)?|video\/(x-)?flv|application\/x-shockwave-flash|.*\/ogg)$

* three variations of MPEG audio
* two variations of flash video
* one variation of flash animation
* any type using ogg encapsulation.

>
> # --------- RULES DEFINITIONS
> http_access deny blockedsites

"blockedsites" can be dropped below the string MSN rules unless you are
blocking some of the MSN sites from the MSN "friend sites" browser.

> http_access allow MSN_methods MSN_ports MSN_hosts
> http_access allow MSN_methods MSN_ports MSN_domains
> http_access allow MSN_methods MSN_ports MSN_nets
> http_access allow localhost
> http_access allow manager localhost
> ttp_access allow purge localhost
> http_access deny purge
> url_rewrite_access deny localhost
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports

The "allow localhost", manager and purge rules should probably drop down
to here.

"blockedsites" rule could probably go right here for a much reduced CPU
workload on garbage and DoS attacks. Of the types caught by CONNECT and
Safe_ports security rules.

> http_access allow office_network
> http_access deny to_localhost
> http_access deny all
> # --------- ICAP Services.(1 service(s))
> # --------- icap_service C-ICAP mode 3.1.x
> icap_service service_antivir respmod_precache routing=on bypass=on
> icap://127.0.0.1:1345/srv_clamav
>
>
>
> # --------- adaptation for C-ICAP service
>
> adaptation_service_set class_antivirus service_antivir
> adaptation_access class_antivirus deny MULTIMEDIA
> adaptation_access class_antivirus allow all
>
>
> icap_enable on
> icap_preview_size 128
> icap_service_failure_limit -1
> icap_preview_enable on
> icap_send_client_ip on
> icap_send_client_username on
> icap_client_username_header X-Authenticated-User
> icap_client_username_encode on
>
>
>
>
> # --------- ident_lookup_access
> hierarchy_stoplist cgi-bin ?
>
> # --------- General settings
> visible_hostname proxyweb
>
>
> # --------- time-out
> dead_peer_timeout 10 seconds
> dns_timeout 2 minutes

in 3.1 DNS timeout of 2 minutes is default.

> connect_timeout 1600 seconds
> persistent_request_timeout 3 minutes
> pconn_timeout 1600 seconds
>
>
> # --------- Objects limits
> request_body_max_size 5 MB
> request_header_max_size 64 KB
> maximum_object_size 300 MB
> minimum_object_size 0 KB
> maximum_object_size_in_memory 8 KB
>
>
> #http/https ports
> http_port 3128 ssl-bump cert=/etc/squid3/ssl/cacert.pem
> key=/etc/squid3/ssl/privkey.pem
>
>
> # --------- SSL Rules
> ssl_bump allow all
> always_direct allow all
>
>
> # --------- Caches
> cache_effective_user squid
> cache_effective_group squid
> #cache_replacement_policy heap LFUDA
> cache_mem 204 MB
> cache_swap_high 90
> cache_swap_low 95
> # --------- DNS and ip caches
> ipcache_size 51200
> ipcache_low 90
> ipcache_high 95
> fqdncache_size 51200
>
>
> # --------- SPECIFIC DNS SERVERS
> dns_nameservers 192.168.82.2
>
> #--------- FTP specific parameters
> ftp_list_width 50
> ftp_passive on
> ftp_sanitycheck off
> ftp_epsv off

Did EPSV cause problems for you?
  This directive is for things like sending two EPSV requests through a
certain router model causes it to buffer overflow.

> ftp_epsv_all off
> ftp_telnet_protocol off
>
> debug_options ALL,1
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440

Missing:
   refresh_pattern -i (/cg-bin/|\?) 0 0% 0

FYI:
  This is carefully crafted pattern to match certain dynamic requests
and responses according to a exception sub-clause of RFC 2616 which
allows Squid to cache dynamic responses *unless* they are missing
Cache-Control AND the Expires header.

At most the safe changes that may be made are additions to include other
dynamic page signatures. Though only old CGI apps are confirmed to have
the relevant legacy problems.

> refresh_pattern . 0 20% 4320
> icp_port 3130
>
> Logs-------------------------------------------------
> #fqdn is disabled to provide IP addresses to filters
> log_fqdn off
> coredump_dir /var/squid/cache
> cache_store_log /var/log/squid/store.log
> cache_log /var/log/squid/cache.log
> pid_filename /var/run/squid.pid
> access_log none manager
> access_log /var/log/squid/access.log common
> access_log /var/log/squid/sarg.log squid
> icap_log /var/log/squid/icap_access.log
>
> cache_dir ufs /var/cache/squid 20000 16 256
> # --------- OTHER CACHES
>

20 GB on UFS (slowest available disk storage IO system) just seems
wrong. If this is a BSD derived system used diskd, if its Linux use aufs.

I'm not aware of any remaining problems in 3.1.10 the aufs/diskd areas.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.11
   Beta testers wanted for 3.2.0.4