RE: [squid-users] Reverse proxy setup with squid 2.6+

Below is relavant portions of the access.log and a netstat listing. I have altered domain information since I don't have a test URL that will work with this configuration. Please assume that www.foobar.com resolves properly, and is a real site on the Iternet. The initial 503 errors are because at first I attempted to have squid deal with the connections without DNS resolvers, but it was unhappy that it couldn't resolve www.foobar.com, timed-out, and reported that error. When I re-enabled the resolvers, the site would load, but my observation of netstat shows that squid seems to be resolving the domain and sending HTTP connections to that IP address. I can confirm that The public 204.x.x.10 address is the proper IP address for my www.foobar.com.

----- custom.log -----
172.16.x.x - - [28/Feb/2008:12:53:29 -0500] "GET http://www.foobar.com/ HTTP/1.1" 503 1492 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:12:53:29 -0500] "GET http://www.foobar.com/favicon.ico HTTP/1.1" 503 1514 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:24 -0500] "GET http://www.foobar.com/ HTTP/1.1" 200 13296 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:24 -0500] "GET http://www.foobar.com/css/default.css HTTP/1.1" 200 5462 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:24 -0500] "GET http://www.foobar.com/css/www.foobar.com/www.foobar.com.css HTTP/1.1" 200 17880 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_REFRESH_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:24 -0500] "GET http://www.foobar.com/j/formbutton.js HTTP/1.1" 200 713 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:24 -0500] "GET http://www.foobar.com/j/popup_library.js HTTP/1.1" 200 1113 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/j/IframeHelper.js HTTP/1.1" 200 2324 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/img/1spacer.gif HTTP/1.1" 200 455 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/j/popup_library.js HTTP/1.1" 200 1102 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_REFRESH_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/img/www.foobar.com/logo.gif HTTP/1.1" 200 2668 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/img/www.foobar.com/tab_flights.gif HTTP/1.1" 200 1703 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/img/www.foobar.com/tab_cars.gif HTTP/1.1" 200 1558 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/j/date.js HTTP/1.1" 200 10956 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/img/www.foobar.com/tab_hotels.gif HTTP/1.1" 200 1667 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT
172.16.x.x - - [28/Feb/2008:13:09:25 -0500] "GET http://www.foobar.com/img/www.foobar.com/tab_cruises_a.gif HTTP/1.1" 200 1580 "http://www.foobar.com/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" TCP_MISS:DIRECT

Except from a netstat -an:

tcp 0 0 10.x.x.90:80 172.16.x.x:4356 ESTABLISHED
tcp 0 0 10.x.x.90:80 172.16.x.x:4357 ESTABLISHED
tcp 0 0 10.x.x.90:80 172.16.x.x:4346 TIME_WAIT
tcp 0 0 10.x.x.90:80 172.16.x.x:4348 TIME_WAIT
tcp 0 0 10.x.x.90:33874 204.x.x.10:80 ESTABLISHED
tcp 0 0 10.x.x.90:33871 204.x.x.10:80 ESTABLISHED

-----Original Message-----
From: shaun p martin [mailto:extremelife@gmail.com]
Sent: Thursday, February 28, 2008 11:54 AM
To: Russ Gnann
Cc: squid-users@squid-cache.org
Subject: Re: [squid-users] Reverse proxy setup with squid 2.6+

can you attach part of your access.log (custom.log in your case) that
shows this behavior?

cheers~

On Thu, Feb 28, 2008 at 9:35 AM, Russ Gnann <Rgnann@revelex.com> wrote:
> We are currently looking up upgrade our squid servers from 2.5 to 2.6 or higher. In our current configuration, we send requests to the origin servers to a single IP address that points to a load balancer which is associated with a pool of web servers. In 2.5, this is easy to do with the httpd_accel_* directives, but in 2.6 I know that those directives have been replaced by the http_port directive with accel, vhost, vport, etc. options. I have supplied the squid.conf we are attempting to use below with a build of 2.6. With this configuration, it appears that any connection attempt that doesn't get a cache hit resolves the virtual host, and makes an HTTP connection to that resolved public IP instead sending the request to the internal 10.x.x.11 address.
>
> Is there a way under squid 2.6 and higher to force any request that doesn't make a cache hit to a single backend IP address? The vhost option is necessary with http_port since the Host: header must contain the Virtual Host name as our web servers use that data to determine what which site to serve.
>
>
> squid build:
> # /opt/squid-2.6.16/sbin/squid -v
> Squid Cache: Version 2.6.STABLE16
> configure options: '--prefix=/opt/squid-2.6.16' '--enable-async-io' '--enable-snmp' '--enable-removal-policies=heap' '--enable-referer-log' '--enable-useragent-log'
>
> ----- squid.conf -----
> acl snmppublic snmp_community local-squid-ro
> acl all src 0.0.0.0/0.0.0.0
> acl manager proto cache_object
> acl localhost src 127.0.0.1/255.255.255.255
> acl to_localhost dst 127.0.0.0/8
> acl local_network src 172.16.0.0/16 10.0.0.0/8
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> acl web_ports port 80
> http_access allow web_ports
> http_access allow manager localhost
> http_access allow manager local_network
> http_access deny manager
> acl purge method PURGE
> http_access allow purge localhost
> http_access allow purge local_network
> http_access deny purge
> http_access allow all
> icp_access allow all
> http_port 80 accel defaultsite=10.x.x.11 vhost
> cache_peer 10.x.x.11 parent 80 0 no-query originserver
> hierarchy_stoplist cgi-bin ?
> acl QUERY urlpath_regex cgi-bin \?
> cache deny QUERY
> memory_replacement_policy heap LFUDA
> cache_replacement_policy heap LFUDA
> logformat CustomLog %>a %ui %un [%{%d/%b/%Y:%H:%M:%S %z}tl] "%rm %ru HTTP/%rv" %Hs %<st "%{Referer}>h" "%{User-Agent}>h" "%{Cookie}>h" %Ss:%Sh
> access_log /opt/squid-2.6.16/var/logs/custom.log CustomLog
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern . 0 20% 4320
> acl apache rep_header Server ^Apache
> broken_vary_encoding allow apache
> cache_effective_user www
> cache_effective_group www
> visible_hostname squid.domain.com
>
>
>
> Regards,
>
> Russell
>
Received on Thu Feb 28 2008 - 11:36:59 MST