Re: [squid-users] Squid and virtual hosts

ok, i had to draw myself a picture

http://wcg2.com/_Photos@transparent_proxy_with_mapped_ports.jpg

let me know if it looks funny, or something is incorrect.

are you certain that the port 3128 traffic coming back from the squid
machine and through the firewall is correctly returning to the client?

i'll think about this some more, maybe have some better idears in the
morning.

thanks

waitman

----- Original Message -----
From: Richard Diaz
To: Waitman C. Gobble, II
Cc: squid-users@squid-cache.org
Sent: Tuesday, August 06, 2002 4:10 PM
Subject: Re: [squid-users] Squid and virtual hosts

I don't get anything using the IP address. It just times out after a while.
I tried watching the log file, and here is what I see:

1028674587.040 115 192.168.XXX.XXX TCP_REFRESH_MISS/200 6427 GET
http://my.page.com/ - DIRECT/YYY.YYY.YYY.YYY text/html

Where the XXX's are our internal addresses and the YYY's are our external
(web server) address. *This is only when I try using lynx from the squid
host.*

From all other clients, I don't see anything. Is this not a squid issue
maybe? Why would the firewall forward all requests to the cache except
requests to the dmz? Thanks for all your help. I think that I'll start
pointing the finger at my firewall instead of squid. I am still open to any
suggestions though ;)

Sincerely,
Rich

"Waitman C. Gobble, II" <waitman@emkdesign.com>
08/06/2002 03:35 PM
Please respond to "Waitman C. Gobble, II"

        To: "Richard Diaz" <rdiaz@nbframing.com>
        cc: <squid-users@squid-cache.org>
        Subject: Re: [squid-users] Squid and virtual hosts

if you type in the web server's ip address on a browser at a client machine,
what comes up?
do the web server logs show activity?

you can do a

tail -f /var/log/httpd/access-log

(well, put in whatever the access log file is really called)

it will sit there and show you log entries whenever the file is changed
(whenever there is traffic).

by the way, you DID do a squid -z
to initiate the cache, correct?

best,

waitman

----- Original Message -----
From: Richard Diaz
To: Waitman C. Gobble, II
Cc: squid-users@squid-cache.org
Sent: Tuesday, August 06, 2002 10:12 AM
Subject: Re: [squid-users] Squid and virtual hosts

I changed the allowed_hosts to use the dotted-slash notation. It looks like
squid likes either.

I also set the httpd_accel options to the values you stated below with the
exception of httpd_accel_port. I set that one to '80' and configured my
firewall to throw requests to that port. Now it looks like the cache isn't
being used at all, and I can still not get to the virtual domains hosted in
our DMZ.

I then reconfigured the firewall to redirect to port 3128 on the squid host.
The clients started using the cache again, but still no virtual hosts.

This just seems strange to me because I figured squid would work fine with
host headers, and of course I'm sure it does. The machine in our DMZ should
appear like just another web server out on the Internet to it. So
theoretically, it should just 'work'. Are there any other steps I can do to
trouble-shoot this?

Thanks,
/rich

"Waitman C. Gobble, II" <waitman@emkdesign.com>
08/06/2002 12:46 PM
Please respond to "Waitman C. Gobble, II"
       To: "Richard Diaz" <rdiaz@nbframing.com>
       cc: <squid-users@squid-cache.org>
       Subject: Re: [squid-users] Squid and virtual hosts

a) i really don't think it truly matters, but i would make the acl

acl allowed_hosts src XXX.XXX.0.0/16

for some reason i seem to recall using the netmask wasn't doing the job...
might just be a bad memory ;-)

i think i was incorrect about the http_accel port setting being used as a
"listen" port. after i read the documentation it seems to be a directive to
"fetch" content.

for instance, if you want to run a transparent proxy on port 80, and have a
web server on port 8000, you can force squid to grab the site content from
port 8000 using the http_accel_port option. it looks like if you set
http_accel_port to 0 then this is a "virtual" setting (the browser specifies
which port to use).

however you obviously cannot have a transparent proxy on 80 and apache on 80
at the same time ;-)

----- Original Message -----
From: Richard Diaz
To: Waitman C. Gobble, II
Cc: squid-users@squid-cache.org
Sent: Tuesday, August 06, 2002 9:19 AM
Subject: Re: [squid-users] Squid and virtual hosts

Ah, it is not using the proxy. Because of a documented configuration issue
on the firewall, I had to add a rule that bypassed the proxy for the squid
host itself on the firewall. I forgot about this, and just figured that
_all_ traffic was going through the proxy.

I added: http_proxy:http://xxx.xxx.xxx.xxx:3128 to the /etc/lynx.cfg file.
It still brings up the page correctly.

We don't have user access control configured in squid.conf. We let the
firewall do that. We just have: acl allowed_hosts src
XXX.XXX.XXX.XXX/255.255.0.0

I did build squid from source and I don't believe that I have it installed
from RPM also (Linux newbie). I issued 'rpm -q squid' and got back a reply
that it is not installed. 'ps -auxww | grep squid' shows it running from
/usr/local/squid/bin/squid. './squid -v' in that dir comes back as
2.4.STABLE7.

I did try those very httpd_accel settings you mention. But I did not tell
my firewall to use port 80 for the proxy. I left it at 3128. I am running
Apache on this box listening on 80 already. Can I use something else, like
8080? I would imagine I could use anything as long as the firewall pointed
at httpd_accel_port and not http_port.

Again, thanks for all your help.

/rich

"Waitman C. Gobble, II" <waitman@emkdesign.com>
08/06/2002 11:42 AM
Please respond to "Waitman C. Gobble, II"

      To: "Richard Diaz" <rdiaz@nbframing.com>
      cc: <squid-users@squid-cache.org>
      Subject: Re: [squid-users] Squid and virtual hosts

hmmm, are you certain that lynx is using the proxy?

also, what sort of acl's do you have set up in squid.conf? do you have user
access control or simple ip or mac authorization?

now that i think about it more, perhaps you can set up squid to behave such
as if you were using it as a gateway to internally hosted web sites.

the example i am thinking about is having several servers on a private
intranet, and only exposing the squid machine to the outside world. dns
would point to the public ip, a site request would hit the squid cache and
you would have dns set up ... for internal use only so that squid could go
out and fetch the appropriate site from the appropriate machine.

questions

- it sounds like you built squid from source (i am not sure that there is
an rpm for STABLE7) - do any rpm installations exist? if so, are you certain
that STABLE7 is running and not the rpm version?

I had zero luck with a binary package, i uninstalled it and built from
source and everything magically worked swell. actually i didn't have any
luck with a redhat binary/rpm OR a mac os x binary distribution (i have a
client that likes macs). both systems - i had to build from source in order
to achieve success.

another thought, thinking about the example i mentioned above ---

perhaps you need to have the accelerator listening on port 80, and you need
to make sure that settings look like

httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

maybe the redirect to port 3128 thingy is the problem? try redirecting to
port 80?

take care

waitman

----- Original Message -----
From: Richard Diaz
To: Waitman C. Gobble, II
Cc: squid-users@squid-cache.org
Sent: Tuesday, August 06, 2002 8:22 AM
Subject: Re: [squid-users] Squid and virtual hosts

oops! scratch that last comment. I fat-fingered it. It _does_ work from
the squid host using lynx. I guess I was just hoping there was a simple
solution :).

/rich

__________________

Currently, the port 80 traffic on the LAN is mapped to 3128 on the squid
cache. Yes, the firewall does this, and as far as I know, no browser is
aware of the proxy.

I read quite a few posts regarding HTTPD acceleration and setting these
options, such as: httpd_accel_host, httpd_accel_port, httpd_accel_with_proxy
and http_accel_uses_host. I didn't think it would work either, but i wanted
to try everything I could before I posted a message.

We use internal DNS servers that cache from our ISP's servers. All internal
clients are configured to use our servers via DHCP.

I would like to avoid 'touching' each desktop. I have several hundred users
in 10 different locations (including Buena Park, CA!). We also use our
firewall for authentication and logging. Piping everyone through the proxy
would break that.

I never even thought about trying from the squid machine, duh! I just tried
to access our virtual hosts from using lynx and received an :
Alert!: Unexpected network read error; connection aborted.
Can't Access `http://www.XXX.com/'
Alert!:Unable to access document.
I am able to access other websites using lynx from this machine. Maybe this
has something to do with it? Thanks for all your suggestions.

Sincerely,
Richard Diaz
Nielsen & Bainbridge
Senior Systems Administrator
Voice:201.368.9191
Fax:201.342.6084

"Waitman C. Gobble, II" <waitman@emkdesign.com>
08/06/2002 10:35 AM
Please respond to "Waitman C. Gobble, II"

     To: <squid-users@squid-cache.org>, <rdiaz@nbframing.com>
     cc:
     Subject: Re: [squid-users] Squid and virtual hosts

hello

if i understand correctly, you have squid running on port 80? or is the port
80 traffic on the LAN mapped to 3128 on the squid cache (default, otherwise
some other port) .... ?

so, no browser on your network is configured to use a proxy, the firewall
just bounces the traffic to the cache.

that is what it sounds like you are doing, please let me know if this is not
correct.

couple of comments -

a) what are you doing with the httpd_accel options? i don't believe these
will deliver a solution.
b) squid is completely compatible with host header virtual hosting - i
haven't seen any trouble. you can verify this by looking at your web server
logs.

a quick thing to check, if you don't want to tinker with other
settings/options first - however may not be prudent - : make sure that
either a) dns (port 53 tcp/udp) is available to the clients in the case of
an external name server, or b) the clients are in fact getting the name
resolutions from an internal machine. if you have your firewall set up to
bounce traffic and no proxy options on the browser, name resolution will
occur at the client not the server.

(you really should have an internal nameserver running to improve
performance)

HOWEVER what i suggest is at minimum blocking port 80 and 443 traffic from
your clients altogether, set up squid to listen on 3128 (default) or some
other port, and have each client configured to use your squid server as a
proxy by explicitly specifying the address and port in the browser settings.
make sure the squid machine can perform name lookups, and verify internal
lookups are correct. drop the bounce and redirect scheme...

of course if you have a ton of clients to configure then manually setting up
the clients could be a real pain, however my intuition tells me that you
won't have much luck using the firewall to solve the problem quick and
dirty. (perhaps some others can comment about this a bit and give some
better analysis/solution).

the only trouble you will likely have will be with software that doesn't
care about proxy settings (i have seen a lot of "live update" kinds of
things bomb out) and the windows "active" desktop (i haven't 100% verified
this but from what i have seen the windows desktop with internet content
doesn't give a care about your proxy settings (ie, they won't work if you
have port 80, etc blocked).

-- can you succesfully use the squid cache from the squid server itself? you
can make tests using lynx, wget etc.... if you don't have an x server /
window environment running.

hopefully some of this helps!!!

take care

waitman gobble
emk design
buena park, california
+1.7145222528
http://emkdesign.com

----- Original Message -----
From: <rdiaz@nbframing.com>
To: <squid-users@squid-cache.org>
Sent: Tuesday, August 06, 2002 6:58 AM
Subject: [squid-users] Squid and virtual hosts

> am new to using squid and have been trying to implement it at my
> company for the past few days. So far, it has been working great with
> only a few small bumps. The most significant of which is the apparent
> lack of support for virtual hosts.
>
> My configuration is as follows: Squid 2.4.STABLE7 running on RedHat
> 7.3. This machine sits on our local network behind a Watchguard
> firewall, on the trusted interface. I have the firewall configured to
> forward all HTTP requests to the squid proxy on the internal network.
> There is a rule in place on the firewall that allows the proxy to
> access the Internet. This appears to be working well.
>
> We also have a web server sitting on the optional/dmz interface of the
> firewall that hosts a few sites. The server has a single public IP
> address. It does not have a private IP address on the local network.
> It uses host headers to direct users to the correct site.
>
> Users outside of the firewall have no problem accessing any of the
> sites. Users inside the firewall cannot access any site. Their
> browser will eventually timeout.
>
> I have researched this topic for a few days, and cannot find a
> solution. I played around with the httpd_accel options to no avail.
> I would appreciate any insight you might have into this configuration.
>
> Thank you.
>
> Sincerely,
> RD
>
> P.S. I originally posted this to the newsgroup via DejaNews unaware of the
> mailing list. I apologize for the duplication.
>
Received on Tue Aug 06 2002 - 21:13:34 MDT