Re: [squid-users] Re: Re: Re: Re: WCCP + squid 2.5-STABLE7 + linux 2.6.10

On Friday 25 February 2005 12:47 am, Reuben Farrelly wrote:
> Hi,
>
> At 02:14 p.m. 25/02/2005, Jesse Guardiani wrote:
> >Henrik Nordstrom wrote:
> >
> > > On Thu, 24 Feb 2005, Jesse Guardiani wrote:
> > >
> > >> I don't think it is anymore. It seems like the packets are just
> > >> dissappearing after they hit my iptables rule. I tried placing OUTPUT and
> > >> POSTROUTING LOG rules around the NAT table, and their hit counters
> > >> increment if I hit the cache directly from a web browser, but if I hit it
> > >> transparently the packet just dissappears after the REDIRECT to port
> > >> 3128.
> > >
> > > Try using DNAT instead of REDIRECT.
> >
> >I thought you might say that, so I tried it with DNAT earlier in the day.
> >I tried destination addresses 192.168.10.2 (my ip alias on eth0:22) and
> >192.168.1.2 (my "real" eth0 ip). Neither worked. Here's an example of the
> >latter:
> >
> ># iptables -t nat -L -v
> >Chain PREROUTING (policy ACCEPT 425 packets, 61769 bytes)
> > pkts bytes target prot opt
> > in out source destination
> > 43 2580
> > DNAT tcp -- gre1 any anywhere anywhere
> > tcp dpt:www to:192.168.1.2:3128
> >
> >Do you see anything wrong with the above?
> >
> >I'm starting to think that something is wrong with linux's gre WCCP
> >decapsulation. That's why I keep asking if anyone actually has
> >this working on my kernel and my squid. But I guess, judging from
> >the silence, that nobody has it working yet.
> >
> >Is there a better alternative to WCCP? I'm particularly interested
> >in the fail-over feature. I'd hate for my user's internet access
> >to go down just because my squid server rebooted.
>
>
> No need. I can confirm it does work, but it does need to be set up in a
> specific way.
>
> I have been using 2.6 series right the way through, now running 2.6.11-rc5,
> and switched to using the gre tunnel method when it became supported by the
> Linux kernel.

Are you running Red Hat or Fedora Core? I'm currently running Gentoo. Here's
my uname:

Linux rhea 2.6.10-gentoo-r6 #1 SMP Mon Feb 21 16:54:22 EST 2005 i686 Pentium II(Deschutes) GenuineIntel GNU/Linux

It's possible that some of the gentoo kernel patches are botching this up.
Are you running a custom compiled kernel? Or a production binary image?
I don't know what Fedora/Red Hat supplies these days...

I am mimicking you now, as a baseline, but it still isn't working. Can you
double check my config for me below?

> ip_wccp is good, but it is not in the kernel and it's a lot
> easier to just use a GRE tunnel built into the kernel instead.
> If you wish to use ip_wccp, I suggest you start by getting this config
> below to work properly first, and then change to ip_wccp and then take down
> the GRE interface, start from a position of it working before you start
> experimenting ;) The router config and squid config would be the same, the
> iptables config is slightly different though.
>
>
> Router config:
> --------------
>
> * My router is running 12.3(11)T3. BE CAREFUL, some versions of IOS do NOT
> work without also turning off CEF and/or fast switching, although most
> recent ones do work OK. Stick to a stable (non T or branch) release if you
> can, such as latest 12.2 or 12.3.

My router is running 12.2(8)T5.

I have:

!
no ip cef
!

> interface Ethernet0
> ip address 192.168.0.1 255.255.255.0
> ip wccp web-cache redirect in
>
> interface Loopback0
> ip address 172.16.1.5 255.255.255.252
> end

interface Loopback0
 ip address 172.16.1.5 255.255.255.252
!
interface FastEthernet0/0
 ip address 192.168.1.16 255.255.255.0
 no ip route-cache
 no ip mroute-cache
 duplex auto
 speed auto
!

> (Note the loopback IP range matches that on the GRE tunnel on my linux box)
>
>
> Linux box core config:
> -----------------
>
> /etc/sysconfig/network-scripts/ifcfg-gre0
>
> DEVICE=gre0
> BOOTPROTO=static
> IPADDR=172.16.1.6
> NETMASK=255.255.255.252
> ONBOOT=yes
> IPV6INIT=no

# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:A0:C9:AD:1E:11
          inet addr:192.168.10.2 Bcast:192.168.10.255 Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:3815156 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6398990 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1037857845 (989.7 Mb) TX bytes:4132984777 (3941.5 Mb)
          Interrupt:20 Base address:0x8000

gre0 Link encap:UNSPEC HWaddr 00-00-00-00-FF-00-00-00-00-00-00-00-00-00-00-00
          inet addr:172.16.1.6 Mask:255.255.255.252
          UP RUNNING NOARP MTU:1476 Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1440 (1.4 Kb) TX bytes:0 (0.0 b)

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:4321 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4321 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:365023 (356.4 Kb) TX bytes:365023 (356.4 Kb)

# iptunnel
gre0: gre/ip remote any local any ttl inherit

> iptables config:
> ----------------
>
> iptables -A PREROUTING -s 192.168.0.0/255.255.0.0 -d !
> 192.168.0.0/255.255.0.0 -i gre0 -p tcp -m tcp --dport 80 -j DNAT --to
> 192.168.0.3:3128

iptables -t nat -A PREROUTING -s 192.168.0.0/255.255.0.0 -d ! 192.168.0.0/255.255.0.0 -i gre0 -p tcp -m tcp --dport 80 -j DNAT--to 192.168.10.2:3128

Here's my iptables, including LOG rules:

# iptables -t nat -L -v
Chain PREROUTING (policy ACCEPT 10 packets, 1940 bytes)
 pkts bytes target prot opt in out source destination
   18 1080 DNAT tcp -- gre0 any 192.168.0.0/16 !192.168.0.0/16 tcp dpt:www to:192.168.10.2:3128

Chain POSTROUTING (policy ACCEPT 4 packets, 270 bytes)
 pkts bytes target prot opt in out source destination
    0 0 LOG tcp -- any any anywhere anywhere tcp dpt:webcache LOG level warning prefix `portthreeonetwoeightpost'
    0 0 LOG tcp -- any any anywhere anywhere tcp dpt:www LOG level warning prefix `porteightypostrouting'

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target prot opt in out source destination
    0 0 LOG tcp -- any any anywhere anywhere tcp dpt:webcache LOG level warning prefix `portthreeonetwoeightoutput'

> This makes sure that traffic from 192.168.0.0/255.255.0.0 destined for
> 192.168.0.0/255.255.0.0 is not redirected to the cache.
>
>
> Squid config:
> -------------
>
> wccp_router 192.168.0.1
> wccp_version 4
> wccp_outgoing_address 192.168.0.3 <<---- I have two IP addresses on this box

# cat squid.conf | grep -Ev '^[[:space:]]*$|^#'
http_port 3128
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_mem 300 MB
maximum_object_size 54096 KB
maximum_object_size_in_memory 18 KB
cache_dir aufs /var/cache/squid 3072 16 256
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
pid_filename /var/run/squid.pid
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl localnetwork src 206.30.56.0/21 206.30.215.0/24 63.99.6.0/24 216.64.96.0/21
acl privatenetwork src 192.168.9.0/24 192.168.88.0/24 192.168.89.0/24 192.168.10.0/24 192.168.1.0/24
acl SSL_ports port 443 563
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager all
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow localnetwork
http_access allow privatenetwork
http_access deny all
http_reply_access allow all
icp_access allow all
cache_mgr webmaster@wingnet.net
visible_hostname rhea.int.wingnet.net
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
snmp_port 3401
wccp_router 192.168.10.1
coredump_dir /var/cache/squid

> I'm not sure if it is optimal or not, but it works with every squid version
> I have ever tried. If I remember correctly, some of these instructions
> came from a page by Joe Cooper @ Swelltech, but I can't put my hands on it
> right now.

It's not working here. I'm still seeing packets on the iptable rule, but
it's still timing out when I attempt to use a client browser. Cache is
up on router:

#sh ip wccp web-cache detail
WCCP Cache-Engine information:
        IP Address: 192.168.10.2
        Protocol Version: 0.4
        State: Usable
        Initial Hash Info: 00000000000000000000000000000000
                               00000000000000000000000000000000
        Assigned Hash Info: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
                               FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
        Hash Allotment: 256 (100.00%)
        Packets Redirected: 15
        Connect Time: 00:12:52

tcpdump still looks the same:

# tcpdump -i any 'not ( host shannon and port 22)and not host 192.168.1.193 and not port syslog and not port domain and not snmpand not port 3632 and not port ssh and not arp'
tcpdump: Symbol `eproto_db' has different size in shared object, consider re-linking
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes

23:42:44.433390 IP 192.168.10.1 > 192.168.10.2: gre-proto-0x883e
23:42:44.454716 IP 192.168.10.5.34918 > 64.233.187.99.www: S 1055483184:1055483184(0) win 5840 <mss 1460,sackOK,timestamp 511766502 0,nop,wscale 2>
23:42:45.940968 IP 192.168.10.2.2048 > 192.168.10.1.2048: UDP, length: 52
23:42:45.942276 IP 192.168.10.1.2048 > 192.168.10.2.2048: UDP, length: 64
23:42:47.432106 IP 192.168.10.1 > 192.168.10.2: gre-proto-0x883e
23:42:47.432106 IP 192.168.10.5.34918 > 64.233.187.99.www: S 1055483184:1055483184(0) win 5840 <mss 1460,sackOK,timestamp 511769502 0,nop,wscale 2>
23:42:53.432007 IP 192.168.10.1 > 192.168.10.2: gre-proto-0x883e
23:42:53.432007 IP 192.168.10.5.34918 > 64.233.187.99.www: S 1055483184:1055483184(0) win 5840 <mss 1460,sackOK,timestamp 511775502 0,nop,wscale 2>
23:42:56.731844 IP 192.168.10.2.2048 > 192.168.10.1.2048: UDP, length: 52
23:42:56.733654 IP 192.168.10.1.2048 > 192.168.10.2.2048: UDP, length: 64

10 packets captured
20 packets received by filter
0 packets dropped by kernel

What version of squid do you run? I'm running 2.5-STABLE7 with a few
gentoo specific patches.

Next step is once you tell me which specific kernel you are running
and which specific squid you are running I will mimic those, unless
you can find anything wrong above.

Thanks!

-- 
Jesse Guardiani, Systems Administrator
WingNET Internet Services,
P.O. Box 2605 // Cleveland, TN 37320-2605
423-559-LINK (v)  423-559-5145 (f)
http://www.wingnet.net