[squid-users] Re: stale client blocking other downloads

2012-09-19 17:07:46 +0300, Eliezer Croitoru:
> On 9/19/2012 4:36 PM, Stephane CHAZELAS wrote:
> >But only when providing with a User-Agent oddly enough (and also
> >when inserting a 10 second delay between the two requests).
> >Then, I tried to do the same query from a different client and
> >since then, I cannot reproduce it anymore (even if I do a
> >different request), and I see no merging in conntrackd. So I
> >don't have the full picture here, and reckon it's going to be
> >difficult to investigate as I don't know how to reproduce it
> >consistently.
> >
> >I've got network captures, but only on the down side (between
> >the client and the proxy).
>
> I think it's better to be moved into squid-users since it's not
> really about squid code.
> so next time reply to squid-users_at_squid-cache.org

OK. Note that squid has been working perfectly with tproxy for
several months, and the issue is only coming up now because of
that new buggy HTTP client we're evaluating that behaves in an
unexpected way (makes a request, doesn't read the response and
makes another identical request). I was raising it to -dev
because it looks potentially like a serious denial of service
vulnerability in squid.

> if you can also give the iptables\tproxy related settings it can of help.
>
> if there is some SNAT and other stuff this should be mentioned.
>
> so the topology is:
>
> 10.10.10.10->10.0.0.1 ??
> are there any other routes in the way?(it's important)

I never said there was a 10.10.10.10 -> 10.0.0.1 connection.

Well, there are other routes on the way from 10.10.10.10 to
10.0.0.1.

The default route on 10.10.10.10 is via 10.10.10.1 which is an
interface to a plain router (no NAT) whose default route is
via 10.0.0.1.

On the 10.0.0.1 machine the relevant iptable rules are:

In the mangle table:

-A PREROUTING -i eth0 -m socket --transparent -j MARK --set-xmark 0x1/0x1

That is fwmark packets coming from the WAN interface matching a
transparent socket with bit 1.

-A PREROUTING -i vlan2 -p tcp -m conntrack --ctstate NEW -m multiport --dports 80,8080 -j CONNMARK --set-xmark 0x1/0x1

connmark the connections from LAN interface to tproxied HTTP
ports with bit 1.

-A PREROUTING -i vlan2 -m connmark --mark 0x1/0x1 -j excltproxy

And for all packets connmarked 1, go to excltproxy which does:

first a few exclusions based on destination address:

-A excltproxy -d xxx/xx -j RETURN excltproxy -p tcp -j TPROXY

and:

-A excltproxy -p tcp -j TPROXY --on-port 3129 --on-ip 0.0.0.0 --tproxy-mark 0x1/0x1

(the -p tcp being superflous on those two rules) to tproxy (and
fwmark with bit 1).

There's a new routing table 100, with:

ip rule add fwmark 1/1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100

So that every packet with fwmark 1 are routed to the loopback
interface.

In the nat table, we have

-A POSTROUTING -s 10.0.0.0/8 -j SNAT --to-source 4.5.6.7

But that SNAT could as well have been done by an upstream
router, so I don't think we need it in the picture.

> and how exactly is this tproxying ?
> 10.10.10.10:46426 -> 216.40.222.19
>
> it's more like natting then TPROXY.

You distorted what I wrote. The packets come in as (src/dst IP
address):

10.10.10.10:33502 -> 69.195.137.28:80
with "Host: rules.emergingthreats.net"

Which is trapped by squid and squid starts a new connection
(marked transparent by squid):
10.10.10.10:46426 -> 216.40.222.19:80
or sometimes:
10.10.10.10:46426 -> 69.195.137.28:80

(69.195.137.28:80 and 216.40.222.19 being the two IPv4 address
that rules.emergingthreats.net resolves to. So it it TPROXY
here.

We have SNAT upstream so it becomes
4.5.6.7:12345 -> 69.195.137.28:80

but that's another story.

> in a case you have only one outgoing address to the internet I
> think TPROXY should not be your default choice but use REDIRECT
> instead since you dont need in any way TPROXY which can cause some
> routing issues in a case not configured properly.
[...]

We switched from REDIRECT to TPROXY to have the packets in the
INPUT chain unmodified for logging and intrusion
detection/prevention purposes (and before you ask, no, that
IDS/IPS takes no part in this issue).

-- 
Stephane