Re: [squid-users] squidaio_queue_request: WARNING - Queue congestion

On Thu, Feb 18, 2010 at 12:27 AM, Henrik Nordstrom
<henrik_at_henriknordstrom.net> wrote:
> ons 2010-02-17 klockan 21:40 -0800 skrev Tory M Blue:
>
>> And sorry "sleeping" was just my way of citing the box shows no load,
>> almost no IO 4-5 when I'm hitting it hard. I do not see this issue
>> with lesser threads, it's only when I turn up the juice. But with
>> turning up the connections per second I would expect to see some type
>> of load and I see none.
>
> Anything in /var/log/messages?
>
> The above problem description is almost an exact match for Linux
> iptables connectiontracking table limit being hit.
>
> Regards
> Henrik

Thanks Henrik, nothing in /var/logs/messages or even dmesg

and iptables

Not running. No rules in place, service shutdown.

Thats not the culprit, with less than 12 children at the beginning of my run

2010/02/17 10:29:51| Completed Validation Procedure
2010/02/17 10:29:51| Validated 948594 Entries
2010/02/17 10:29:51| store_swap_size = 3794376k
2010/02/17 10:29:51| storeLateRelease: released 0 objects

2010/02/18 09:53:08| squidaio_queue_request: WARNING - Queue congestion
2010/02/18 09:53:12| squidaio_queue_request: WARNING - Queue congestion
2010/02/18 09:53:17| squidaio_queue_request: WARNING - Queue congestion

Even dropped my thread count and as soon as my load test starts (get
maybe 10 children launched), I get the error

2010/02/18 09:56:18| squidaio_queue_request: WARNING - Queue congestion
2010/02/18 09:56:28| squidaio_queue_request: WARNING - Queue congestion

Okay I've found some issues that I had not seen before,

Feb 18 18:37:06 kvm0 kernel: nf_conntrack: table full, dropping packet.

I would like to kick the netfilter team and fedora team in the shins.
The issue was my squid boxes are virtual and the errors were being
logged on the domain box (not domain as in MS). So now I'm trying to
go through the system and remove all this garbage. This server does
not need to track the connections and or log them. There does not seem
to be a simple way to disable, just a lot of sysctl options and I'm
unclear if these will do it entirely.

net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.all.arp_filter=0
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.default.arp_filter=0
net.ipv4.conf.lo.rp_filter=0
net.ipv4.conf.lo.arp_filter=0
net.ipv4.conf.eth0.rp_filter=0
net.ipv4.conf.eth0.arp_filter=0
net.ipv4.conf.eth1.rp_filter=0
net.ipv4.conf.eth1.arp_filter=0
net.ipv4.conf.br0.rp_filter=0
net.ipv4.conf.br0.arp_filter=0
net.ipv4.conf.br1.rp_filter=0
net.ipv4.conf.br1.arp_filter=0
net.ipv4.conf.vnet0.rp_filter=0
net.ipv4.conf.vnet0.arp_filter=0
net.ipv4.conf.vnet1.rp_filter=0
net.ipv4.conf.vnet1.arp_filter=0

But I'll be quiet here for a few, until I need assistance from the
squid community. I'm still seeing the queue congestion, but if it
actually doubles the thresholds each time, I may get to a good place,
or can be okay to ignore the messages. Obviously the queue congestion
was not causing the 500's, the dropping of packets by netfilter was

Thanks

Tory
Received on Thu Feb 18 2010 - 19:49:09 MST