Re: [squid-users] Increasing File Descriptors - Fixed!!

Hi!

There are a couple of tricks to get this, at least on Linux systems.

1. I used to use a fancy trick for this: conntrack.

cat /proc/net/ip_conntrack

With conntrack module loaded, usually, just add a few iptables rules,
maybe just as simple as:

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

That should do the trick, then on the file you get something like:

cat /proc/net/ip_conntrack

tcp 6 431989 ESTABLISHED src=10.16.0.73 dst=74.125.45.83
sport=50048 dport=443 packets=11 bytes=8765 src=74.125.45.83
dst=10.16.0.73 sport=443 dport=50048 packets=9 bytes=2218 [ASSURED]
mark=0 secmark=0 use=2
tcp 6 431997 ESTABLISHED src=10.16.0.73 dst=74.125.67.17
sport=49949 dport=443 packets=15 bytes=10729 src=74.125.67.17
dst=10.16.0.73 sport=443 dport=49949 packets=15 bytes=2785 [ASSURED]
mark=0 secmark=0 use=2
tcp 6 431999 ESTABLISHED src=95.211.91.20 dst=10.16.0.73
sport=1723 dport=43648 packets=13385 bytes=19006700 src=10.16.0.73
dst=95.211.91.20 sport=43648 dport=1723 packets=6446 bytes=257840
[ASSURED] mark=0 secmark=0 use=2
tcp 6 431987 ESTABLISHED src=10.16.0.73 dst=74.125.45.83
sport=34690 dport=443 packets=11 bytes=3759 src=74.125.45.83
dst=10.16.0.73 sport=443 dport=34690 packets=9 bytes=1903 [ASSURED]
mark=0 secmark=0 use=2
tcp 6 431987 ESTABLISHED src=10.16.0.73 dst=74.125.45.83
sport=34692 dport=443 packets=25 bytes=11174 src=74.125.45.83
dst=10.16.0.73 sport=443 dport=34692 packets=23 bytes=15733 [ASSURED]
mark=0 secmark=0 use=2

As you can see, from there you can extract the source IP and source
port,and other info, luke the connection state, the connection or sec
mark, packets and bytes associated with the connection, that info can
be useful.

This will help to know which local ports are being used on active connections.

2. Another, maybe simpler, way:

netstat -na

That gives something like this:

tcp 0 0 10.16.0.73:34692 74.125.45.83:443 ESTABLISHED
tcp 0 0 10.16.0.73:34690 74.125.45.83:443 TIME_WAIT
tcp 0 0 10.16.0.73:49949 74.125.67.17:443 ESTABLISHED
tcp 0 0 10.16.0.73:50048 74.125.45.83:443 ESTABLISHED
tcp 0 0 10.16.0.73:43648 95.211.91.20:1723 ESTABLISHED

This doesn't give bytes nor packets, but enough to get the ports
involved on the connection.

I hope this helps,

Ildefonso Camargo

On Thu, May 13, 2010 at 8:32 AM, Nyamul Hassan <mnhassan_at_usa.net> wrote:
> Is it possible to have a CacheMgr parameter that shows the total
> number of ports that are being used by Squid?
>
> That might be complicated though.  We use different IP addresses for
> outgoing requests, so number of ports would make sense if available on
> a IP wise breakdown.
>
> Regards
> HASSAN
>
>
> 2010/5/13 Bradley, Stephen W. Mr. <bradlesw_at_muohio.edu>
>>
>> I wish I had snapshots of our stats when we were up and under full load so I could give you and idea of how it was working.
>>
>> But it was working well right up until the ragged edge (and my FD limit at the time).
>>
>> steve
>>
>> -----Original Message-----
>> From: Henrik Nordström [mailto:henrik_at_henriknordstrom.net]
>> Sent: Wednesday, May 12, 2010 3:57 PM
>> To: Bradley, Stephen W. Mr.
>> Cc: Squid Users
>> Subject: RE: [squid-users] Increasing File Descriptors - Fixed!!
>>
>> ons 2010-05-12 klockan 10:17 -0400 skrev Bradley, Stephen W. Mr.:
>> > I went ahead and updated /etc/sysctl.conf to allow for 1024-64000 ports open for connections.
>> >
>> > That should help; I just wish I could test it before August.........  :-(
>>
>> Reminds me that we should implement some smart address + port selection
>> algorithm. The default port selection algorithm provided by the
>> operating systems is very stupid and quickly runs out of free ports even
>> if you increase the range to max possible. A maximized range of 60K
>> ports can sustain only about 500 connections / second.
>>
>> Thankfully that's quite sufficient for most Squid setups still, but not
>> all, and with improvements in multi-core support it will become much
>> more of an issue.
>>
>> Regards
>> Henrik
>>
>>
>>
>
Received on Thu May 13 2010 - 14:41:35 MDT