Re: [squid-users] TProxy Issues

Jamie Orzechowski wrote:
> I have made the recommended changes but still everything is a TCP_MISS
> and going DIRECT ... I removed the direct-urls for now just in case.
> Any further suggestions?
<snip config: it does permit caching **if allowed**>
>
>
> http://www.google.com/calendar/feeds/8vm1eh2d7rcltngl0omsa4qc3g%40group.calendar.google.com/private/full?
> - DIRECT/208.67.217.230 text/html
> http://www.google.com/calendar/feeds/8vm1eh2d7rcltngl0omsa4qc3g%40group.calendar.google.com/private/full?
> - DIRECT/208.67.217.230 text/xml
> 1236865766.338 991 66.78.98.25 TCP_MISS/200 33924 GET
> http://www.maximumsignal.ca/store/product.php? - DIRECT/209.200.251.37
> text/html
> 1236865766.993 546 66.78.98.25 TCP_MISS/200 2435 GET
> http://www.maximumsignal.ca/store/antibot_image.php? -
> DIRECT/209.200.251.37 image/png

Well, thanks for these. I ran a few of the pages through the
cacheability engine:
   http://www.ircache.net/cgi-bin/cacheability.py

And got this most of the time:
   Expires 606 weeks 5 days ago (Sat, 26 Jul 1997 05:00:00 GMT)
   Cache-Control post-check=0, must-revalidate, no-store, no-cache,
pre-check=0
   Last-Modified now (Fri, 13 Mar 2009 01:08:54 GMT)

"
  * This object has been deliberately marked stale.
  * The object had changed when validation was attempted.
  * It will be revalidated on every hit, because it has a Cache-Control:
no-cache header.
  * It won't be cached at all, because it has a Cache-Control: no-store
header.
  * Because of the must-revalidate header, all caches will strictly
adhere to any freshness information you set.
  * It doesn't have a Content-Length header present, so it can't be used
in a HTTP/1.0 persistent connection.
"

Sometimes they also have:
"
  * This object requests that a Cookie be set; this makes it and other
pages affected automatically stale; clients must check them upon every
request.
"

Only the images seem to be cachable and only by cache with reasonable
refresh_patterns set.

Amos

>
>
> Amos Jeffries wrote:
>>> Here is the config ... it does work fine in "transparent" mode just not
>>> tproxy mode
>>>
>>> acl manager proto cache_object
>>> acl localhost src 127.0.0.1/32
>>> acl to_localhost dst 127.0.0.0/8
>>>
>>
>>
>>> acl localnet src 66.78.96.0/19
>>> acl localnet src 64.235.192.0/19
>>> acl localnet src 72.0.192.0/19
>>> acl localnet src 192.168.1.0/24
>>> acl localnet src 192.168.254.0/24
>>>
>>> acl QUERY urlpath_regex cgi-bin \?
>>> cache deny QUERY
>>>
>>
>> You can drop the above. It's handled by the refresh_patterns.
>>
>>
>>> hierarchy_stoplist cgi-bin ?
>>>
>>> acl directurls url_regex -i "/etc/squid3/direct-urls"
>>> cache deny directurls
>>>
>>
>> You are not testing with one of these URLs are you?
>>
>>
>>> cache deny localnet
>>>
>>
>> This is definately causing MISS. Your http_access rules below prevent
>> non-localnet access. Therefore every received request will hit this cache
>> prevention.
>>
>> You want "cache allow all" at the end of the cache list.
>>
>>
>>> always_direct allow directurls
>>> always_direct allow localnet
>>>
>>> acl SSL_ports port 443
>>> acl Safe_ports port 80 # http
>>> acl Safe_ports port 21 # ftp
>>> acl Safe_ports port 443 # https
>>> acl Safe_ports port 70 # gopher
>>> acl Safe_ports port 210 # wais
>>> acl Safe_ports port 1025-65535 # unregistered ports
>>> acl Safe_ports port 280 # http-mgmt
>>> acl Safe_ports port 488 # gss-http
>>> acl Safe_ports port 591 # filemaker
>>> acl Safe_ports port 777 # multiling http
>>> acl CONNECT method CONNECT
>>>
>>> http_access allow manager localhost
>>> http_access deny manager
>>> http_access deny !Safe_ports
>>> http_access deny CONNECT !SSL_ports
>>> http_access deny to_localhost
>>> http_access allow localnet
>>> http_access allow localhost
>>> http_access deny all
>>> icp_access allow localnet
>>> htcp_access allow localnet
>>> icp_access deny all
>>> htcp_access deny all
>>> htcp_clr_access deny all
>>> ident_lookup_access deny all
>>>
>>> http_port 66.78.102.2:3128
>>> http_port 66.78.102.2:3129 tproxy
>>>
>>> cache_mgr support_at_ripnet.com
>>>
>>> acl snmp snmp_community s64hf2
>>> snmp_access allow snmp all
>>>
>>> snmp_port 3401
>>> snmp_incoming_address 192.168.1.8
>>> snmp_outgoing_address 192.168.1.8
>>>
>>> shutdown_lifetime 10 seconds
>>> pid_filename /var/run/squid3.pid
>>> mime_table /usr/share/squid3/mime.conf
>>> icon_directory /usr/share/squid3/icons
>>>
>>
>>
>>> error_directory /usr/share/squid3/errors/en
>>>
>>
>> :( I would recommend removing that for a better experience by your users.
>> (allows errors to appear in the visitors own language)
>>
>>
>>> cache_effective_user proxy
>>> ignore_unknown_nameservers on
>>> dns_nameservers 66.78.99.4 66.78.99.5
>>>
>>>
>> <snip>
>>
>>> refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
>>>
>>
>> Double check this pattern you have files ending in:
>> '.index' any-character 'htm' optional-'l'
>>
>> I think you maybe meant:
>> -i .index\.(html|htm)$
>>
>>
>>> refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
>>> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>>> refresh_pattern . 0 40% 40320
>>>
>>>
>>>
>>> Amos Jeffries wrote:
>>>
>>>>> I am using squid 3.1.0.6
>>>>>
>>>>> If I check the disk free while the cache is running I do not see
>>>>> any of
>>>>> my cache directories incrementing at all.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>>
>>>> Okay, so much for the easy answer. We will have to see your config to
>>>> tell
>>>> why its not caching.
>>>>
>>>> Amos
>>>>
>>>>
>>>>
>>>>> Amos Jeffries wrote:
>>>>>
>>>>>
>>>>>> Jamie Orzechowski wrote:
>>>>>>
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> My post does not seem to be going to the list. Wondering if you
>>>>>>> have
>>>>>>> any ideas?
>>>>>>>
>>>>>>> I think I have TPROXY working but running into some issues.
>>>>>>> Checking my logs all my traffic shows up as a TCP_MISS
>>>>>>>
>>>>>>>
>>>>>> Squid 3.1.0.3? it has a storage problem that can show like this.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> 1236698452.579 79 66.78.98.194 TCP_MISS/200 542 GET
>>>>>>> http://l1.zedo.com//log/p.gif? - DIRECT/72.247.244.10 image/gif
>>>>>>> 1236698452.634 293 66.78.98.194 TCP_MISS/200 4972 GET
>>>>>>> http://blstb.msn.com/i/9B/DDD13A38CB8B34F4DFA3F7BFFF71.jpg -
>>>>>>> DIRECT/192.221.114.124 image/jpeg
>>>>>>> 1236698452.878 100 66.78.98.194 TCP_MISS/200 1076 GET
>>>>>>> http://h.foxsports.com/HG? - DIRECT/64.154.81.231 image/gif
>>>>>>> 1236698453.367 252 66.78.98.194 TCP_MISS/200 1368 GET
>>>>>>> http://www.myinternetservices.com/live/visitor/index.php? -
>>>>>>> DIRECT/72.232.167.111 image/gif
>>>>>>> 1236698454.087 13 66.78.98.194 TCP_MISS/200 812 GET
>>>>>>> http://weyedata.pelmorex.com/WeatherEye/ObsData/CAON0090.xml -
>>>>>>> DIRECT/207.96.160.37 text/xml
>>>>>>> 1236698455.251 116 66.78.98.194 TCP_MISS/200 1368 GET
>>>>>>> http://www.myinternetservices.com/live/visitor/index.php? -
>>>>>>> DIRECT/72.232.167.111 image/gif
>>>>>>> 1236698456.570 6451 66.78.98.194 TCP_MISS/200 45898 GET
>>>>>>> http://www.facebook.com/profile.php? - DIRECT/69.63.176.140
>>>>>>> text/html
>>>>>>> 1236698456.876 77 66.78.98.194 TCP_MISS/200 2765 GET
>>>>>>> http://profile.ak.facebook.com/v227/2005/50/q638320646_36.jpg -
>>>>>>> DIRECT/209.170.91.178 image/jpeg
>>>>>>>
>>>>>>> My iptables is the following
>>>>>>>
>>>>>>> echo 1 > /proc/sys/net/ipv4/ip_forward
>>>>>>> /sbin/iptables -t mangle -N DIVERT
>>>>>>> /sbin/iptables -t mangle -A DIVERT -j MARK --set-mark 1
>>>>>>> /sbin/iptables -t mangle -A DIVERT -j ACCEPT
>>>>>>> /sbin/iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
>>>>>>> /sbin/iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
>>>>>>> --tproxy-mark 0x1/0x1 --on-port 3129
>>>>>>> //
>>>>>>> any idea why I am not getting any TCP_HITS? ...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Amos
>>>>>>
>>>>>>
>>>>> --
>>>>> =-=-=-=-=-=-=-=-=-=-=-=-=
>>>>> Jamie Orzechowski - CCNA
>>>>> RipNET Ltd. System/Network Administrator
>>>>> Tel.: 613-342-3946 x294
>>>>>
>>>>>
>>>>> THIS MESSAGE IS INTENDED ONLY FOR THE ADDRESSEE,
>>>>> IT MAY CONTAIN PRIVILEGED OR CONFIDENTIAL INFORMATION.
>>>>> ANY UNAUTHORIZED DISCLOSURE IS STRICTLY PROHIBITED.
>>>>> IF YOU HAVE RECEIVED THIS MESSAGE IN ERROR,
>>>>> PLEASE NOTIFY ME IMMEDIATELY SO THAT I MAY CORRECT MY
>>>>> INTERNAL RECORDS. PLEASE THEN DELETE THE ORIGINAL MESSAGE.
>>>>> =-=-=-=-=-=-=-=-=-=-=-=-=
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>>
>

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13
   Current Beta Squid 3.1.0.6