Re: [squid-users] squid TPROXY and empty access.log

Jack Daniels wrote:
> Hello,
> I've a problem to log client request activities when Squid is in TPROXY mode.
> In squid.conf I have 'access_log /var/log/squid/access.log squid',
> this file is correctly created but results empty.
>
> # ls -la /var/log/squid/
>
> -rw-r----- 1 proxy proxy 0 24 mar 16:09 access.log
> -rw-r----- 1 proxy proxy 413659 24 mar 16:09 cache.log
> -rw-r----- 1 proxy proxy 0 24 mar 16:09 referer.log
> -rw-r--r-- 1 root proxy 6 24 mar 16:09 squid.pid
> -rw-r----- 1 proxy proxy 0 24 mar 16:09 store.log
>
> If I remove 'tproxy' option from http_port in squid.conf it seems work
> (access.log is correctly written) but I lose the TPROXY Ip spoofing
> feature.
>
> Is this a bug?
> There is a way to log client request activities in tproxy mode? I need
> this feature to use Sarg in this environment.
>

Sounds like a bug, but there are a few things below you need to clear up
before you can be sure its not them...

> Thanks in advance.
>
>
> Here some details:
>
> # uname -r
> 2.6.28.8
>
> # squid -v
> Squid Cache: Version 3.1.0.6
> configure options: '--enable-linux-tproxy' '--enable-linux-netfilter'

http://wiki.squid-cache.org/Features/Tproxy4

"Obsolete --enable-tproxy option. Remains only for legacy v2.2 cttproxy
support."

It does things to the kernel TPROXYv4 may not like. Kill it.

> '--enable-ssl' '--with-openssl=/usr/include/openssl/'
> '--enable-err-languages=EnglisItalian'
> '--enable-default-err-language=Italian'

the err options are also DEAD.

> '--disable-htcp'
> '--with-large-files' '--prefix=/usr' '--exec_prefix=/usr'
> '--bindir=/usr/sbin' '--sbindir=/usr/sbin' '--libdir=/usr/lib/squid'
> '--libexecdir=/usr/lib/squid' '--docdir=/usr/share/doc/squid'
> '--datarootdir=/usr/share/squid' '--datadir=/usr/share/squid'
> '--sysconfdir=/etc/squid' '--localstatedir=/var/spool/squid'
> '--mandir=/usr/share/man' '--with-logdir=/var/log/squid'
> '--enable-inline' '--enable-async-io=8' '--with-pthreads'
> '--enable-storeio=ufs,aufs,diskd' '--enable-removal-policies=lru,heap'
> '--enable-snmp' '--enable-delay-pools' '--enable-cache-digests'
> '--enable-underscores' '--enable-icap-client' '--enable-referer-log'
> '--enable-useragent-log' '--enable-follow-x-forwarded-for'
> '--enable-auth=basic,digest,ntlm' '--enable-basic-auth-helpers=NCSA'
> '--enable-digest-auth-helpers=password'
> '--enable-external-acl-helpers=ip_user,unix_group'
> '--with-filedescriptors=65536' '--with-default-user=proxy'
> --with-squid=/usr/src/squid-3.1.0.6 --enable-ltdl-convenience
>
> # iptables -V
> iptables v1.4.3-rc1
>
> # iptables -t mangle -L
> Chain PREROUTING (policy ACCEPT)
> target prot opt source destination
> DIVERT tcp -- anywhere anywhere socket
> TPROXY tcp -- anywhere anywhere tcp
> dpt:www TPROXY redirect 0.0.0.0:3128 mark 0x1/0x1
>
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
>
> Chain FORWARD (policy ACCEPT)
> target prot opt source destination
>
> Chain OUTPUT (policy ACCEPT)
> target prot opt source destination
>
> Chain POSTROUTING (policy ACCEPT)
> target prot opt source destination
>
> Chain DIVERT (1 references)
> target prot opt source destination
> MARK all -- anywhere anywhere MARK xset
> 0x1/0xffffffff
> ACCEPT all -- anywhere anywhere
>
> # cat /etc/squid/squid.conf
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32
> acl to_localhost dst 127.0.0.0/8
> acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
> acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localnet
> http_access deny all

> http_port 3128 tproxy
> #http_port 3128

http://wiki.squid-cache.org/Features/Tproxy4

"NP: A dedicated squid port for tproxy is REQUIRED"

> hierarchy_stoplist cgi-bin ?
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
> coredump_dir /var/spool/squid/cache
> cache_effective_user proxy
> cache_effective_group proxy
> access_log /var/log/squid/access.log squid
> cache_store_log /var/log/squid/store.log
> referer_log /var/log/squid/referer.log

> log_access allow all
> cache allow all

Two very redundant settings.

>
> # squid -X -d1 -N
<snip squid.conf parse>
> 2009/03/24 16:04:06.961| Initializing https proxy context
> 2009/03/24 16:04:06.964| Using SSLv2/SSLv3.
> 2009/03/24 16:04:06.965| Setting RSA key generation callback.
> 2009/03/24 16:04:06.965| Setting certificate verification callback.
> 2009/03/24 16:04:06.965| Setting CA certificate locations.
> 2009/03/24 16:04:06.965| leave_suid: PID 11891 called
> 2009/03/24 16:04:06.965| leave_suid: PID 11891 giving up root, becoming 'proxy'

> 2009/03/24 16:04:06.965| Stopping full transparency: Missing needed
> capability support.

That looks serious. It has something to do with libcap-dev.

> 2009/03/24 16:04:06.965| command-line -X overrides: ALL,1
> 2009/03/24 16:04:06.966| Starting Squid Cache version 3.1.0.6 for
> i686-pc-linux-gnu...
> 2009/03/24 16:04:06.966| Process ID 11891
> 2009/03/24 16:04:06.966| With 1024 file descriptors available
> 2009/03/24 16:04:06.966| Initializing IP Cache...
> 2009/03/24 16:04:06.966| ipcacheAddEntryFromHosts: Bad IP address '::1'
> 2009/03/24 16:04:06.966| ipcacheAddEntryFromHosts: Bad IP address 'fe00::0'
> 2009/03/24 16:04:06.967| ipcacheAddEntryFromHosts: Bad IP address 'ff00::0'
> 2009/03/24 16:04:06.967| ipcacheAddEntryFromHosts: Bad IP address 'ff02::1'
> 2009/03/24 16:04:06.967| ipcacheAddEntryFromHosts: Bad IP address 'ff02::2'
> 2009/03/24 16:04:06.967| ipcacheAddEntryFromHosts: Bad IP address 'ff02::3'
> 2009/03/24 16:04:06.967| DNS Socket created at 0.0.0.0, FD 5
> 2009/03/24 16:04:06.967| Adding domain isiline.net from /etc/resolv.conf
> 2009/03/24 16:04:06.967| Adding nameserver 213.144.64.1 from /etc/resolv.conf
> 2009/03/24 16:04:06.967| Adding nameserver 213.144.66.1 from /etc/resolv.conf
> 2009/03/24 16:04:06.968| User-Agent logging is disabled.
> 2009/03/24 16:04:07.234| Unlinkd pipe opened on FD 11
> 2009/03/24 16:04:07.238| Local cache digest enabled; rebuild/rewrite
> every 3600/3600 sec
> 2009/03/24 16:04:07.238| Swap maxSize 0 KB, estimated 0 objects
> 2009/03/24 16:04:07.238| Target number of buckets: 0
> 2009/03/24 16:04:07.238| Using 8192 Store buckets
> 2009/03/24 16:04:07.238| Max Mem size: 262144 KB
> 2009/03/24 16:04:07.238| Max Swap size: 0 KB
> 2009/03/24 16:04:07.238| Using Least Load store dir selection
> 2009/03/24 16:04:07.239| Set Current Directory to /var/spool/squid/cache
> 2009/03/24 16:04:07.335| Loaded Icons.
> 2009/03/24 16:04:07.336| Accepting spoofing HTTP connections at
> 0.0.0.0:3128, FD 13.
> 2009/03/24 16:04:07.336| HTCP Disabled.
> 2009/03/24 16:04:07.337| Squid modules loaded: 0
> 2009/03/24 16:04:07.337| Adaptation support is off.
> 2009/03/24 16:04:07.337| Ready to serve requests.

Once the above issues are resolved. I think you will need a cache.log
trace of whats happening when a request goes through.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13
   Current Beta Squid 3.1.0.6