Re: [squid-users] Squid 3.2 Logging, and HTTPS post problem

On 26/01/2013 6:11 p.m., dweimer wrote:
> If your asking yourself why is Logging and HTTPS posts a problem. I
> have no idea either, but turns out two of my recent posts are related.
> somehow, in that the fix for the logging issue caused the HTTPS post
> issue.
>
> Logging issue:
> http://www.squid-cache.org/mail-archive/squid-users/201301/0301.html
>
> End result move some additional access_log entries that create
> separate logs for each domain above some rewrite rules to fix some
> requests not getting logged.
>
> HTTPS post issue, resulting in TCP_MISS_ABORTED/000, and post not
> succeeding:
> http://www.squid-cache.org/mail-archive/squid-users/201301/0320.html
>
> Couldn't solve this one, put old server running 3.1.21 back in
> production.
>
> Ran some testing with debugging level raised to 2, capturing logs.
> Oddly enough the post still didn't work but the TCP_MISS_ABORTED/000
> disappeared from the access log, and a normal TCP_MISS/200 entry shows
> up as if it worked.

Then it did work. The 200 is a value received from upstream servers. You
would still have 000 otherwise.

>
> While putting together a information with the logs from the debugging,
> and thinking about what has changed since the upgrade to 3.2.6 from
> 3.1.21 other than the new version. I remember this access_log change,
> and with the change in what showed up with increased debugging. I get
> an idea, what if this is causing that is causing the issue... So I
> test and move the access log entries back below the url_rewrite
> rules. Well I am on to something, the HTTPS posts work again, so I
> start trying to think why? I put them back above the rewrite logs,
> and set the to use stdio instead of the new daemon method. Problems
> back again, so its not specific to the new daemon logging method. I
> am left with thinking its either something in my configuration, or
> there is an I/O blocking issue of some sort. Here is the full
> configuration with domains changed, perhaps someone can see what I am
> missing that's causing the problem or suggest what I should be looking
> for to track down whats blocking the post.
>
> ## Define Listening Port Options
> # Port 80 HTTP traffic
> http_port 10.50.20.10:80 accel defaultsite=www.website1.com
> # Port 443 HTTPS traffic
> sslproxy_options NO_SSLv2:NO_TLSv1:CIPHER_SERVER_PREFERENCE
> sslproxy_cipher RC4:!MD5:!aNULL:!EDH
> https_port 10.50.20.10:443 accel defaultsite=www.website1.com \
> cert=/usr/local/etc/squid/certs/orscheln-bundle.crt \
> key=/usr/local/etc/squid/certs/orscheln.key \
> options=NO_SSLv2:NO_TLSv1:CIPHER_SERVER_PREFERENCE \
> cipher=RC4:!MD5:!aNULL:!EDH
>

Well one thing to be aware of that may be related is a change in vhost.
I n3.1 series the above http_port settings used to force every incoming
request to have FQDN "www.website1.com". With 3.2 the Host: header is
used to reconstruct the URL.
  URL-rewrite helper operates on the URL after this reconstruction happens.

> ## Access Control Lists
> # Port ACL Lists
> acl PORT80 port 80
> acl PORT443 port 443
> # Network ACL Lists
> acl DMZ src 10.50.0.0/16
> acl LAN src 10.0.0.0/8 192.168.0.0/16 172.16.0.0/12
> acl OPMANAGER src 10.20.10.55/32
> # Protocol ACL Lists
> acl SSL proto HTTPS
> # Method ACL Lists
> acl PURGE method PURGE
> # Destination Domain ACL Lists
> acl WEBSITE1COM dstdomain www.website1.com website1.com
> acl WEBSITE2COM dstdomain www.website2.com website2.com
> acl WEBSITE3COM dstdomain www.website3.com website3.com
> acl WEBSITE4COM dstdomain www.website4.com website4.com
> acl WEBSITE5COM dstdomain www.website5.com website5.com
> acl WEBSITE6COM dstdomain www.website6.com website6.com
> acl REVPROXYCOM dstdomain revproxy1.website1.com
> # Url Regex ACL Lists
> acl FILETRANSFER url_regex -i ^https://www.website1.com/FileManager
> acl ECLUB url_regex -i ^https://www.website4.com/SSL/E-Club/
> acl EMPAPP url_regex -i
> ^https://www.website1.com/SSL/EmploymentApplication/
> acl WEBSVRSSL url_regex -i /SSL/
> acl OTHERSSL url_regex -i ^https
> acl WEBSITE1 url_regex -i ^http://websvr:8130
> acl WEBSITE4 url_regex -i ^http://websvr:8090
> acl WEBSITE5 url_regex -i ^http://websvr:8110
> acl WEBSITE6 url_regex -i ^http://websvr:8120
> acl PLONE url_regex -i "/usr/local/etc/squid/plone.acl"
> acl ZOPE url_regex -i "/usr/local/etc/squid/zope.acl"
>
> ## Access Rules
> # Deny First For Security
> http_access deny PLONE !LAN
> http_access deny ZOPE !LAN
> http_access deny MANAGER !LAN
> http_access deny PURGE !LAN
> http_access deny WEBSVRSSL !SSL
> http_access deny EMPAPP !SSL
> http_access deny ECLUB !SSL
> # Allow Specific SSL
> http_access allow FILETRANSFER SSL
> http_access allow WEBSVRSSL SSL
> http_access allow WEBSITE4COM SSL
> http_access allow ECLUB SSL
> http_access allow WEBSITE1COM SSL
> http_access allow EMPAPP SSL
> http_access allow WEBSITE1HRCOM SSL
> http_access allow REVPROXYCOM SSL
> # Prevent Other SSL
> http_access deny OTHERSSL
> http_access deny SSL
> # Allow Non SSL stuff
> http_access allow MANAGER LOCALHOST
> http_access allow PURGE LOCALHOST
> http_access allow WEBSITE1COM
> http_access allow WEBSITE2COM
> http_access allow WEBSITE3COM
> http_access allow WEBSITE4COM
> http_access allow REVPROXYCOM
> http_access allow WEBSITE5COM
> http_access allow WEBSITE6COM
> # Deny Last For Security
> http_access deny all
>
> # Uncomment and adjust the following to add a disk cache directory.
> cache_dir ufs /var/squid/cache 5120 16 256
> maximum_object_size 8 MB
>
> # Set Cache Log Options
> debug_options rotate=20 ALL,2
> cache_log /var/log/squid/cache.log
>
> # Set Cache Access Logs
> access_log daemon:/var/log/squid/access.log squid all
> ## Site Logs
> # WEBSITE1 Hits
> access_log daemon:/var/log/squid/website1.log combined WEBSITE1
> !FILETRANSFER
> # File Transfer Hits
> access_log daemon:/var/log/squid/filetransfer.log combined FILETRANSFER
> # WEBSITE2 Hits
> access_log daemon:/var/log/squid/website2.log combined WEBSITE2COM
> # WEBSITE3 Hits
> access_log daemon:/var/log/squid/website3.log combined WEBSITE3COM
> # WEBSITE4 hits
> access_log daemon:/var/log/squid/website4.log combined WEBSITE4
> # WEBSITE5 Hits
> access_log daemon:/var/log/squid/website5.log combined WEBSITE5
> # WEBSITE6 Hits
> access_log daemon:/var/log/squid/website6.log combined WEBSITE6
> # Local Host Hits (REVPROXY1)
> access_log daemon:/var/log/squid/revproxy1.log combined REVPROXYCOM
>
> # URL Rewrites
> url_rewrite_program /usr/local/etc/squid/url_rewrite.py
> url_rewrite_children 20
> url_rewrite_access allow WEBSITE1COM !SSL
> url_rewrite_access allow WEBSITE4COM !SSL
> url_rewrite_access allow WEBSITE5COM
> url_rewrite_access allow WEBSITE6COM
> url_rewrite_access deny all
>
> # Set Cache Log Options
> debug_options rotate=20 ALL,2
> cache_log /var/log/squid/cache.log
>
> # Leave coredumps in the first cache dir
> coredump_dir /var/squid/cache
>
> # Add any of your own refresh_pattern entries above these.
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
>
> # Set Email of Cache Administrator
> cache_mgr dweimer_at_website1.com
>
> # Define Password for Cache Manager CGI
> cachemgr_passwd ASuperSecretPasswordWasHere all
>
> # Set effective user and group for Cache
> cache_effective_user squid
> cache_effective_group squid
>
> # Set vissable Host Name, for error messages
> visible_hostname www.website1.com
>
> # Define Custom Error Pages
> err_page_stylesheet /usr/local/etc/squid/custom_errors/errorpage.css
> error_directory /usr/local/etc/squid/custom_errors
>
> # DNS servers
> dns_nameservers 10.50.20.5 10.50.20.4
>
> # Hosts File
> hosts_file /etc/hosts
>
> ## Define Parents
> # http://www.website1.com
> # http://website1.com
> cache_peer websvr parent 8130 0 no-query no-digest no-netdb-exchange
> originserver name=website1_parent
> cache_peer_domain website1_parent websvr
> cache_peer_access website1_parent allow WEBSITE1 !SSL
> cache_peer_access website1_parent deny all
> # https://www.website1.com/FileManager
> cache_peer 127.0.0.1 parent 443 0 ssl no-query no-digest
> no-netdb-exchange originserver name=website1_transfer_parent
> sslcapath=/usr/local/share/certs sslflags=DONT_VERIFY_PEER
> cache_peer_domain website1_transfer_parent www.website1.com website1.com
> cache_peer_access website1_transfer_parent allow FILETRANSFER SSL
> cache_peer_access website1_transfer_parent deny all
> # http://www.website2.com
> # http://website2.com
> cache_peer 127.0.0.1 parent 80 0 no-query no-digest no-netdb-exchange
> originserver name=website2_parent
> cache_peer_domain website2_parent www.website2.com website2.com
> cache_peer_access website2_parent allow WEBSITE2COM
> # http://www.website3.com
> # http://website3.com
> cache_peer 127.0.0.1 parent 80 0 no-query no-digest no-netdb-exchange
> originserver name=website3_parent
> cache_peer_domain website3_parent www.website3.com website3.com
> cache_peer_access website3_parent allow WEBSITE3COM
> # https://www.website4.com
> # https://www.website4.com/SSL/E-club
> # https://www.website1.com
> cache_peer websvr parent 443 0 ssl no-query no-digest
> no-netdb-exchange originserver name=websvr_secure
> sslcapath=/usr/local/share/certs sslflags=DONT_VERIFY_PEER
> cache_peer_access websvr_secure allow WEBSITE1COM SSL
> cache_peer_access websvr_secure allow WEBSITE4COM SSL
> cache_peer_access websvr_secure allow ECLUB SSL
> cache_peer_access websvr_secure allow EMPAPP SSL
> cache_peer_access websvr_secure deny all
> # http://www.website4.com
> cache_peer websvr parent 8090 0 no-query no-digest no-netdb-exchange
> originserver name=website4_parent
> cache_peer_domain website4_parent websvr
> cache_peer_access website4_parent allow WEBSITE4
> cache_peer_access website4_parent deny all
> # http://www.website5.com
> # http://website5.com
> cache_peer websvr parent 8110 0 no-query no-digest no-netdb-exchange
> originserver name=website5_parent
> cache_peer_domain website5_parent websvr
> cache_peer_access website5_parent allow orschelnproperties
> # http://www.website6.com
> # http://website6.com
> cache_peer websvr parent 8120 0 no-query no-digest no-netdb-exchange
> originserver name=website6_parent
> cache_peer_domain website6_parent websvr
> cache_peer_access website6_parent allow WEBSITE6
> # https://revproxy1.website1.com
> cache_peer 127.0.0.1 parent 443 0 ssl no-query no-digest
> no-netdb-exchange originserver name=local_ssl_parent
> sslcapath=/usr/local/share/certs sslflags=DONT_VERIFY_PEER
> cache_peer_domain local_ssl_parent revproxy1.website1.com
> cache_peer_access local_ssl_parent allow REVPROXYCOM SSL
> # http://revproxy1.website1.com
> cache_peer 127.0.0.1 parent 80 0 no-query no-digest no-netdb-exchange
> originserver name=local_parent
> cache_peer_domain local_parent revproxy1.website1.com
> cache_peer_access local_parent allow REVPROXYCOM !SSL
>
>

NOte that cache_peer_domain and cache_peer_access are conflicting acess
controls. Use one or the other.
I suggest going with only cache_peer_access since you have criteria
other than FQDN in the selection criteria.

Amos
Received on Sat Jan 26 2013 - 06:25:43 MST