Re: [squid-users] Reverse Proxy Funny Logging Issue

On 2013-01-23 08:40, dweimer wrote:
> On 2013-01-22 23:30, Amos Jeffries wrote:
>> On 23/01/2013 5:34 a.m., dweimer wrote:
>>> I just upgraded my reverse proxy server last night from 3.1.20 to
>>> 3.2.6, all is working well except one of my log rules, and I can't
>>> figure out why.
>>
>> Please run "squid -k parse" and resolve the WARNING or ERROR which
>> are listed.
>>
>> There are two possible reasons...
>>
>>>
>>> I have a several sites behind the server, with dstdomain access
>>> rules setup.
>>>
>>> acl website1 dstdomain www.website1.com
>>> acl website2 dstdomain www.website2.com
>>> acl website2 dstdomain www.website3.com
>>
>> Possible reason #1 (assuming thi is an accurate copy-n-paste from
>> yoru config file)..... you have no website3 ACL definition?
>
> That was a typo in the email, correct ACL is in the configuration,
> squid -k parse outputs no warnings or errors.
>
>>
>>> ...
>>>
>>> Followed by the access rules
>>>
>>> http_access allow website1
>>> http_access allow website2
>>> http_access allow website3
>>> ...
>>> http_access deny all
>>>
>>> Some are using rewrites
>>> url_rewrite_program /usr/local/etc/squid/url_rewrite.py
>>> url_rewrite_children 20
>>> url_rewrite_access allow website1
>>> url_rewrite_access allow website3
>>> ...
>>> url_rewrite_access deny all
>>>
>>> Then my access logs
>>>
>>> # First I grab everything in one
>>> access_log daemon:/var/log/squid/access.log squid all
>>>
>>> # Then individual logs for each webiste, to be parsed with
>>> webalizer
>>> logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st
>>> "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
>>
>> Possible problem #2. "combined" is now a built-in format name
>> beginning with 3.2 series.
>> * If you want the Apache format, please remove this line and use
>> the
>> built-in formater.
>> * If you want your own alterations to the format, please use
>> another name.
>
> Checking on this and will make some adjustments, to see if this ends
> up being the cause.

Dropped the logformat combined line, parse still completed no errors,
restart no-change.

>>
>>> access_log daemon:/var/log/squid/website1.log combined website1
>>> access_log daemon:/var/log/squid/website2.log combined website2
>>> access_log daemon:/var/log/squid/website3.log combined website3
>>> ...
>>>
>>> everything works, write down to one of the access logs, the data
>>> shows up in the access.log file, the data shows up in the individual
>>> logs for all the others, except that one. If we use website3 from
>>> the above example like my actual file the access rule works on the
>>> url_rewrite_access allow line, but for some reason is failing on the
>>> log line. squid -k parse doesn't show any errors, and shows a
>>> Processing: access_log daemon:/var/log/squid/website3.log combined
>>> website3 line in the output.
>>>
>>> The log in question was originally at the end of my access_log list
>>> section, so I changed the order around to see if for some reason it
>>> was only the last one not working, no change still only that one not
>>> working, And the new last one in the list still works as expected.
>>>
>>> I know the ACL is working as it works correctly on the rewrite rule
>>> and the http access just above the log rules, anyone have any ideas
>>> on how I can figure out why the log entry isn't working?
>
>> Is the domain name in question being rewritten by the helper? That
>> would cause it to be accepted in http_access then
>> url_rewrite_access,
>> then different for logging.
>
> Yes it is being rewritten, however so are a few others, which are
> properly logged, will also try moving the log statements above the
> url_rewrite statements to see if that makes a difference.

Moved lines above url_rewrite no change, but did notice that even
though some appeared to be working not all hits were being logged, only
some were.

Changed all lines to use stdio, instead of daemon, thought maybe there
was a bug, no change again.

Changed lines back to daemon, changed acl on logs to the rewrite side
used on the cache_peer_access lines later in the configuration. Works
now, and logs even show up with the pre-rewrite rule host information...

That does make me wonder why some lines were getting logged but not
all, the sites I thought were working do have higher usage, maybe I was
still missing a lot from them, and just not knowing it. I guess I will
see if my webalizer reports show a huge gain in hit count over the old
records from the the 3.1.20 installation, of if this behavior is only
evident in the 3.2 branch.

-- 
Thanks,
    Dean E. Weimer
    http://www.dweimer.net/