Re: [squid-users] Re: cache deny, but still getting tcp hits

On 02/05/2010 01:11 PM, Amos Jeffries wrote:
> Isaac Witmer wrote:
>> On 02/04/2010 03:41 PM, Amos Jeffries wrote:
>>> Isaac Witmer wrote:
>>>> Sorry, I did a bad job of explaining.
>>>> I had SquidGuard as a url_rewrite_program redirecting all Ubuntu
>>>> Repository links to 10.42.43.1:9999, (the local net IP).
>>>> So Apt-proxy (working off this port) was doing the caching, and squid
>>>> shouldn't be. (in the case of a corrupt file getting cached, it would
>>>> only be in one place, instead of both)
>>>>
>>>> After looking in the cache.log file with debug_options All,3
>>>> 2010/02/04 13:51:28.288| ACLChecklist::preCheck: 0x85f8fb0 checking
>>>> 'cache deny ubuntu_repo'
>>>> 2010/02/04 13:51:28.288| ACLList::matches: checking ubuntu_repo
>>>> 2010/02/04 13:51:28.288| ACL::checklistMatches: checking 'ubuntu_repo'
>>>> 2010/02/04 13:51:28.289| aclMatchDomainList: checking '10.42.43.1'
>>>> 2010/02/04 13:51:28.289| aclMatchDomainList: '10.42.43.1' NOT found
>>>>
>>>> It turns out that I needed one extra 'cache deny' rule in this case. I
>>>> don't understand it entirely but it seems adding a cache deny rule for
>>>> "localnet" (which was already defined for my local area network) also
>>>> helped to blacklist the 10.42.43.1 ip address.
>>>>
>>>> -Isaac
>>>
>>> Ah. Okay.
>>>
>>> (I assume 10.42.43.1:9999 is the apt-proxy address and listening port?)
>>>
>>> What you need to do is drop the redirector and the special cache deny.
>>> And replace it with this:
>>>
>>> cache_peer 10.42.43.1 parent 9999 0 no-query proxy-only name=apt
>>>
>>> acl ubuntu dstdomain archive.ubuntu.com archive.canonical.com
>>> security.ubuntu.com ke.archive.ubuntu.com
>>>
>>> cache_peer_access apt allow ubuntu
>>> cache_peer_access apt deny all
>>> never_direct allow ubuntu
>>>
>>>
>>> NP: the "proxy-only" option is equivalent to "cache deny" but only
>>> affects objects successfully fetched through the peer proxy.
>>>
>>>
>>> Amos
>> So that makes squidguard's work irrelevant....
>> Is this more efficient? It makes sense that it would be...
>
> Yes. Lag time and delays processing the details for the re-writer and
> handling its results are omitted.
>
>>
>> The issue is that the urls have to be re-written more then just the
>> domain:
>> http://ke.archive.ubuntu.com/ubuntu/@ > http://10.42.43.1:9999/ubuntu/@
>> http://security.ubuntu.com/ubuntu/@ >
>> http://10.42.43.1:9999/ubuntu-security/@
>> http://archive.canonical.com/ubuntu/@ > http://10.42.43.1:9999/partner/@
>
> Do they? With the domain being passed back to apt-proxy, I would
> expect it can then use the domain name as part of the URL keys. You
> do not need to alter the path portion to cope with the identical
> "10.42.43.1:9999" 'domain' portions.
>
>>
>> (where @ is the trailing url simply passed on)
>>
>> I'm intrigued by the cache_peer idea, but I'm not sure if it works since
>> apt-proxy is built quite differently...
>
> AFAIK apt-proxy still acts as a proper HTTP source. Just the way it
> retrieves and caches things is a bit different from normal proxies.
>
> There are quite a few people using it this way.
>
> Amos
(sorry, somehow I failed to mail the mailing list on the last one)
I implemented squid the way you said (dropped my redirectors, and cache
deny... etc)

These are my results for the three repositories:
archive.ubuntu.com: worked (requests showed up properly in apt-proxy log)
security.ubuntu.com: went to the wrong repo (requests showed up as
pointing to the archive.ubuntu.com repo in the apt-proxy log)
archive.canonical.com: returned 404's and never showed up.

the thing is, the initial folder (10.42.43.1:9999/ubuntu or
10.42.43.1:9999/ubuntu-security or 10.42.43.1:9999/partner) each point
to the different repository. And these folder names are determined by
the apt-proxy configuration file.

I'd be interested in looking into it a bit further to figure out how
exactly it's working. It seems debug 4 wasn't quite enough for me to
figure out what's going on.
I have heard that apt-cacher works though, and with apt-proxy being
buggy... Maybe it's time to switch

Note:
my previous 'special cache deny' actually failed cause it simply blocked
caching for the local area network... I could 'cache deny' the url_regex
^http://10.42.43.1:9999.* but I'm not sure if it's worth it. I'm
interested in this new approach...
Received on Fri Feb 05 2010 - 13:25:27 MST