Re: [squid-users] dns-related cache problem (squid 3.3.8)

On 10/10/2013 1:00 a.m., Dash Four wrote:
>
> Amos Jeffries wrote:
>> On 9/10/2013 9:39 a.m., Dash Four wrote:
>>> I have the following problem: I use the "hosts" file to store static
>>> address mappings, usually containing sites which use geo address
>>> mapping (in other words, determine the ip address one is going to
>>> use depending on the geographic region the request is made from).
>>> One such site is youtube for example.
>>>
>>> Squid caches web pages content from these sites, but when I change
>>> the IP address in "hosts", squid is still using the old IP address
>>> for some reason, even though I verified that the new mapping is used
>>> by the system.
>>>
>>> In such cases, I have to stop squid, wipe out the entire cache and
>>> then restart squid in order for the new host->ip mapping to be used.
>>> This can't be right!
>>>
>>> What I presume is happening is that somewhere along the line, squid
>>> is storing clear ip addresses in the cache (probably to speed up
>>> serving page content requests) and then it does not check whether
>>> that ip address has changed. If that is so, could this be fixed,
>>> because the idea of me clearing the entire cache every time I change
>>> one of the "hosts" mappings does not exactly appeal to me. Thanks.
>>
>> Sort of. Squid loads the hosts file contents into DNS result cache
>> with an infinite TTL in order to provide a single lookup mechanism
>> for any DNS entry and prevent remote lokups of those FQDN. These are
>> only loaded on startup and reconfigure time.
> Thanks Amos, but I am not sure what you describe above is actually
> happening.
>
> When the "hosts" file is updated, I tried to reload squid's
> configuration (-k reconfigure), but that did not help as squid was
> still trying to connect to the "old" ip address. I then stopped (-k
> shutdown) and then started squid without touching the existing cache -
> again, squid was still referring to the old host->ip mapping.

This is not possible in Squid DNS resolver. Squid ipcache is held only
in RAM. When you stop the process and restart it a whole new area of RAM
gets allocated and the ipcache starts from empty.
You can verify this using the "ipcache" cachemgr report, which lists all
the host->IP mappings known to Squid. After a restart the only entries
will be those freshly loaded from hosts file, names resovled while
loading the configuration file, and the newly resolved hostnames from
any traffic between sartup and your report fetch.

>
> The only way I "fixed" this is when I shutdown squid, delete the
> entire cache and then start it - only then the correct host->ip
> mapping was used, which would suggests that squid keeps the "old" ip
> addresses somewhere, I presume in the cache. I should also say this:
> the incorrect host->ip mapping is (mainly) used to retrieve secondary
> links (not main page urls), which appear as links on the main web pages.
>
> I initially thought that my web browser cache is to blame, but this
> was eliminated as a possibility since I manually wiped out the entire
> web browser cache and restarted the browser and still got the "old"
> mapping.
>
>
>> Squid at this time does not yet have any mechanism for watching the
>> file for changes (patches welcome!).
>> If your OS supports a filesystem trigger feature such as inotify or
>> dnotify you can script up a restart of Squid when hosts changes.
> Yes, inotify is fully supported, but my issue is with squid caching
> dns requests - that should not happen, or, at the very least, I should
> be able to disable this behaviour. DNS-caching is the job of the DNS
> server or, as in my case, the dnsmasq program I am using as dns
> resolver, not squid.

DNS has the same client/proxy/origin server model as HTTP, with the same
caching hierarchy model on a simpler scale. Authoritative resolvers are
the origins in DNS. Recursive resolvers are the proxies in DNS. And
software like Squid internal resolver are the "browser" equivalent -
with their own data cache.
  Squid internal resolver is a standards compliant (RFC 1035, 2181,
2874, 3226, 3596, 6762) DNS client and obeys the storage TTL associated
with each DNS record it has cached.

Amos
Received on Wed Oct 09 2013 - 12:19:55 MDT