Re: [squid-users] Pessimal behavior with Windows Update (Long)

At 05:08 AM 6/30/2005, Henrik Nordstrom wrote:

>>Unfortunately, this causes yet another problem to surface. If the file in question is larger than "maximum_object_size" (the maximum object size defined for the cache), Squid retrieves the entire file in response to each request but then fails to store it!
>
>Is this verified with 2.5.STABLE10, or only earlier versions?

I have not verified it with 2.5.STABLE10. However, there was a message in this
forum a week or two ago which suggested that this behavior still occurred.

>>1. Modify Squid so that before it attempts to prefetch an entire object in response to a range request, it first checks the size of the object against "maximum_object_size". It should also perform all the other checks it normally performs to determine whether an object is cacheable (including evaluation of the ACLs, if any, used with the "no_cache" parameter). If the entire file is not cacheable for any reason whatsoever, the range request should be passed through verbatim. (This seems like common sense, but it is apparently not done now.)
>
>Problem with this is that to do this Squid has to forward the request to the web server, at which point it is too late to change the request to/from being a partial request.

One could always do a "HEAD" first. It would be worth it in terms of time,
especially if the result was remembered for awhile.

>>2. Allow the "maximum_object_size" parameter to be selected, for each transfer, via an ACL. (Fortuitously, the standard syntax of the Squid configuration file both allows this and provides backward compatibility in the case where no ACL is specified. See the "tcp_outgoing_address" parameter, which acquired the ability to use ACLs only recently and is backward compatible with configuration files that don't exploit this new capability.) With this modification, an administrator could allow the caching of large Windows Update files but not large files from other sources.
>
>Problem with this is developer time. There currently is very few developers working on Squid.

I wish I could help. If the code were not GPLed I'd dive right in. But as a commercial
developer I cannot work on GPLed code, alas.

>>3. If a range transfer is to be expanded into a transfer of the entire file, exempt the transfer from the "quick_abort" mechanism. (After all, it's normal for the client to disconnect after it receives the data it has requested.)
>
>Same as 2.

If the cache can revert to forwarding the request, there should not be a problem.

>>4. Encourage Microsoft to modify Windows Update so that it can "discover" a server on which updates are preloaded or cached. Currently, SUS and WUS require modification to a client machine's registry; this is practical for organizations with IT staffs but not for ISPs.
>
>Historically Windows Update has became increasingly less and less friendly to caching.

I don't know if this is intentional. After all, Microsoft pays Akamai huge amounts of money
to distribute its updates, and I'm sure it would like to pay them less. It seems just to be
a side effect of fetching the file in chunks.

>As a result it's my impression they do not want Windows Update to be cached by ISPs. But it may simply be an lack of thought.

There's an old saying: "Never assume malice when incompetence will do." ;-)

>Would probably help if some large ISPs tried to talk them to senses, but I suppose their preferred solution would be to simply set up an Akamai at the ISP..

Akamai refuses to talk to us. Won't even return phone calls. Unless you're a huge ISP,
they won't give you the time of day. (Which begs the question, "How can you ever become one
if your bandwidth is all consumed by Windows Update?" The answer, of course, is that you
must cache on your own.)

--Brett Glass
Received on Thu Jun 30 2005 - 11:25:42 MDT