Re: [squid-users] Update Accelerator, Squid and Windows Update Caching

Amos Jeffries wrote:
> Richard Wall wrote:
>> On Fri, Oct 10, 2008 at 12:30 PM, Amos Jeffries
>> <squid3_at_treenet.co.nz> wrote:
>>> Richard Wall wrote:
>>>> Hi,
>>>>
>>>> I've been reading through the archive looking for information about
>>>> squid 2.6 and windows update caching. The FAQ mentions problems with
>>>> range offsets but it's not really clear which versions of Squid this
>>>> applies to.
>>> All versions. The FAQ was the result of my experiments mid last
>>> year. With
>>> some tweaks made early his year since Vista came out.
>>> We haven't done a intensive experiments with Vista yet.
>>
>> Hi Amos,
>>
>> I'm still investigating Windows Update caching (with 2.6.STABLE17/18)
>>
>> First of all, I have been doing some tests to try and find out the
>> problem with Squid and Content-Range requests.
>> * I watch the squid logs as a vista box does its automatic updates
>> and I can see that *some* of its requests use ranges. (so far I have
>> only seen these when it requests .psf files...some of which seem to be
>> very large files...so the range request makes sense) See:
>> http://groups.google.hr/group/microsoft.public.windowsupdate/browse_thread/thread/af5db07dc2db9713
>>
>>
>> # zcat squid.log.192.168.1.119.2008-10-16.gz | grep
>> "multipart/byteranges" | awk '{print $7}' | uniq | while read URL; do
>> echo $URL; wget --spider $URL 2>&1 | grep Length; done
>> http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/10/windows6.0-kb956390-x86_2d03c4b14b5bad88510380c14acd2bffc26436a7.psf
>>
>> Length: 91,225,471 (87M) [application/octet-stream]
>> http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/05/windows6.0-kb950762-x86_0cc2989b92bc968e143e1eeae8817f08907fd715.psf
>>
>> Length: 834,868 (815K) [application/octet-stream]
>> http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/03/windows6.0-kb948590-x86_ed27763e42ee2e20e676d9f6aa13f18b84d7bc96.psf
>>
>> Length: 755,232 (738K) [application/octet-stream]
>> http://www.download.windowsupdate.com/msdownload/update/software/crup/2008/09/windows6.0-kb955302-x86_1e40fd3ae8f95723dbd76f837ba096adb25f3829.psf
>>
>> Length: 7,003,447 (6.7M) [application/octet-stream]
>> ...
>>
>> * I have found that curl can make range requests so I've been using
>> it to test how Squid behaves....and it seems to do the right thing. eg
>> - First ask for a range : The correct range is returned X-Cache: MISS
>> - Repeat the range request : The correct range is returned X-Cache:
>> MISS
>> - Request the entire file: The entire file is correctly returned
>> X-Cache: MISS
>> - Repeat the request: X-Cache: HIT
>> - Repeat the previous range request: X-Cache: HIT
>> - Request a different range: X-Cache: HIT
>>
>> curl --range 1000-1002 --header Pragma: -v -x http://127.0.0.1:3128
>> http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/05/windows6.0-kb950762-x86_0cc2989b92bc968e143e1eeae8817f08907fd715.psf
>>
>>> /dev/null
>>
>> Looking back through the archive I find this conversation from 2005:
>> http://www.squid-cache.org/mail-archive/squid-users/200504/0669.html
>>
>> ...but the behaviour there sounds like a result of setting:
>> range_offset_limit -1
>>
>> Seems to me that Squid should do a good job of Windows Update caching.
>> There is another thread discussing how to override MS update cache
>> control headers:
>> http://www.squid-cache.org/mail-archive/squid-users/200508/0596.html
>>
>> ....but I don't see anything evil in the server response headers
>> today. I guess the client may be sending no-cache headers...I'll
>> double check that later.
>>
>> Is there some other case that I'm missing?
>
> As I said. I have not seen Vista in detail. I just had to turn off my
> old hack to get around the SP1 hanging. (that huge .psf perhapse?)
> Never had to do anything with headers.
>
> When I did my testing it was with outdated Win98-WinXP machines (often
> needing SP1 in XP's case).
>
> The WU on them made an HTTPS request (seems to be auth-related even
> today) requested one or more update indexes fine. Then proceeded to
> random-access range requests out of the middle of the update *.cabs
> using dynamic urls at various update sites.
> This was causing bandwidth blowout with all the MISS'es when I had
> several machines a week coming through.
>
> I _think_ but have no confirmation, that the early patch-tuesday
> releases were done as large single .CAB files and a particular machine
> may only need updating from individual fixes inside them.
>
> As your test showed, fetching the whole file squid can handle the
> ranges fine. It's when they are still in MISS state that ranges become
> trouble.
>
>>
>>>> I'm going to experiment, but if anyone has any positive or
>>>> negative experience of Squid and windows update caching, I'd be really
>>>> interested to hear from you.
>>>>
>>>> In case Squid cannot do windows update caching by its self, I'm also
>>>> looking at integrating Update Accelerator
>>>> (http://update-accelerator.advproxy.net/) script with standard squid
>>>> 2.6 and wondered if anyone else had any experience of this.
>>>> The update accelerator script is just a perl wrapper around wget which
>>>> is configured as a Squid url_rewrite_program. It's not clear to me
>>>> what this script is doing that Squid wouldn't do by its self.
>>> Strange indeed.
>>
>> I got update accelerator working with Squid but I'm still not
>> convinced that it's necessary (see above).
>>
>> -RichardW.
>
>
Hi Guys,

We've implemented our own 'flavour' of the cache accelerator for
windows-updates and have been rather happy with it. Microsoft have
gotten rather funky and are appending datetime stamps to the cab files
in an attempt to reduce their cachability. If you strip the DT stamp and
make a request the correct file is still fetched from the windows update
sites.

I'm trying to move towards a combination of stroreurl_rewrite_program to
try handle these files inside squid instead of externally.

If anyone feels they have windowsupdates working 100% without external
rewriters their comments would be greatly appreciated.
Received on Thu Oct 23 2008 - 13:27:35 MDT