[squid-users] Regarding long pauses with Squid3 as a reverse proxy to Exchange 2010 OWA

Apologies in advance for the long posting. I have tried to provide what I
hope is sufficient information to explain a problem I am experiencing.

The problem is very similar to that reported to this list by James Nguy on
March 17 2010 -
http://www.squid-cache.org/mail-archive/squid-users/201003/0397.html.
Briefly this involves a significant pause(s) when loading the Exchange 2010
Outlook Web App (OWA) page after login when using squid3 as a reverse proxy.
I observe two pauses, each of approx. 120 sec. Once these pass, OWA works
smoothly. Closing the browser and restarting it and logging in again works
without any of the long pauses. If the browser cache is cleared and a new
connection to OWA is made, the pauses return.

I am using squid v3.0stable19 running on Ubuntu 10.04TLS compiled from the
Ubuntu source but with ssl enabled. Exchange 2010 (not SP1) is running on
Windows 2008 R2 (not SP1).

The problem does not appear to be related to the browser as I have reproduced
the problem with Firefox 3.5.3, IE 7 and IE 8. I have also confirmed the
problem exists when squid communicates with Exchange 2010 via http or https.
The client is connecting via https. If I connect directly to exchange from
the client, the pauses do not occur.

My investigations so far have involved trying to determine what requests are
taking longer than expected to complete. I also changed my configuration so
that squid communicated with exchange using http instead of https so I could
collect packet traces. For the investigations, I am connecting to squid from
the client browser via the local LAN - not via the internet, just to simplify
things. Most packet traces have been collected on the squid reverse proxy,
collecting traffic between squid and exchange.

My preliminary findings are that there are 2 GET requests to exchange for
javascript files that are taking approx. 120 seconds to complete. These
files are approx. 330KB and 800KB. I am by no means an expert at analysing
packet traces but so I apologise if my analysis is faulty or incomplete:-)

Looking at the packet traces, I see that exchange is returning all the data
to squid but that after a long pause (approx. 120sec) exchange (IIS) issues
an RST. I presume this might be a connection timeout in IIS 7.5 (which is
running OWA).

As far as I can tell, exchange has sent all the data to fulfil the GET.
However I think squid is still waiting for more and eventually exchange (IIS)
sends a RST to reset the connection. See below for some of the output from
wireshark at the end of the session:

90 15:57:49.024563 192.168.110.13 192.168.200.15 HTTP Continuation or
non-HTTP traffic
91 15:57:49.024659 192.168.200.15 192.168.110.13 TCP 34732 > http
[ACK] Seq=774 Ack=68161 Win=44928 Len=0 TSV=7798268 TSER=787616
92 15:57:49.025563 192.168.200.15 192.168.110.13 TCP 34732 > http
[ACK] Seq=774 Ack=88943 Win=27072 Len=0 TSV=7798268 TSER=787616
93 15:57:49.027238 192.168.200.15 192.168.110.13 TCP [TCP Window
Update] 34732 > http [ACK] Seq=774 Ack=88943 Win=54560 Len=0 TSV=7798268
TSER=787616
94 16:00:00.497321 192.168.110.13 192.168.200.15 TCP http > 34732
[RST, ACK] Seq=88943 Ack=774 Win=0 Len=0

Note: 192.168.200.15 is the squid reverse proxy and 192.168.110.13 is the
exchange 2010 server.

Looking at the tcp stream view of the same data using Wireshark, I see the
following:

GET /owa/14.0.702.0/scripts/premium/startpage.js HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword,
application/x-shockwave-flash, application/x-ms-application,
application/x-ms-xbap, application/vnd.ms-xpsdocument, application/xaml+xml,
*/*
Accept-Language: en-au
UA-CPU: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR
1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022;
.NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
Host: the.squid.reverse.proxy
Via: 1.1 the.squid.reverse.proxy (squid/3.0.STABLE19)
X-Forwarded-For: 192.168.110.140
Cache-Control: max-age=259200
Connection: keep-alive

HTTP/1.1 200 OK
Cache-Control: public,max-age=2592000
Content-Type: application/x-javascript
Content-Encoding: gzip
Last-Modified: Fri, 25 Sep 2009 04:59:56 GMT
Accept-Ranges: bytes
ETag: "c08cce69d3dca1:0"
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
X-UA-Compatible: IE=EmulateIE7
Date: Wed, 25 Aug 2010 05:57:51 GMT
Connection: close

After reading a number of posts on Google which looked they could be remotely
related, I thought I would try changing the client browser (IE7) to only use
HTTP 1.0 for all connections. The pauses no longer occurred. As soon as I
re-enabled HTTP 1.1 on the browser, I could reproduce the pauses.

I also found the following url which may be related -
http://squidproxy.wordexpress.com/2008/04/29/chunked-decoding/. Although I
was not seeing a blank page, just two long pauses during which any input to
the browser was ignored. I implemented one of the suggestions by Amos -
using request_header_access and found it to work.

Has anyone else observed this or come across it before?

Is this something related to some aspect of HTTP 1.1 support in squid or
perhaps a quirk of IIS/Exchange?

I am happy to conduct additional testing if required and can provide offline
some packet traces.

Thanks in advance.

Regards

Paul Freeman
EML AIR Pty Ltd
Australia

__________ Information from ESET Smart Security, version of virus signature
database 5394 (20100824) __________

The message was checked by ESET Smart Security.

http://www.eset.com
 
Received on Wed Aug 25 2010 - 09:03:50 MDT