I've set "forwarded_for off" and now msn.com search results load.
We also had the same problem with their site http://maps.live.com. It
also fixed that one as well.
The other suggestion of adding an acl for "header_access
Accept-Encoding deny" for msn did not work for us.
Thank you Andrew for providing deeper debugging and the solution.
M$ must be up to something...
-Bill
On Fri, Mar 27, 2009 at 9:34 AM, Andrew Farr <a.farr_at_ntlworld.com> wrote:
> wrh_at_wootsie.com wrote:
>> Hello,
>>
>> This just started happening -
>>
>> We are getting blank pages returned from our Squid 2.6.STABLE4 proxy
>> from the search results pages off www.msn.com. And MSN's
>> maps&directions site comes up blank as well. The www.msn.com home
>> page is fine.
>>
>> Happening with IE7 and FF3.
>>
>> No explanation for it on the squid server, nothing in the logs.
>>
>> If I don't use squid, the pages appear.
>>
>> Our squid has been unchanged for a long while.
>>
>> Is anyone else experiencing this issue?
>>
>>
>>
>> Thanks,
>>
>> -Bill
>>
> I may be wrong, but it looks like MSN explicitly refuses proxied
> connections. If I hide the proxy with "forwarded_for off" it suddenly works!
>
> The following shows radically different headers depending on whether the
> X-Forwarded-For header is present or not.
>
> Trying the URL
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> using wget -S
>
> First with standard proxy settings
> ==================================
> --2009-03-27 14:10:40--
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> Resolving localhost... 127.0.0.1
> Connecting to localhost|127.0.0.1|:8080... connected.
> Proxy request sent, awaiting response...
> HTTP/1.1 200 OK
> X-Powered-By: ASP.NET
> Content-Length: 0
> Expires: Fri, 27 Mar 2009 14:10:40 GMT
> Cache-Control: max-age=0, no-cache, no-store
> Pragma: no-cache
> Date: Fri, 27 Mar 2009 14:10:40 GMT
> X-Cache: MISS from cpc3-staf1-0-0-cust235.sol2.cable.ntl.com
> Via: 1.1 cpc3-staf1-0-0-cust235.sol2.cable.ntl.com:8080
> (squid/2.7.STABLE6)
> Connection: close
> Length: 0
> Saving to: `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11'
>
> [
> <=>
>
> ] 0 --.-K/s in 0s
>
> 2009-03-27 14:10:40 (0.00 B/s) -
> `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11' saved [0/0]
>
> Without the proxy
> =================
> --2009-03-27 14:10:56--
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> Resolving search.msn.com... 84.53.178.209, 84.53.178.218
> Connecting to search.msn.com|84.53.178.209|:80... connected.
> HTTP request sent, awaiting response...
> HTTP/1.0 200 OK
> Content-Length: 31048
> Content-Type: text/html; charset=utf-8
> X-Powered-By: ASP.NET
> P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND",
> policyref="http://privacy.msn.com/w3c/p3p.xml"
> Date: Fri, 27 Mar 2009 14:10:56 GMT
> Connection: keep-alive
> Set-Cookie: AFORM=MSNH11; expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
> Set-Cookie: mkt1=norm=-GB; domain=.msn.com; path=/
> Set-Cookie: mkt2=ui=en-GB; domain=search.msn.com; path=/
> Set-Cookie: MUID=95278E835C21487EB0C3B5390C5735BE; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
> Set-Cookie: OrigMUID=95278E835C21487EB0C3B5390C5735BE; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
> Set-Cookie:
> OVR=flt=0&PerfTracking=0&DomainVertical=0&Cashback=0&MSCorp=kiev;
> domain=.msn.com; path=/
> Set-Cookie: RMS=SERP=320&T=48; path=/
> Set-Cookie: SRCHD=D=650290&AF=MSNH11; expires=Mon, 20-Jul-2015
> 23:59:59 GMT; domain=.msn.com; path=/
> Set-Cookie:
> SRCHSESS=GUID=659B63EAB2A64E9597884654B10FF49E&TS=1238163056;
> expires=Fri, 27-Mar-2009 14:30:56 GMT; path=/
> Set-Cookie: SRCHUID=V=2&GUID=83EF0003D92A4C429D5D6DBCA44B86C8;
> expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
> Set-Cookie: SRCHUSR=AUTOREDIR=0&GEOVAR=&DOB=20090327; expires=Sun,
> 27-Mar-2011 14:10:56 GMT; path=/
> Cache-Control: private
> Length: 31048 (30K) [text/html]
> Saving to: `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.1'
>
> 100%[==========================================================================================>]
> 31,048 199K/s in 0.2s
>
> 2009-03-27 14:10:57 (199 KB/s) -
> `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.1' saved [31048/31048]
>
> Now set "forwarded_for off" in squid.conf
> =========================================
> --2009-03-27 14:27:51--
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> Resolving localhost... 127.0.0.1
> Connecting to localhost|127.0.0.1|:8080... connected.
> Proxy request sent, awaiting response...
> HTTP/1.1 200 OK
> Content-Length: 31048
> Content-Type: text/html; charset=utf-8
> X-Powered-By: ASP.NET
> P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND",
> policyref="http://privacy.msn.com/w3c/p3p.xml"
> Date: Fri, 27 Mar 2009 14:27:51 GMT
> Set-Cookie: AFORM=MSNH11; expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
> Set-Cookie: mkt1=norm=-GB; domain=.msn.com; path=/
> Set-Cookie: mkt2=ui=en-GB; domain=search.msn.com; path=/
> Set-Cookie: MUID=71DF4074D8BE456381CDA9945EE5AE96; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
> Set-Cookie: OrigMUID=71DF4074D8BE456381CDA9945EE5AE96; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
> Set-Cookie:
> OVR=flt=0&PerfTracking=0&DomainVertical=0&Cashback=0&MSCorp=kiev;
> domain=.msn.com; path=/
> Set-Cookie: RMS=SERP=320&T=48; path=/
> Set-Cookie: SRCHD=D=650307&AF=MSNH11; expires=Mon, 20-Jul-2015
> 23:59:59 GMT; domain=.msn.com; path=/
> Set-Cookie:
> SRCHSESS=GUID=E8E98A7C50064824A06EC246214D405D&TS=1238164071;
> expires=Fri, 27-Mar-2009 14:47:51 GMT; path=/
> Set-Cookie: SRCHUID=V=2&GUID=BE07F5751A95407FB6E94683FB41EDBB;
> expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
> Set-Cookie: SRCHUSR=AUTOREDIR=0&GEOVAR=&DOB=20090327; expires=Sun,
> 27-Mar-2011 14:27:51 GMT; path=/
> Cache-Control: private
> X-Cache: MISS from cpc3-staf1-0-0-cust235.sol2.cable.ntl.com
> Via: 1.1 cpc3-staf1-0-0-cust235.sol2.cable.ntl.com:8080
> (squid/2.7.STABLE6)
> Connection: close
> Length: 31048 (30K) [text/html]
> Saving to: `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.2'
>
> 100%[==========================================================================================>]
> 31,048 --.-K/s in 0.1s
>
> 2009-03-27 14:27:51 (214 KB/s) -
> `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.2' saved [31048/31048]
>
>
>
Received on Fri Mar 27 2009 - 15:49:00 MDT
This archive was generated by hypermail 2.2.0 : Fri Mar 27 2009 - 12:00:02 MDT