Re: [squid-users] Expect Request headers

On 21/01/11 02:48, Rolf Loudon wrote:
> Hello
>
> I have some software that is producing repeated TCP_MISS/417 errors in the logs and is failing to connect to the server
>
> I have found previous discussions that show the software is not behaving correctly to the extent that after the proxy sends back the 417 - expectation failed, the client should send the request without the header. This is not happening. A sniff of the traffic shows that after the 417 is sent back the client simply repeats the request unchanged. This repeats.
>
> All of this is using squid 2.7STABLE3 on debian 5 (lenny).
>
> As an aside, nothing on these proxies has changed so I am at a loss in that respect.
>
> These proxies have a cache-peer configured. When I point the client software directly at these cache-peer hosts (which I do not control) the connection works. These proxies are squid 2.6STABLE22. With these upstream ones there may be other differences (such as OS etc) but I have no visibility to them to check).
>
> (obfuscated) Extract of sniff while connecting as best I can do it is as follows.
>
> The fail to connect via my proxy (squid 2.7STABLE3):
>

This is what is happening...

> Client request:
>
> Hypertext Transfer Protocol
> POST http://www.somehost.com/api/SomeService.svc HTTP/1.1\r\n
> Request Method: POST
> Request URI: http://www.somehost.com/api/SomeService.svc
> Request Version: HTTP/1.1
> Content-Type: text/xml; charset=utf-8\r\n
> SOAPAction: "http://tempuri.org/SomeService/GetServerSettings"\r\n
> Host: www.somehost.com\r\n
> Content-Length: 233
> Expect: 100-continue\r\n
> Proxy-Connection: Keep-Alive\r\n
>

Notice how there is zero bytes of body here. Despite the headers saying
233 would follow.

What this client is *Expecting* back is a 100 continue response
indicating "Send me the body". This is one of the more advanced HTTP/1.1
efficiency features.

Squid-2 is of course HTTP/1.0 software and can't do that reply
correctly. Cannot even transmit them properly.

2.6 sends HTTP/1.0 to the server which all seem smart enough to identify
the problems and not screw the transaction by sending 100 replies back.

2.7 is *just* smart enough with its HTTP/1.1 support to tell the client
about this problem via the 417 and swallow silently any 100 messages
received.

> Proxy response:
>
> Hypertext Transfer Protocol
> HTTP/1.0 417 Expectation failed\r\n
> Request Version: HTTP/1.0
> Response Code: 417
> Server: squid/2.7.STABLE3\r\n
> Date: Thu, 20 Jan 2011 13:27:49 GMT\r\n
> Content-Type: text/html\r\n
> Content-Length: 1476
> Expires: Thu, 20 Jan 2011 13:27:49 GMT\r\n
> X-Squid-Error: ERR_INVALID_REQ 0\r\n
> X-Cache: MISS from my.squid.proxy\r\n
> X-Cache-Lookup: NONE from my.squid.proxy:8080\r\n
> Via: 1.0 my.squid.proxy:8080 (squid/2.7.STABLE3)\r\n
> Connection: close\r\n
> \r\n
>
> And this simply repeats - the client sends the same request again.
>

Client software that follows a 417 message with an repeat using Expect:
header is broken. Please mention this to the relevant people. Point them
at RFC 2616.

>
> Using the upstream proxy (squid 2.6 STABLE22) - it works
>
<snip>
>
> Any suggestions on why the difference but more importantly how to fix would be enormously appreciated.
>
> Many thanks

2.7 has more HTTP/1.1 support. To the point of advertising to the server
that it can receive HTTP/1.1 replies.

In Squid-2.7 you can suppress the 417 message generation and cause one
of the following to happen by setting "ignore_expect_100 on" in squid.conf.

  (1) server responds earlier than it should, causing the transfer to be
truncated mid-transfer if the POST was long. Corrupted data etc.

  (2) Squid and Server both waits for the mentioned 233 bytes of body.
and waits, and waits.
   (2a) Eventually the client times out and sends the body, allowing the
server to create the right response.
   (2b) Eventually the client times out and displays an error page.

These *will* cause slower transfers for *all* clients attempting this
feature, not just the broken ones.

HTTP/1.1 *requires* all software sending "Expect:" to retry without the
Expect: header on receiving a 417 response.

For the record:
  2.6 and 3.0 are paired in their non-support.
  2.7 and 3.1 are paired in their partial support with the same override
directive.
  3.2 supports the expect-100 feature and has no known problems with this.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.10
   Beta testers wanted for 3.2.0.4