Re: [squid-users] An option to force keepalives for POST during forward proxy

It's not so much that it isn't well-defined, it's that it's dangerous.

Since a server can close an idle connection at any time, there's a
chance that as you start your POST, the server will close the
connection, leaving things in an indeterminate state. Unlike GET, POST
can't be automatically retried if it encounters this kind of failure.

Note that this risk is present even with a new connection -- i.e., a
server can close a connection you've just opened to it before you send
the first request. However, the chances of this happening are a lot
less than when the conn has been idle for a while.

All of that said, it would be nice to do something about this. There
are a few possible short-term solutions;

  - A hop-by-hop protocol extension that allows a server to advertise
how long it will keep an idle connection open, so that the client can
judge whether it's safe to reuse it for a POST. While there are still
some circumstances where this could fail (network segment, server
running out of resources and ditching existing conns, etc.), it would
make it somewhat safer.

  - Having clients make the POST request with Expect: 100-continue to
make sure that the connection is still usable. However, Squid
currently doesn't support Expect/Continue (at least well).

See <http://www.squid-cache.org/bugs/show_bug.cgi?id=1979> for more
discussion along these lines.

The long-term solution is to layer HTTP over another protocol, like
SCTP. This is already in discussion in the IETF and there are a few
papers and test implementations available. I'd love to see Squid do
this, but it's probably going to be a while.

Cheers,

On 20/03/2009, at 5:49 PM, Michael Spiegle wrote:

> I have a situation where being able to use keepalives for POST
> methods would be very handy. Essentially, I have a series of .NET
> webservers that POST data to a linux-based webservice. The .NET
> webservers and the the linux servers are geographically distributed,
> so we have a 65ms latency between them. The average POST body size
> is 40-50KB.
>
> Our goal is to significantly reduce the time it takes to make these
> POSTs. We have tried using TCP Window scaling and keepalives on
> the .NET servers, however the .NET framework appears to have many
> bugs in this regard and is not reliable. We have decided that our
> best course of action is to have the .NET servers talk to the
> webservice through a small cluster of squid proxies. This way, we
> can use Linux's reliable TCP Window scaling. Unfortunately, squid
> doesn't keep persistent connections on the backend for POSTs.
>
> I read a message from the list a while back that this was done on
> purpose because using POSTs with keepalives is not well defined. I
> believe there should be an option to allow this sort of behavior and
> would like to investigate implementing it myself if there are no
> other plans to do it. My webservice is running behind Apache, and
> it seems to properly handle POSTs with keepalive.
>
> Can anyone point me in a general direction to get started on this?
> I am not familiar with the squid codebase. Also, does anyone know
> how much work it might be to implement this?
>
> Thanks,
> Mike

--
Mark Nottingham       mnot_at_yahoo-inc.com