Re: Introduction / accelerator feature ideas

Henrik Nordstrom wrote:
> Hi, and welcome to Squid-dev.

Thanks, I'll try not to do too much faq asking:) but why is reply-to set
to the poster rather than the list:)

> Not entirely sure about your twist however. It smells more of a
> bandaid fix in the reverse proxy to work around a inheriently broken
> application.

Well, sort of, the application does plenty of locking at the database
level and that is what keeps it from corrupting data when users make
parallel requests, but that's just not any fun, because then you have to
go and roll back the transaction or simply sit and be deadlocked until
the database shoots the connection in the head.

The base problem is that putting a big lock on everything the
application is going to need during a request is both expensive (in
terms of ipc and bogged down apache processes) and unnecessary except
under very, very high loads when requests are slow and users impatient.

So If I do implement locking in the application to prevent races and
deadlocks (no matter how I do it) I'm going to waste an apache process
waiting for another apache to release the lock, I'd much rather have the
second request from the user stalled in squid where it doesn't cost me
anything.

One could also argue that allowing a user to have only one running
request is a generally useful feature to stop one client from
monopolizing the webservers time...

> For this to work it must be very carefully specified how
> to identify that the request is exacly the same and should be allowed
> to take over the pending result for a previously aborted identical
> requests.. and when to keep waiting for responses to aborted requests
> in hope that the user simply retried the same request..

Hmm, well, in my case the response is never cachable and the locking
would be by the users session id (provided by a cookie) so there isn't
much ambiguity.

> The exact same problem will be seen if the application is published
> directly on the Internet with no reverse proxy infront of it even for
> light loads which the server can perfectly well handle on it's own.

Correct, I'm not saying that it's squids fault that I have
raceconditions in my application, it does however provide a nice place
to put the fix:)

>>B) When a client hits a webserver it's more or less random what
>>webserver he hits, now my application does a lot of caching so the
>>first time a client hits another apache process it's a much harder
>>hit than if the client had hit a resently used one.
>
> Will however increase the demand on your backend servers as more
> connections will be needed between Squid and the web server..

Hmm, I wasn't talking about having one apache pr client, or even tying
each client to one server for all eternity, but rather perfering a
server that had been used resently by the same client, to a cold one.

> Well, this actually already have a partial solution. See squid.conf.
> (hint: half_closed_clients)
> Also related to 'A' I think.

Yes, it's related, actually it's the only way I can see for A to happen
without the users madly reloading, but I wouldn't put it past them:)

> Objects which are not cacheable should not be written to disk. What
> makes you think they are? I have seen no evidence that they are in
> all my hacking on Squid..

Nada, just speaking to keep the flies away at this point...

-- 
  Regards Flemming Frandsen - http://dion.swamp.dk
  PartyTicket.Net co founder & Yet Another Perl Hacker