Re: [PATCH] schedule connect timeouts via eventAdd

On 01/24/2013 11:16 AM, Rainer Weikusat wrote:
>> * there are well-justified concerns about RAM or performance overhead of
>> event-based timeouts with the current event queue,
>
> The concern I originally wrote about was whether it would be better to
> let the event-based timeout expire or to cancel it instead. The 'RAM
> overhead' of using the event queue would be the memory needed for an
> ev_entry object. That's 48 bytes, compared to the size of the
> ConnOpener object figuring at 112 bytes.

For add-and-forget, the RAM overhead is not a few bytes per ConnOpener
object because those events stay in the queue a lot longer than
ConnOpener objects stay alive. With 60'000 alive events, the RAM
overhead concern is justified.

For add-and-delete, there is justified CPU overhead (and no RAM
overhead) IMO.

>> * replacing legacy event queue with a better one is a serious
>> stand-alone project, with no "obviously best" replacement available in
>> STL, and
>
> That's easily doable in an afternoon and completely straight-forward

I bet the authors of the original event queue code had similar thoughts.
And we are still fixing occasional bugs in that code or add bugs while
trying to interface with it. The theoretical concept is indeed
straightforward, but things usually get complex when it clashes with
Squid realities. And since you do not necessarily have a comprehensive
view on how the event queue is used by the rest of Squid code, I am not
sure you can select the right algorithm alone. This means more
discussions, iterations, etc.

There have been many innocent-looking projects that were later
discovered to significantly drop Squid performance. And here we are
talking about very performance sensitive code in the very middle of
Squid main loop! So please forgive my skepticism regarding your
estimates of complexity and implementation speed of the event queue
optimization project...

> And the fd-based timeout code does linear searches of
> the FD table in order to determine when a timeout occurred which is
> also not exactly a 'scalable' approach.

If you are talking about the checkTimeouts() loop, it will not become
longer or shorter after this project is completed, so it does not seem
to be relevant? That loop has its own problems, and it can be optimized
or even removed, but that would be a separate project.

Please give me a few hours to implement initial fd-based ConnOpener
changes so that we can discuss a specific alternative to event queue
optimizations.

Thank you,

Alex.
Received on Thu Jan 24 2013 - 19:11:58 MST