Re: Squid-smp : Please discuss

tis 2009-09-15 klockan 05:27 +0200 skrev Kinkie:

> I'm going to kick-start a new round then. If the approach has already
> been discussed, please forgive me and ignore this post.
> The idea is.. but what if we tried using a shared-nothing approach?

Yes, that was my preference in the previous round as well, and then move
from there to add back shared aspects. Using one process per CPU core,
non-blocking within that process and maybe internal offloading to
threads in things like eCAP.

Having requests bounce between threads is generally a bad idea from a
performance perspective, and should only be used when there is obvious
offload benefits where the operation to be performed is considerably
heavier than the transition between threads. Most operations are not..

Within the process I have been toying with the idea of using a message
based design rather than async calls with callbacks to further break up
and isolate components, especially in areas where adding back sharedness
is desired. But that's a side track, and same goals can be accomplished
with asynccall interfaces.

> Quick run-down: there is farm of processes, each with its own
> cache_mem and cache_dir(s). When a process receives a request, it
> parses it, hashes it somehow (CARP or a variation thereof) and defines
> if should handle it or if some other process should handle it. If it's
> some other process, it uses a Unix socket and some simple
> serialization protocol to pass around the parsed request and the file
> descriptor, so that the receiving process can pick up and continue
> servicing the request.

Just doing an internal CARP type forwarding is probably preferable, even
if it adds another internal hop. Things like SSL complicates fully
moving the request.

> There are some hairy bits (management, accounting, reconfiguration..)
> and some less hairy bits (hashing algorithm to use, whether there is a
> "master" process and a workers farm, or whether workers compete on
> accept()ing), but on a first sight it would seem a simpler approach
> than the extensive threading and locking we're talking about, AND it's
> completely orthogonal to it (so it could be viewed as a medium-term
> solution while AsyncCalls-ification remains active as a long-term
> refactoring activity, which will eventually lead to a true MT-squid)

I do not think we will see Squid ever become a true MT-squid without a
complete ground-up rewrite. Moving from single-thread all shared single
access data without locking to multithread is a very complex path.

Regards
Henrik
Received on Tue Sep 15 2009 - 19:20:36 MDT