Re: [PATCH] Bug 3150: do not start useless unlinkd.

On Sun, 09 Oct 2011 12:29:03 -0600, Alex Rousskov <rousskov_at_measurement-factory.com> wrote:
> On 10/08/2011 01:25 PM, Henrik Nordström wrote:
> > fre 2011-10-07 klockan 11:50 +0400 skrev Dmitry Kurochkin:
> >
> >> I did not know that -N was used for anything but testing and debugging.
> >> Perhaps we should also use diskers in -N mode?
> >
> > -N is not only a debugging flag. It's used in production in many setups
> > where a usable system monitor keeps track of the process. It's "Do not
> > daemonize", functionality should otherwise be the same.
> >
> >> (Though, neither Mac OS X nor Debian (by default) use upstart. Ubuntu
> >> does.)
> >
> > And fedora uses systemd.
> >
> >> We should add mayUseUnlinkd() method to DiskIOStrategy. But only UFS
> >> SwapDir would relay answer from it. Other store modules do not need
> >> unlinkd even if the underlying IO strategy may use it. Do you agree?
> >
> > Or perhaps simply start it on first use, similar to how we runtime add
> > more helpers of other types when needed?
>
> I like the direction of that approach in general.
>
> If we fork() to start unlinkd, then the fork() becomes the more
> expensive the longer we delay it because Squid allocates more and more
> memory, right? If so, waiting until the first unlink seems wrong.
> However, if we already fork other helpers that way, perhaps that is OK.
>
> [Forking during reconfigure will be the most expensive, but it is a rare
> exception rather than a rule so we can ignore it.]
>
> If late forking is a problem, perhaps folks suffering from it should
> invest in a simple protocol that would allow Controller or even the
> Master process to fork new kids on demand?
>
>
> For now, we have a choice of starting a lot of useless unlinkd processes
> (without Dmitry's patch) OR starting only what we need and not
> supporting switching to unlinkd during reconfigure (with Dmitry's
> patch). I think the latter is better. What do you think?
>

The latest patch supports switching to unlinkd during reconfigure.

I see two (primary) reasons to fork helpers at runtime:

* number of helpers that Squid needs changes at runtime (e.g. depends on traffic)
* it is too hard to determine at startup if the helpers will be needed

In case of unlinkd, we always use one process per worker and it is
relatively easy to determine if unlinkd is used in a given
configuration. So it is not clear to me what benefits runtime forking
would give us here.

Regards,
  Dmitry

> Cheers,
>
> Alex.
Received on Sun Oct 09 2011 - 18:47:49 MDT