Re: [RFC] Squid process model and -z

On 11/01/2013 10:09 AM, Alex Rousskov wrote:
> I would also like to propose an alternative:
>
> Make "squid -z" a no-op (Squid would just exit immediately after
> detecting that option). Teach Squid to always prime cache_dirs at
> startup. This will remove all problems discussed above. Moreover, it
> will make admins live easier because they do not need to think about
> writing correct scripts to detect cache_dir validity. It was probably a
> mistake to add "squid -z" in the first place. We can correct it now.

The "get rid of squid -z" idea is not new, of course. I now reviewed
several related threads on squid-users. I found only one class of use
cases where admins believe they need squid-z:

* Squid should quit if there is no cache_dir instead of creating and
using that cache_dir.

All the use cases in this class revolve around something that must
happen before Squid starts so that Squid disk caching works correctly
instead of destroying the server. The simplest example is when some
external force must mount the cache directory. If that external force
fails to do so, Squid must quit instead of, say, overflowing the root
partition. A more complex example is a broken filesystem that should be
repaired before Squid corrupts it further, making repairs more difficult
or impossible.

In all these cases, the admins rely on Squid quitting if cache_dir is
missing. Since Squid does not really verify that subdirs are present,
not starting Squid if the top-level directory is missing ought to be
equivalent to the current Squid behavior. Thus, is we make squid-z a
no-op, the admins that do not want Squid to start when a cache_dir is
missing, can implement that directory check in their Squid startup script.

There were also suggestions to add a "create cache_dirs if missing"
option to squid.conf. I think that (together with making squid-z a
no-op) would be OK and would help in use cases described above. However,
the option should be ON by default for new Squid installations to work
seamlessly.

To summarize, the proposal is

1) Teach Squid to create missing disk cache structures at runtime.
2) Make squid-z a no-op and deprecate that option.
3) Add a squid.conf option to disable (1).

Squid would still exit if the top level cache_dir is missing and cannot
(or failed to) be created. No change there.

I think the above is the "Right Design" because it is simple, robust,
and prevents exposure of internal store details (which naturally creates
more problems than it solves). Making squid-z work correctly _and_
remain compatible with most startup scripts out there will result in the
opposite design (complex, fragile, and exposing).

If we make squid-z a no-op, startup scripts that do not verify that
squid-z worked will continue to work fine. However, there will be some
upgrade pains for admins that use more complex startup scripts today. On
the other hand, many of the admins are already experiencing upgrade
pains because of how squid-z interacts with SMP now. It is difficult for
me to say what kind of pain is preferable, but I think we should
seriously [re]consider making squid-z a no-op as sketched above.

Thank you,

Alex.

> The performance cost for always checking cache_dirs at startup is
> negligible for Rock.
>
> If that cost is too much for ufs-based dirs, we can alter their
> algorithm to do nothing if top-level directories exist. This will be
> very similar to what the vast majority of startup scripts are doing now
> (because those script do not verify that _all_ UFS cache_dir subdirs are
> present when deciding whether to run -z). Alternatively, we can teach
> UFS-based dirs to create cache_dir sub-directories runtime if they are
> needed but are missing (triggered by UFS file creation errors). Both
> approaches would reduce performance overhead to negligible levels.
>
> If Squid needs additional information to create missing cache_dirs
> correctly, we can add more options to squid.conf to supply that information.
Received on Wed Nov 06 2013 - 09:03:46 MST