Re: Future squid development

On Wed, Oct 04, 2000, Andres Kroonmaa wrote:
> On 3 Oct 2000, at 22:00, Henrik Nordstrom <hno@hem.passagen.se> wrote:
>
> > > Would someone describe what "HEAD" exactly means? squid-2.4-devel?
> >
> > HEAD is a CVS term for the HEAD revision which is the newest bleading
> > edge version.
>
> you mean, including unfinished total rewrite also, for eg?
>
> > Then from the HEAD version you have branches when a release stabilizes
> > and gets a live of it's own, such as the STABLE releases for Squid.
>
> So, until HEAD-based new Stable is released, HEAD is pretty much
> unstable and unusable, right?
> Make it clear, how stable is it compared to 2.3Stable4?

Thats like asking how long pieces of coloured string are. Just because
they're coloured differently doesn't have any bearing on how long they'll
comparitvely be. :-)

> > HEAD is always the current development version of the Squid which is
> > meant to be the next Squid version someday.
>
> ie, currently 2.4devel? or is it 2.3stable-latest?

Ok. I think you need a picture here.

HEAD
|
2.4DEVEL4
|
2.4DEVEL3
|
2.4DEVEL2
|
2.3STABLE -- 2.3STABLE1 -- 2.3STABLE3 -- ..
|
2.2STABLE -- 2.2STABLE1 -- 2.2STABLE2 -- ..
|
2.2PRE2
|
2.2PRE1
|
2.1STABLE -- 2.1STABLE1 -- 2.1STABLE2 -- ..
|

Now, this isn't the *exact* branch tree setup, but its usefully close to
the truth for the purposes of demonstration.

HEAD is always the main branch. It is the development branch. Whenever
a devel release is rolled, it is simply another tag (it could be a branch
but I don't see the point of committing *to* a DEVEL "branch" since that
should be HEAD anyway, right? :-). The DEVEL tag acts as a "marker" or
"snapshot" of the current HEAD to say "This here is a good place to call
DEVEL". Note that DEVEL releases never have any guarantees of usefulness.

Then you hit the STABLE branches. Whenever a STABLE release is rolled,
note how it is a *BRANCH* not a tag. This is so development on HEAD can
go forward and not interfere with the stability of the STABLE branch.

When people commit to HEAD, it does not show up in the STABLE branches.
This is a normal branch-based development problem, and it forces the
coders to commit their changes to more than one branch if they want it to
show up. FreeBSD has the concept of an 'MFC' which is simply the act of
taking changes made on HEAD (freebsd-current) and applying it to one of
the release branches to "backport" changes into the release trees.

(the PRE's are just pre-release snapshot tags, again useful for things like
bugreports and so on. when you lay down a PRE tag, you generally ask the
developers to NOT commit any new and funky change, and only do bugfixes
until the STABLE branch is made. This is what state we are in now.)

Right now, HEAD corresponds to 2.4-DEVEL. There are no stability guarantees,
just as there are no stability guarantees when a DEVEL snapshot is rolled
(except this is much, much more alpha. :-)

Does this clear it up a little bit?

> > The goal is to allow expermental features to rapidly develop without
> > having a negative impact on the overall stability of the Squid release,
> > and to allow rapid inclusion of these into the distribution once tested
> > and proved useful.
>
> Whats been frustating me abit is that it is quite hard to find out what
> is what. What the HEAD is based on, how stable it is, etc. If I understand
> it right, then HEAD is currently 2.4, alpha, and noone even really knows
> how stable it is on production box, because noone really uses it.

Well, DEVEL has always to me been aimed towards developers and not production
machines.

> Usually, when people feel that it is time to make a new stable release,
> then spike of activity starts during which some good coders take a very
> close look at the code and bring it up to stable. Then its released and
> during some few months several new releases are made with often critical
> fixes.
> This is very discouraging to use HEAD as your development platform.
> I have to date sticked with latest Stable and hacked on it. At least
> I have a solid ground and blame myself first when I screw it.
> And I have a good reference point, snapshots or devel-HEAD are moving
> targets. If you are not too much CVS-guy, then it isn't attractive.

Thats fine. Noone is going to stop you DEVELOPING on a STABLE branch. Its
just when you want a feature committed, you're going to have to make it
work on the DEVEL branch. At least once you've done that, we can commit your
change to BOTH the latest STABLE branch and HEAD. :-)

When you get used to squid internals however, you'll probably fall towards
HEAD in any case.

> > > In general, I think we should have a version that fits between Stable
> > > and Beta of next version, and be usable on production caches. Such
> > > version should be based strictly on currently stable version, and only
> > > differ from it by new committed features added to it.
> >
> > This we already have in the nightly snapshots.
>
> hmm. isnt these daily snapshots just bugfixes to stable? ie. latest
> current snapshot is early 2.3Stable5 ? And btw, why daily, why not
> "only if changed"?
> And do you mean that Cygwin and NTLM branches would appear here after
> merging?

Yup. The nightly snapshots can be thought of as squid-2.4DEVEL-NOW.tar.gz,
which funnily enough corresponds to squid-HEAD.

> > The question is if we
> > should branch off a Squid release from HEAD prior to releasing the first
> > STABLE version, or if new features should be deferred while the next
> > release matures to STABLE.
>
> What I'm aiming at is that new HEAD should be branched from latest STABLE
> immediately after first STABLE release, next release ver (DEVEL) be based
> on that HEAD, and all branches based on that HEAD. All critical changes
> should be hacked on DEVEL and left out of HEAD. When a new feature (branch)
> is reaching stable state, it is committed to HEAD (and thus automatically
> to DEVEL).
> When DEVEL reaches milestone of releasing new Stable, then all features
> of HEAD are reevaluated, cleaned up, integrated with changes in DEVEL,
> and new STABLE released. After that old HEAD is retired and no new updates
> to it. Old STABLE continues to have updates by bugfixes for some time.
> New HEAD is spawned from new STABLE immediately and new circle begins.
> If this is currently the case, then I only wish to see more clear
> description of that on the web.

You can't branch HEAD from anything. And, the above makes things a little
roundabout. Think of the tree I gave above:

* all hacking happens on HEAD
* all bugfixes happen on HEAD and relevant STABLE branches
* When a useful change is made on HEAD and is stable enough, the change is
  made to work in one or more relevant STABLE branches, and committed
* Throughout the lifetime of a STABLE branch, you roll new STABLE releases
  from it, as more bugfixes and changes are brought forward into it from
  HEAD in the above process.

> For me, to date 2.3.STABLE-hno has been closest to what I want, ie.
> latest Stable + more). What I'd like to see is that there is 2.3HEAD4,
> that is clearly known to be based on 2.3STABLE4, and that this HEAD
> could have branches committed to it when ready instead of waiting for
> next (2.4) release. And this be publicly available and encouraged to
> be used by brave people. Someone could for eg hack together a cool
> new feature just for the heck of it, and we currently have no public
> place to put it for use - STABLE tree is sacred, 2.4DEVEL/HEAD is
> unstable..

There has been a problem with people (notably hno) keeping big patchsets
in his own tree and rolling seperate releases. I'd like to see this go away,
since it confuses the hell out of people and when you want to submit
patches, it makes things difficult to line up with the 'official tree'.

What I personally would like to see there is all of hno's patches brought
into HEAD, and once they're working there they are backported (well, since
they're already AGAINST STABLE branches, he can just apply them :-) push
them out onto the STABLE branches. This will tidy up quite a lot of things.
This should only happen if Duane and myself don't disagree with a patch,
of course.

> If latest Stable-HEAD isn't stable, people could revert back to
> previous or STABLE, or take that HEAD and get it fixed.

There's no concept of STABLE-HEAD.

> > > So, we should have 2 heads: Stable-head and Devel-head, I think.
> >
> > HEAD is what will become the next Squid version.
>
> well, maybe my naming isn't correct, I don't know. Maybe multiple HEADs
> don't even make sense from CVS perspective. Maybe we should call it
> STABLE+NEWFEARURES (eg. 2.3STABLE4-NF, or smth)
>
> My point was that there are different kinds of new features, radical
> and not so radical. For eg, modio cannot be merged with latest stable,
> as it needs some redesign in squid, but commloops could be. Yet I
> cannot really participate in commloops easily because its based on
> something I understand is unstable. I have no time nor resources to
> test it on separate box. But if I knew its based on stable release,
> I'd run it on my production boxes and would iron out any bugs I
> discover. Adrian is also in some trouble developing it, because he
> has very little feedback, and it won't appear before next release(2.4)
> is out. This also doesn't encourage to wrap it up yet, isn't it?

This is just a general problem we have right now due to a lack of developers
(which is slowly being fixed again, yayayay.) The trouble is that I would
*NEVER* suggest right now commloops is ready for the production systems -
even though I have great faith in my code, regardless of whether it was
based upon HEAD or STABLE is irrelevant. Its an architectural change. :-)

So yes, to test things like this you really do need a seperate box to run
the changes on. I know its a pain, but its been proven in the past by
various CVS-based projects that developing on STABLE and then backporting
to DEVEL/HEAD can be neat in the short term, but really hard to maintain
from a code development perspective in the long term.

> Of course, I'm not insisting on anything, I just feel that some
> features should be added to public for use before next major release
> and after feature-freeze, besides to stable tree.

Yup. This is why we should keep HEAD at least slighly stable. This is
what squid.sourceforge.net is for. :-)

Adrian

-- 
Adrian Chadd			"If a butterfly flaps its wings in China,
<adrian@creative.net.au>	    will a women get naked in Amsterdam?"
				      -- Ashley Penney on Chaos Theory