Re: Release Engineering initiatives for the XEmacs project

mb> 8. Abolish the No-Commit period during releases. The release mb> engineer's computers should, in principle, be able to spend mb> days testing the release before making it official, and other mb> developers should not have to delay their work during this mb> process. ST> What makes you think this is possible? Are there other projects that ST> have done this? What's the point of having weekly (or biweekly) beta ST> "releases" if by the time the tarballs actually get uploaded the ST> active developers all have trees that vary significantly from what the ST> release engineer (RE) is using? How do you handle patches the RE must ST> make to get the thing to build (and pass make check)---and what if it ST> never does? I think the trend in free software projects is for the developers to be using CVS, while folks outside the project who are early adopters will get beta releases. So, for example, I am not a gcc developer, but I occasionally get gcc snapshots so that I can check whether our software is compatible with their software. I don't want the instability of their CVS tree. If the release engineer can't make a release because it's broken, fixing that becomes the number one priority for the project. Whatever change caused the problem needs to be backed out, if it cannot be easily fixed. But the fix is made on the main line of development, not another branch. The beta release is delayed until a non-broken state is re-established in CVS. ST> I don't see any particular point in having a "release" that doesn't ST> mean coordinating the various efforts to make sure that the code ST> builds on almost all platforms and passes "make check" on most, with a ST> list of known problem platforms, otherwise you're shipping alpha code ST> to the beta testers. But this really implies either a freeze on ST> non-release-fix commits or a new branch for the RE's fixes (and ST> concommitent merge headaches) for each "release", doesn't it? No. You can do cvs update; build; regression test for a while; tag the workspace with a release tag; build and release tarballs; independently of whether development has continued after the cvs update. The key is that the `tag' command tags the current state of the *workspace*, which is tested, not the CVS HEAD. The only tricky thing is adding those ``XEmacs 21.2.1234 is released'' ChangeLog entries everybody loves, since those need to be added at release time, not `cvs update' time. I like the concept of `tinderbox', that continually builds on multiple platforms to make sure that the tree maintains sanity, and to help identify when a patch introduces a compile problem or test regression. ST> I'm also having trouble with the semantics of "developers delay work". ST> Do you mean "developers", or "reviewers"? The former won't notice a ST> few hours or even a couple of days; it's the latter who can't do ST> anything during a freeze. But I thought the point of having frequent ST> "releases" was at least in part to _intentionally_ slow the commit ST> process down every so often so that the reviewers can agree on ST> something something that at least builds and passes tests on some ST> platforms, then start the cycle again. Slowing down the cycle while there is no feature or code freeze is evil because it makes being a contributor more work. Not only does the developer have to keep track of current freeze/thaw status, which may change hourly, but the release engineer has to make an extra effort to communicate that status (which might change if there is a build problem, for example). I want to eliminate *unnecessary* communication. We already have too much mail on this mailing list. I would think key active developers would use CVS to stay at the bleeding edge. Kyle doesn't use CVS, but the kind of work he does doesn't require that. Similarly, I don't use CVS for packages (yet), since I am mostly a simple `user' of packages, not a developer. The release engineer can push the button for making a release when a particularly important bug fix has been committed to CVS. mb> 10. Add a special tag like `xemacs-21-2-latest-beta' for the mb> latest released beta. This is always the latest level of CVS mb> that is *guaranteed* to run `make check' on at least the mb> release engineer's machine. ST> What's special about the RE's machine? What does it mean to have a ST> "release" that only passes tests on one machine? And if it doesn't, ST> then what? Ideally, the release process would use 10 machines on the Net for doing pre-beta-release testing, mostly guaranteeing that no one will ever get a compile error in a beta release. I don't know if we will find the Net resources to implement that. ST> You can't guarantee success, even on the RE's box, at regular ST> intervals; you just have to take it when it comes. I don't understand. You can't guarantee that the state of the code in CVS is non-broken, but you just don't make a beta release when that happens. Martin