I've finally gotten around to writing up some packaging notes for Gnus (attached). I also included some general notes for working with Mercurial. I plan to push this to Bitbucket next Thursday; feedback is welcome. mike -*- outline -*- * Introduction This file contains notes on packaging new versions of Gnus, as well as some general information for package maintainers. * Supplement to DevGuide Section 4.3 The XEmacs Developers Guide is included in the "general-docs" package. Section 4.3 of the Developers Guide contains useful information for package maintainers, but it has some gaps, and its discussion of source code management tools is obsolete. ** General Information *** Mercurial, not CVS The Developers Guide assumes CVS. XEmacs now uses Mercurial for source code control, so skip the parts of Section 4.3.2 that discuss CVS. Instead, see http://www.xemacs.org/Develop/hgaccess.html for information on using Mercurial for XEmacs development. *** metadata The package Makefile contains a few variables that you need to manage. AUTHOR_VERSION This should reflect the upstream version that you are packaging. MAINTAINER This should contain your name and email address. REQUIRES See "package dependencies", below. You should never change VERSION, as that variable is owned by the package buildmeister. The "provides" clause in package-info.in lists the libraries that your package provides. **** package dependencies The XEmacs package framework provides structure for declaring inter-package dependencies. The REQUIRES list in the package Makefile lists the other packages that your package depends on at compile time. In addition, PUI, the package user interface, uses the REQUIRES list to determine dependencies for downloads, but that is not the primary purpose of the REQUIRES list. *** ChangeLog files Many upstream packages provide their own ChangeLog files. You may wish to rename those to ChangeLog.upstream (or some variation), so that they are separate from the XEmacs ChangeLog files. You can refer to the upstream ChangeLog file in the ChangeLog entry that you create when updating to a new upstream release. For example: YYYY-MM-DD J. Random Hacker <jrh(a)xemacs.org&gt; * Sync with foobar 2.7. Please see the ChangeLog.upstream files for details. ** Source Tree Organization *** top-level and subrepos The packages tree is stored on bitbucket.org as a top-level repository with subrepositories (subrepos). This simplifies access control, as each package maintainer can be easily given write access to just her own package(s). There are a few gotchas that package maintainers should be aware of. - Not all Mercurial commands automatically recurse into subrepos. See the section "Mercurial recipes" below for recommended commands. - The repository organization on Bitbucket is set up such that you can clone and build the entire package tree, but you cannot easily clone your clone. This is primarily because the subrepo configuration assumes that the parent has a flat layout, whereas your clone will have the individual package repositories underneath the top-level repository. (In addition, a couple of the subrepo names get tweaked during the cloning process.) This is not a problem if you are working with a single package repository. Nor is it a problem if you don't want to clone your clone. But if, for example, you want to keep a pristine master repository and a working repository, you will need to massage the master repo before cloning it to create your working repo. A Unix shell script is provided below to do this massaging for you. - If you do "hg stat -S" in the top-level repo, you may see changes listed that you have committed in the subrepo. Mercurial will stop listing them when the top-level repo is committed with the subrepo changes. Normally only the Packages buildmeister will commit changes in the top-level repo, so to avoid confusion, you may want to avoid running "hg stat -S" in the top-level repository. Most of the time, you will only need to make changes in the subrepo for your package. But if you change the source organization of your tree (e.g., to simplify merging with upstream), you will need to update package-directory-map in package-compile.el, which is in the top-level repository. Ask on xemacs-beta(a)xemacs.org how to proceed. Additional information about subrepos is available at http://mercurial.selenic.com/wiki/Subrepository *** variances between upstream and the XEmacs package **** source variances The source organization of the upstream package may not match the organization used in the XEmacs package tree. In fact, for large, complicated packages (e.g., Gnus), it's possible that the upstream package contains files that are not in the XEmacs package and vice versa. In these cases, it can be a challenge to update an XEmacs package to a new upstream version. Nonetheless, the update algorithm described below under "Mercurial recipes" should be adequate. **** content variances You can make XEmacs-specific changes easier to find by tagging them with this comment: ;; XEmacs change Of course, the more variances there are between the upstream code and the XEmacs package, the harder it will be to update to new upstream versions. So the general recommendation is to work with the upstream developers to get any such changes incorporated into the upstream code base. ** Mercurial recipes This section has sample commands for a simple package workflow. It assumes that you have already reviewed http://www.xemacs.org/Develop/hgaccess.html. *** initial clone The first thing you need to do is create a local clone of the repo on Bitbucket. **** single package If you are just cloning the "gnus" package, you can use $ hg clone ssh://hg＠bitbucket.org/xemacs/gnus We use an ssh URL here under the assumption that you will eventually want to push changes back to Bitbucket. **** the entire package tree If you are cloning the entire package tree and don't plan to create any additional clones from it, you can just do $ hg clone https://bitbucket.org/xemacs/xemacs-packages mypackages On the other hand, if you do plan to clone from your local copy, you should create one more directory layer, e.g., $ mkdir mypackages $ cd mypackages $ hg clone https://bitbucket.org/xemacs/xemacs-packages and then run the following script. In the example above, you would run it from the mypackages/xemacs-packages directory. --8<---------------cut here---------------start------------->8--- #! /bin/sh top=$(basename $(pwd)) for d in mule-packages xemacs-packages; do for f in $d/*; do [ -d $f ] && ln -s $top/$f .. done done mv ../Sun ../sun mv ../hm--html-menus ../hm-html-menus --8<---------------cut here---------------end--------------->8--- Independent of which method you just used, for the packages that you want write access to, change the "default" line in the subrepo's .hg/hgrc from the https URL to an ssh URL, e.g., default=ssh://hg＠bitbucket.org/xemacs/gnus *** updating and merging your clone As a general recommendation, you should commit your local changes before pulling from the parent repository. If there are new changes in the parent repo, having your changes committed means they will not be lost in case something goes awry during the merge process. If you need to run "hg merge", you should do an "hg commit" after you have finished merging the changes. This will make it easier to distinguish additional work from changes that resulted from the merge. **** single package If you have a single package repository, you can use "hg pull" and "hg merge" as described in http://mercurial.selenic.com/wiki/Tutorial. **** the entire package tree If you have the entire package tree, the most reliable way to get an update is to do "hg pull -u" in the top-level repository. The -u (or a separate "hg update") is necessary to pull the subrepo updates from the parent repository. If a merge is needed in your package subrepo, it will be indicated during the update step with the string "+1 heads". You can then use the normal "hg merge" and "hg commit" commands within the subrepo. *** committing and pushing changes Normally you will just push changes from your package (sub)repo, not the top-level packages repo. You can use "hg stat" to check for uncommitted changes. After your final commit, you can use "hg outgoing" to list the changeset(s) that you will be pushing. If your push fails with an error message about creating multiple heads, that usually means that changes have been made in the parent repo since the last time you pulled from it. It's usually best to pull down those changes and (re)run "hg merge". Retest as needed, review your changes, commit, and try again to push. *** updating to a new upstream version Probably the most straightforward way to handle updates is to keep a pristine copy of the most recent upstream version (i.e., the version that the XEmacs package is based on). Diff the new upstream version against the previous upstream version to generate a patch file. If necessary, massage the patch file to reflect the XEmacs file layout, or to remove diffs for files that are not in the XEmacs package. Make sure that your repository has the latest changesets from Bitbucket. Apply the patch and resolve any conflicts by hand. Manually register any new files with Mercurial ("hg add") and remove any files that have been deleted ("hg rm --after"). Build your package (and optionally build the entire package tree). Test as seems appropriate. When you are ready (e.g., after testing), commit and push the new version as described in the previous section. Monitor the package smoketest in case your changes break the build of other packages. ** changes to other packages or the top-level repo Sometimes your changes may require changes to code that you don't own. For example, API changes in a new release of your package may break other packages. Also, the top-level repo has some package-specific data, like whether the sources are in the package's top-level directory or in a "lisp" subdirectory. So it's possible you will need to make changes in the top-level repo as well. If you need to make changes in code that you don't own, send mail to xemacs-beta to ask for assistance. Describe the changes that you want to make (and why) and ask for instructions on how to proceed. * Gnus-Specific Information ** What to Merge XEmacs distributes the "stable" release of Gnus. For a long time this was the 5.10 branch. Currently there are at least 3 contenders for the "stable" release, and no direction has been forthcoming from the Gnus developers. The 3 contenders are - the latest release in the 5.10 branch - the last release in the previous development branch (e.g., if "Ma" Gnus is the current development branch, then use the last release from the "No" Gnus branch) - the version of Gnus that was included in the most recent release of GNU Emacs. In any event, tarballs are no longer being posted on gnus.org. Instead, you must download a snapshot from an upstream repo, either the Git repository on gnus.org or the Git repository for Emacs. ** How to Merge *** Files to (Not) Include Upstream Gnus releases contain many files that are not included in the XEmacs Gnus package. The following files from the "lisp" directory should _NOT_ be included in the XEmacs Gnus package, as they are delivered in other XEmacs packages: dig.el dns.el hex-util.el netrc.el pgg-def.el pgg-gpg.el pgg-parse.el pgg-pgp.el pgg-pgp5.el pgg.el pop3.el rfc2104.el sha1.el sieve-manage.el sieve-mode.el sieve.el starttls.el tls.el Corresponding files from the texi directory should also be omitted. We have not been delivering the current smiley.el, either, because an earlier maintainer preferred an old version over the one that is currently included in Gnus releases. The XEmacs Gnus package does include the following files from the "contrib" directory: gpg-ring.el gpg.el hashcash.el nnir.el vcard.el It would make more sense for vcard.el to be included in the mail-lib package, so that it can be more easily used by other MUAs, notably VM. Getting that fixed will require negotiation with the maintainers for mail-lib and VM. _______________________________________________ XEmacs-Beta mailing list XEmacs-Beta(a)xemacs.org http://lists.xemacs.org/mailman/listinfo/xemacs-beta