Re: The tm* binaries should not live in core XEmacs

[Moved to xemacs-beta] William M Perry <wmperry(a)aventail.com&gt; writes in xemacs-review(a)xemacs.org: Standalone in the sense that you can plop it down anywhere on a sytem with just a functional emacs and have it work. Standalone also in the sense that it carries around all the autoconf cruft files (config.sub, config.guess, configure.in, etc.). Have you looked at how I currently have the xemacs-packages directory hierarchy arranged? Here's a brief summary. * Major design goals ** Hierarchical in structure and grouped by major functionality. The top level of the structure looks something like: xemacs-packages/: comm/ games/ libs/ mule/ oa/ os/ prog/ wp/ Lisp packages are stored in the second level. For example, the games directory looks like: xemacs-packages/games/: cookie/ games/ mine/ misc-games/ where each of the directories represents a package. Each package is stored in CVS as a separate individual module. There are a number of `glue' files to paste everything together. xemacs-packages/package-compile.el handles setting up the load path and feeding .el files to the bytecompiler. xemacs-packages/XEmacs.rules common Makefile rules that are to be `include'd by each individual package Makefile. xemacs-packages/Makefile Top level Makefile responsible for calling sub-Makefiles. Each functional directory has a Makefile that calls each of the package Makefiles in turn. This is all complicated by the fact that order is important in compiling. The `xemacs-base', `fsf-compat', `mule-base' and various other packages are so frequently used as libraries by other packages that they need to be bytecompiled first. Thus it is not possible to blindly do a depth first walk of the tree (eg. in shell script: for group in *; do cd ${group} for package in *; do cd ${package} make done done ** Provide a clean room-type controlled environment for bytecompilation. It is a deliberate feature that the source directories are not directly usable as an XEmacs package run-time hierarchy. Rules for building individual packages bring in only the environment necessary for bytecompilation and everything else is ignored. In particular, except for core lisp, *all* of the XEmacs run-time Lisp in the binary being used for bytecompilation is ignored. Thus, barring radical changes in the bytecode, it should be possible a year from now to bootstrap a Lisp run-time for XEmacs 21.5 with an XEmacs 21.0 executable. ** The ability to issue single commands at the top level to perform building, cleaning, etc. At the current time, the working targets are `clean' -- wipe out .elcs, but not auto-autoloads.el and custom-load.el files, `distclean' -- wipe out everything except raw source, `bindist' -- generate a directory of binary package tarballs for distribution via ftp.xemacs.org. A planned, but currently unimplemented target is `install' -- bypass tarball generation and install directly into a run-time XEmacs package hierarchy. It is also planned that the `install' target be able to consult some kind of configuration to limit what gets installed. For example, it should be possible to have the VM (a random choice) package present as source but not installed into the run-time. ** The ability to gracefully handle missing modules to the extent possible. In other words, if the `gnus' package isn't present, toplevel commands shouldn't fail. ** The ability to deal with Mule -vs- Non-Mule builds. Non-Mule building should fail gracefully, but not otherwise impede progress. Everything described to this point except as noted has been implemented. My implementation is heavily dependent on GNU make and works in fairly un-make-like fashion except at the individual package level. Thus, if autoconf is up to the task, there is nothing conceptually prohibiting its usage other than my own almost-total lack of expertise with it. * Stuff that isn't being handled that should be. ** There is no good way to build binary executables. This has to be carefully thought out. There is a dual purpose that must be kept -- building for distribution must not build and include binaries but building for local use should. ** There is no good way to bootstrap 100% from source. That is, given an XEmacs core distribution tarball from ftp.XEmacs.org and given a fresh `cvs checkout' of all the XEmacs packages, there isn't a single command to issue to build everything. The steps would be roughly: 1. Configure XEmacs and build all the way to temacs. 2. Use temacs to build run-time packages of fsf-compat and mule-base (in that order) and install them into the directory being used to build XEmacs. 3. Go back to the core and create a dumped XEmacs binary, which will now work due to the presence of Mule elcs. 4. Go back to the package source area and wipe out the products of step 2. 5. Build the package world and install it into the location that the XEmacs binary will find it. 6. Make install in the XEmacs core. Note: this isn't very different than how XEmacs 20.4 is built from scratch. The temacs bytecompiling in step 2 is equivalent to running update-elc.el. ** single-file packages are not completely implemented. Packages labelled `single-file' are otherwise unrelated collections of independent Lisp files. It is intended by design that one may specify a subset of files in them and only build & install the subset. Implementing this is probably along the same lines as the subset install feature described above. What kind of magic can you do with autoconf to help? Any questions? :-)