RFC: test-harness and regression tests as (a) package(s)
Stephen J. Turnbull
stephen at xemacs.org
Thu Dec 23 21:50:18 EST 2004
>>>>> "Ville" == Ville Skytt <Ville> writes:
Ville> On Thu, 2004-12-23 at 12:28 +0900, Stephen J. Turnbull
>> Why don't we have a regression test? Is it too hard to write
>> them (ie, something better documentation would help with), or
>> lack of time, or what?
Ville> I guess it's just that nobody has done it. Has anyone
Ville> thought how tests should be integrated into the packages
Ville> tree? Common infrastructure for that would be nice, and
Ville> even if it's only that at the beginning, having it around
Ville> could lower the barrier of folks starting to write tests.
(defun test-harness-install (add-to-load-path)
"Install the `test-emacs-test-file' facility.
With prefix argument, also put the test suite directory on the path.
Then you can easily run the test suite and do `find-library' on
setq `xemacs-source-root' to the root of your XEmacs source tree."
(let ((d (file-name-as-directory
(expand-file-name "tests/automated" xemacs-source-root))))
(load (expand-file-name "test-harness.el" d) nil nil t)
(when add-to-load-path (add-to-list 'load-path d))))
I've had making a unit-test package and XEmacs regression test package
(a la Python's py-unit and doctest) in mind for a long time, but was
discouraged by Martin Buchholz, and didn't feel like fighting it at
the time. With both you and Skip suggesting this, I think it's time,
but it will probably take a couple of months.
If somebody wants to do this, it shouldn't be hard---just bundle up
the stuff in CVS tests/automated as a package, make sure that it has
no autoloads except for an install function similar to the above.
Optionally run all the tests on 21.1.14 and 21.1.16, and report any
failures as bugs. We'll need to make a version test and disable any
tests depending on features a version doesn't have, then turn anything
that 21.1 doesn't pass into an EXPECTED ERROR---WON'T FIX, and consult
with Vin about what needs fixing in 21.4.
We will need to make the test-harness a little more friendly to
non-core usage, but I think it's quite usable as it is.
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
More information about the XEmacs-Beta