>>>> "mb" == Martin Buchholz
<martin(a)xemacs.org> writes:
mb> I hope you're not suggesting that there be only one set of
mb> tests, to be applied to all different versions of the
Of course I am.
mb> software. The point of a test suite is to reflect what works
mb> in any particular version.
Well, why not just reduce it to
(message "All tests passed.")
and make absolutely sure that works, then? Because the point of a
test suite is to reflect a _specification_, ie, what _should_ work, of
course!
If "stuff" depends on new features, then condition on them, and emit
"unimplemented" warnings. Don't condition on versions; who knows if a
fix might not be backported? If you must condition on versions, when
"stuff" is known not to work in a particular version and there's no
feature to test, you should either accept the failure or (if it would
impact the system) disable the test and emit a "disabled" warning.
mb> Test suites are the ultimate non-portable software,
Test _scaffolding_ is of course non-portable, since it may depend on
implementation-specific hooks and extensions to instrument behavior.
Test _suites_ should be portable to any implementation that purports
to comply with the specification.
mb> and rightly so.
Necessary evil, at best.
--
Institute of Policy and Planning Sciences
http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.