>>>> "mb" == Martin Buchholz
<martin(a)xemacs.org> writes:
mb> Stephen, I consider your argument here utterly wrong.
I know. That's why I asked you, after all. So why don't you debunk
it, instead of answering rhetorical questions with punctuation marks
and appealing to ideals that XEmacs will not satisfy unless somebody
with deep pockets hires a few of us to work on XEmacs full-time?
mb> Please reconsider.
I haven't decided anything yet. But so far all your arguments
have done is support my feeling that this is a good idea as a
practical matter.
mb> If you backport a fix, you can backport the regression test
mb> that goes with it. Porting the test is usually much easier.
This is incorrect thinking, for a practical and a theoretical reason.
In practice, it is not done, until somebody scratches their
perfectionist itch. Often years later, as many of our developers
aren't even aware that they should be writing tests, and it is a skill
that very few of us have, as yet. It's nearly completely undocumented
(I wrote what docs exist AFAICT; comments in test-harness.el don't
count---if you can find those, you've already passed the halfway point
in the obstacle course).
In theory, this ignores the fact that the test should be run against
previous versions still in use (_not_ in current distribution) to
determine whether they also suffer from the bug.
mb> I think precisely the opposite. Disciplined organizations
mb> require bug fixes to be accompanied by a test case which fails
mb> before the bug fix and succeeds afterwards.
XEmacs is not, and will not be in the forseeable future, a disciplined
organization by that definition. Even those individuals who try to be
disciplined in their own work disagree, often violently, on (a) what
that entails and (b) how to extend that to the whole project.
SJT> Test _suites_ should be portable to any implementation that
SJT> purports to comply with the specification.
mb> Implementation-independent test suites are most useful when
mb> there is a formal standard with multiple implementations.
mb> Even if such tests existed, it does not relieve implementors
mb> of the burden of creating their own white-box tests for their
mb> own specific implementation, as happens, e.g. with gcc and Sun
mb> Java.
Of course. So what?
As I made clear, I see the necessity for version-specific tests. I
think it is very bad practice to duplicate the vast majority of tests
which should be passed by all recent Emacsen or which can require a
feature. Your answer to that is "if we were disciplined, we'd do it
my way." Too bad; there are a lot of things that would work better in
this project if we raised the level of discipline. This aspect of
discipline is a very low priority.
OTOH, packaging test-harness and tests is likely to get more tests run
_and_ written IMO.
--
Institute of Policy and Planning Sciences
http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.