Re: [V21.5] test-harness.el to log results when run noninteractive
Adrian Aichner <adrian(a)xemacs.org> writes:
>>>>> "SJT" == Stephen J Turnbull
<stephen(a)xemacs.org> writes:
SJT> VETO 21.5
SJT> I recommend reversion in 21.4, too.
SJT> Adrian, please revert the hunk below. It pollutes all
SJT> reports, even
No problem. I'll revert under mild protest.
SJT> those which have 100% passes. It is of dubious value in helping to
SJT> get problems fixed (there still is no patch to work around the
SJT> Microsoft implementation of %e format specifiers, for example).
Following that "Logic" I will have to VETO your new
test for breakage in regexps \{m,n\} construct
patch is a few weeks if it does not magically fix the breakage it
tests for.
So basically you don't want to see failures because they won't get
fixed anyways.
And we cannot include test results because they include so many
passes.
That all makes no sense to me.
Stephen,
I happen to agree that we don't really care about passes, but I ALWAYS
want to know about failures. Your latest patch doesn't produce the
failure information needed:
SUMMARY for /opt/src/xemacs-21.4-2003-09-25/tests/automated/mule-tests.el:
2 passes
0 assertion failures
0 errors that should have been generated, but weren't
0 wrong-error failures
0 missing-message failures
0 other failures
Testing /opt/src/xemacs-21.4-2003-09-25/tests/automated/regexp-tests.el...
regexp-tests.el: 248 of 252 tests successful (98%).
I'd much rather wade through tons of bogus passes in order to find out
what tests actually failed.
Adrian, perhaps it would be best to have all the PASS results printed
only optionally. What do you think?
- Vin