5:13 a.m.
These are the current failures from regexp-tests.el. Could somebody
please look into these with the intent to fix the bugs. And if you
could let me know that you are working on this, that would be
fantastic and a great help to me.
The results were obtained on a i586 GNU/Linux system by:
xemacs -no-autoloads
M-x load-file RET ./tests/automated/test-harness.el RET
M-x test-emacs-test-file RET ./tests/automated/regexp-tests.el RET
Testing /usr/local/src/xemacs/tests/automated/regexp-tests.el...
Testing Interpreted Lisp
KNOWN BUG: Assertion failed: (not (string-match "\\b" ""))
KNOWN BUG: Assertion failed: (not (string-match "\\b" " "))
KNOWN BUG: Assertion failed: (not (string-match " \\b" " "))
KNOWN BUG: Assertion failed: (not (string-match "\\b " " "))
KNOWN BUG: Assertion failed: (eq (string-match "[^A]" (string ch0)) 0)
KNOWN BUG: Assertion failed: (not (string-match "＠[A]" (string ?＠ ch0)))
KNOWN BUG: Assertion failed: (eq (string-match "＠[^A]" (string ?＠ ch0)) 0)
FAIL: Assertion failed: (not (progn (string-match "a" "a")
(string-match "b" "a") (match-string 0 "a")))
FAIL: Assertion failed: (not (progn (string-match "\\(a\\)" "a")
(string-match "\\(b\\)" "a") (match-string 0 "a")))
FAIL: Assertion failed: (not (progn (string-match "\\(a\\)" "a")
(string-match "\\(b\\)" "a") (match-string 1 "a")))
Testing Compiled Lisp
KNOWN BUG: Assertion failed: (not (string-match "\\b" ""))
KNOWN BUG: Assertion failed: (not (string-match "\\b" " "))
KNOWN BUG: Assertion failed: (not (string-match " \\b" " "))
KNOWN BUG: Assertion failed: (not (string-match "\\b " " "))
KNOWN BUG: Assertion failed: (eq (string-match "[^A]" (string ch0)) 0)
KNOWN BUG: Assertion failed: (not (string-match "＠[A]" (string ?＠ ch0)))
KNOWN BUG: Assertion failed: (eq (string-match "＠[^A]" (string ?＠ ch0)) 0)
FAIL: Assertion failed: (not (progn (string-match "a" "a")
(string-match "b" "a") (match-string 0 "a")))
FAIL: Assertion failed: (not (progn (string-match "\\(a\\)" "a")
(string-match "\\(b\\)" "a") (match-string 0 "a")))
FAIL: Assertion failed: (not (progn (string-match "\\(a\\)" "a")
(string-match "\\(b\\)" "a") (match-string 1 "a")))
SUMMARY for /usr/local/src/xemacs/tests/automated/regexp-tests.el:
184 passes
20 assertion failures
0 errors that should have been generated, but weren't
0 wrong-error failures
0 missing-message failures
0 other failures
Please don't ignore those "KNOWN BUG" failures, they are still bugs
that need to be fixed.
--
|---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---|
| XEmacs - It's not just an editor. |
| It's a way of life. |
|------------------------------------<youngs(a)xemacs.org>---|
3:27 a.m.
Steve Youngs writes:
FAIL: Assertion failed: (not (progn (string-match "a"
"a") (string-match "b" "a") (match-string 0
"a")))
FAIL: Assertion failed: (not (progn (string-match "\\(a\\)" "a")
(string-match "\\(b\\)" "a") (match-string 0 "a")))
FAIL: Assertion failed: (not (progn (string-match "\\(a\\)" "a")
(string-match "\\(b\\)" "a") (match-string 1 "a")))
Emacs 21.2.1 also fails these three tests. For the four tests at the bottom
of regexp-tests.el, Emacs and XEmacs agree. So there might actually be
compatibility issues with fixing these...
Cheers,
Sean MacLennan
4:22 a.m.
On further investigation, it is debatable whether the other failures
are errors. (Please ignore the rest of the email if this has been
talked to death).
The regexps in question all deal with variations of \b on empty or one
space strings. For example
(match-string "\\b" "") ==> 0 but the test expects nil.
If we accept that (string-match "" "") ==> 0, which currently both
Emacs and XEmacs agree on, then you could argue that the match is at
the start of a empty word.
Now my perl is very weak, but I though perl would give an alternative
regexp implementation. Please correct me if any of the following is
wrong.
Perl seems to agree that (string-match "" "") ==> 0
$text = "";
$text =~ // && print "OK\n";
OK
But:
$text =~ /\b/ && print "OK\n";
is silent.
Cheers,
Sean
8:46 a.m.
>>>> "Sean" == Sean MacLennan
<seanm(a)seanm.ca> writes:
Sean> The regexps in question all deal with variations of \b on
Sean> empty or one space strings. For example (match-string "\\b"
Sean> "") ==> 0 but the test expects nil.
Sean> If we accept that (string-match "" "") ==> 0, which
Sean> currently both Emacs and XEmacs agree on, then you could
Sean> argue that the match is at the start of a empty word.
A word is defined to be non-empty. Even if you allowed empty words, I
don't think that interpretation is tenable. It implies
(string-match " \\b" " ") ==> 0
(The target string is two ASCII spaces.) The degenerate example is
sort of an identity relationship.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
4:32 p.m.
Ok, here is a rough patch that fixes the \b failures. I have not
tested it extensively but it does pass the regexp test. However, I
notice the tests never try the \B case so I want to test that more.
The patch is against 21.5-b11 but the same problem exists in 21.4.
Cheers,
Sean
Index: regex.c
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/regex.c,v
retrieving revision 1.42
diff -u -r1.42 regex.c
--- regex.c 2003/03/09 12:59:44 1.42
+++ regex.c 2003/03/21 16:29:17
＠＠ -6071,8 +6071,13 ＠＠
{
/* XEmacs change */
int result;
- if (AT_STRINGS_BEG (d) || AT_STRINGS_END (d))
- result = 1;
+ // SAM
+ if (AT_STRINGS_BEG (d))
+ {
+ re_char *d_after = POS_AFTER_GAP_UNSAFE (d);
+ Ichar emch2 = itext_ichar_fmt (d_after, fmt, lispobj);
+ result = WORDCHAR_P(emch2);
+ }
else
{
re_char *d_before = POS_BEFORE_GAP_UNSAFE (d);
＠＠ -6114,8 +6119,10 ＠＠
case notwordbound:
DEBUG_PRINT1 ("EXECUTING notwordbound.\n");
- should_succeed = 0;
- goto matchwordbound;
+ if (AT_STRINGS_BEG (d) || AT_STRINGS_END (d))
+ goto fail;
+ should_succeed = 0;
+ goto matchwordbound;
case wordbeg:
DEBUG_PRINT1 ("EXECUTING wordbeg.\n");
2:16 a.m.
Thanks very much to both of you, Sean & Steve, for looking into this.
I really do appreciate it.
--
|---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---|
| XEmacs - It's not just an editor. |
| It's a way of life. |
|------------------------------------<youngs(a)xemacs.org>---|
1:29 a.m.
Steve Youngs writes:
Thanks very much to both of you, Sean & Steve, for looking into
this.
I really do appreciate it.
You're welcome. I have also been looking into the other regexp
problem, but as everybody knows Slackware 9.0 came out this week and
this has been a Slackware weekend :-)
Two other things, the regexp debug is too deadly to turn on. I have
made it into a list variable (debug build only) that allows you to
turn it on just when you need it.
Also, I noticed that the regexp tests need a lot of additions. Just in
the \b case, only certain boundry conditions are checked, not that it
works. Also, there are no tests for \B which is tightly bound to
\b. If you work on \b there is a good chance you will break \B.
Cheers,
Sean
2:27 a.m.
>>>> "Sean" == Sean MacLennan
<seanm(a)seanm.ca> writes:
Sean> Also, I noticed that the regexp tests need a lot of
Sean> additions. Just in the \b case, only certain boundry
Sean> conditions are checked, not that it works.
I checked what I could think of at the time. There's no reason not to
add more if you can think of them.
Sean> Also, there are no tests for \B which is tightly bound to
Sean> \b. If you work on \b there is a good chance you will break
Sean> \B.
Not if you do it the way I did, fixing the bug with changes restricted
to the compound statement controlled by the `wordbound:' label.
But it's probably a good idea to add them.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
2:35 a.m.
Stephen J. Turnbull writes:
I checked what I could think of at the time. There's no reason
not
to add more if you can think of them.
The problem with regexps is that there are too many border
conditions. You could right 1,000,000 tests, and still miss one :-(
Not if you do it the way I did, fixing the bug with changes
restricted
to the compound statement controlled by the `wordbound:' label.
Ahhh, but my patch moves one `if' to the notwordbound: case, since it
does not make sense in the wordbound: case (and in fact causes the
'known bug').
Cheers,
Sean
6:41 a.m.
>>>> "Sean" == Sean MacLennan
<seanm(a)seanm.ca> writes:
Sean> Stephen J. Turnbull writes:
> I checked what I could think of at the time. There's no
reason
> not to add more if you can think of them.
Sean> The problem with regexps is that there are too many border
Sean> conditions. You could right 1,000,000 tests, and still miss
Sean> one :-(
Doesn't mean we shouldn't try to approximate it. If you can thinks of
new tests, please don't hesitate to suggest them. (BTW, have you
signed, or would you be willing to sign an FSF assignment, for tests?
It would be a shame, Martin's theories on test suites not
withstanding, if we didn't eventually contribute this to GNU Emacs.)
> Not if you do it the way I did, fixing the bug with changes
> restricted to the compound statement controlled by the
> `wordbound:' label.
Sean> Ahhh, but my patch moves one `if' to the notwordbound: case,
Sean> since it does not make sense in the wordbound: case (and in
Sean> fact causes the 'known bug').
That whole approach is wrong, IMHO, because it makes verifying the
code twice as hard. At least, I can come up with no conditions where
\B != not \b when that code is reached. Thus you only have to check
one. With your method, all test cases need to be checked on both
paths since they use different code to handle them.
Furthermore, your patch probably introduces a crash, because you
indeed can "fall off the end" of a Lisp string. There's no guarantee
that &string + length(string) is in the data segment, let alone that
it is '\000'. (In fact, allowing NUL bytes in strings is one of the
reasons why Lisp doesn't use the C representation for strings.) So
you _must_ check for AT_STRINGS_END(d) in both cases.
Finally, I suspect that \B will fail the corresponding tests using
your patch. You've just "moved the bug" into the other branch of
execution!:-)
Two notes on patch style: don't use // comments in C code; most
platforms don't support it. Second, that particular comment is
unneeded. If you comment on code written by someone else, it _may_ be
appropriate to include some identifier so that you can be asked what
in the world you were thinking. But code changes, if accepted, should
speak for themselves---if they don't, explain in an explicit comment
(and author identity will be in the ChangeLog).
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
3:10 a.m.
Stephen J. Turnbull writes:
Doesn't mean we shouldn't try to approximate it. If you can
thinks of
new tests, please don't hesitate to suggest them. (BTW, have you
signed, or would you be willing to sign an FSF assignment, for tests?
It would be a shame, Martin's theories on test suites not
withstanding, if we didn't eventually contribute this to GNU
Emacs.)
No problem.
That whole approach is wrong, IMHO, because it makes verifying the
code twice as hard. At least, I can come up with no conditions where
\B != not \b when that code is reached. Thus you only have to check
one. With your method, all test cases need to be checked on both
paths since they use different code to handle them.
Ok, agreed. I will remove the if from the notwordbound case. This was
more of an optimization than a necessity.
Furthermore, your patch probably introduces a crash, because you
indeed can "fall off the end" of a Lisp string. There's no guarantee
that &string + length(string) is in the data segment, let alone that
it is '\000'. (In fact, allowing NUL bytes in strings is one of the
reasons why Lisp doesn't use the C representation for strings.) So
you _must_ check for AT_STRINGS_END(d) in both cases.
Can you, or anybody else, come up with a case that causes it to fail?
I have been trying and can't get it to fail, or is it something that
"will work most of the time but will fail sometimes"? Or is
string-match just to restricted to show the failure?
Finally, I suspect that \B will fail the corresponding tests using
your patch. You've just "moved the bug" into the other branch of
execution!:-)
No, because the not case is simplier. The problem with \b is that
currently it is too permissive. If we are at the start or end we
automatically pass. This is what causes (string-match "\\b" " ")
problems. We are at the start of the string, so we pass. With no other
qualifiers, we are done. However the opposite is not
true. (string-match "\\B" " ") fails, which we expect.
Cheers,
Sean
4:31 a.m.
>>>> "Sean" == Sean MacLennan
<seanm(a)seanm.ca> writes:
Sean> Can you, or anybody else, come up with a case that causes
Sean> [lack of AT_STRINGS_END(d)] to fail?
Not easily. We don't "own" the next byte after the end of the
string. To get it to crash depends on the implementation of malloc,
as would getting it to behave incorrectly. But that's exactly why the
post-patch code is wrong, because it accesses something after the end
of the string.
Sean> No, because the not case is simplier.
It's _not_ simpler. It's exactly the same, except that you reverse
the sense of the return value. If this doesn't work something is
seriously wrong; you'll break the algebraic laws that regexps obey.
Have you looked at my patch at all? (BTW, I committed it to 21.5 last
night, with Ben's blessings. ;-)
Sean> The problem with \b is that currently it is too
Sean> permissive. If we are at the start or end we automatically
Sean> pass.
Sean> (string-match "\\B" " ") fails, which we expect.
Huh? There are no word boundaries in the target string; it should
match at 0.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
4:52 a.m.
Stephen J. Turnbull writes:
Have you looked at my patch at all? (BTW, I committed it to 21.5
last
night, with Ben's blessings. ;-)
No. I rarely update the 21.5 stream :-(
My only comment is why `BEGIN_REGEX_MALLOC_OK ();' needs to be
protected by #ifdef emacs, but not `END_REGEX_MALLOC_OK ();'
Cheers,
Sean
6:47 a.m.
>>>> "Sean" == Sean MacLennan
<seanm(a)seanm.ca> writes:
Sean> Stephen J. Turnbull writes:
> Have you looked at my patch at all? (BTW, I committed it to
> 21.5 last night, with Ben's blessings. ;-)
Sean> No. I rarely update the 21.5 stream :-(
If that's a firewall issue, write Steve Youngs. There was a nightly
snapshot tarball, but that went away for some reason. I suppose he
still has his scripts, though, so it should be easy to restart.
Sean> My only comment is why `BEGIN_REGEX_MALLOC_OK ();' needs to
Sean> be protected by #ifdef emacs, but not `END_REGEX_MALLOC_OK
Sean> ();'
Ben agrees that's ugly. I don't understand the macros involved well
enough to be willing to change it at this point. The REGEX_MALLOC_OK
macros just set and reset a flag, and the END version leaves it safely
"off".
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
7913
days inactive
7918
days old
13 comments
3 participants
participants (3)
-
Sean MacLennan -
Stephen J. Turnbull -
Steve Youngs