11:09 p.m.
Any clue how the hell I work around this?
Go to your *scratch* buffer. Insert "Ü" (that's \334, Uuml)
(re-search-forward "Ü") => works
(re-search-forward "[Ü]") => fails
XEmacs 21.4.12, Red Hat 9.
--
Jamie Zawinski
jwz(a)jwz.org http://www.jwz.org/
jwz(a)dnalounge.com http://www.dnalounge.com/
2:24 a.m.
|--==> "JZ" == Jamie Zawinski <jwz(a)jwz.org> writes:
JZ> Any clue how the hell I work around this?
Seems to work for me in 21.5.15. Lemme check in 21.4, no doesn't work
there.
I suppose you'd have to balance up how much you need this
functionality against the big can of worms you'll open by moving to
21.5. And I'm sure that someone like yourself, Jamie, will find lots
of worms in 21.5, big fat hairy ones.
--
|---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---|
| XEmacs - The only _______ you'll ever need. |
| Fill in the blank, yes, it's THAT good! |
|------------------------------------<youngs(a)xemacs.org>---|
4:22 a.m.
Er, since when did it break? Might it have always been broken (in
Mule)? (I can't recall any recent changes up to 21.4.12 that could
cause this. Sorry, I myself can't test quickly, all my old xemacsen
are broken by wanting old libraries. :-( )
>>>> "Jamie" == Jamie Zawinski
<jwz(a)jwz.org> writes:
Jamie> Any clue how the hell I work around this? Go to your
Jamie> *scratch* buffer. Insert "Ü" (that's \334, Uuml)
Jamie> (re-search-forward "Ü") => works
Jamie> (re-search-forward "[Ü]") => fails
Nice catch! Sorry about that.
(re-search-forward "Ü\\|[xyz]")
(re-search-forward "\\(Ü\\|Ü\\)")
(re-search-forward "\\[Ü\\]")
(re-search-forward "\\[Ü\\|Ü\\]")
all work as expected for me in this buffer (21.4.almost14 Mule build,
IIRC RH is Mule too). YMMV, and I haven't tested more complex
expressions.
ETA (maybe even a patch) on a real fix in a followup (but it'll be in
the C code, which won't be relevant to you for a while I guess, sorry
again:-( ).
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
2:35 p.m.
Jamie Zawinski <jwz(a)jwz.org> writes:
Any clue how the hell I work around this?
Go to your *scratch* buffer. Insert "Ü" (that's \334, Uuml)
(re-search-forward "Ü") => works
(re-search-forward "[Ü]") => fails
I can repeat this.
I think the problem is in the case conversions inside the regexp
engine. If I search for "[ü]" instead, or if I set `case-fold-search'
to nil, the regexp succeeds. I'm not sure if either of these
workarounds would work for you, but that's the best I can come up with
right now.
Yoshiki Hayashi, who introduced non-ASCII case folding to regex.c,
might be in the position to fix this one. Yoshiki, if you're still
around, could you take a look at this one?
7:11 p.m.
Hrvoje Niksic <hniksic(a)xemacs.org> writes:
> Go to your *scratch* buffer. Insert "Ü" (that's
\334, Uuml)
>
> (re-search-forward "Ü") => works
> (re-search-forward "[Ü]") => fails
I can repeat this.
I think the problem is in the case conversions inside the regexp
engine. If I search for "[ü]" instead, or if I set `case-fold-search'
to nil, the regexp succeeds. I'm not sure if either of these
workarounds would work for you, but that's the best I can come up with
right now.
Yes, it's case conversion bug.
Yoshiki Hayashi, who introduced non-ASCII case folding to regex.c,
might be in the position to fix this one. Yoshiki, if you're still
around, could you take a look at this one?
OK, here's the patch to fix it.
I take the blame for not implementing it correctly. I must
confess that I've never really understood charset_mule part
of regex engine.
We need more test cases to see fastmap and charset_mule are
working correctly but I don't have time to write thorough
test cases right now. Please wait until the weekend. I'll
probably be able to find some time to work on this again.
2003-09-12 Yoshiki Hayashi <yoshiki(a)xemacs.org>
* regex.c (TRANSLATE_EXTENDED_UNSAFE): Remove.
(re_search_2): Match the fisrt byte of Bufbyte, not Emchar.
(re_match_2_internal): Use TRANSLATE instead of
TRANSLATE_EXTENDED_UNSAFE. The latter was a hack to bypass
non-ASCII char case conversion.
Index: src/regex.c
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/regex.c,v
retrieving revision 1.25.2.4
diff -u -r1.25.2.4 regex.c
--- src/regex.c 2003/06/19 03:34:42 1.25.2.4
+++ src/regex.c 2003/09/12 18:01:01
＠＠ -1591,13 +1591,6 ＠＠
when we use a character as a subscript we must make it unsigned. */
#define TRANSLATE(d) (TRANSLATE_P (translate) ? RE_TRANSLATE (d) : (d))
-#ifdef MULE
-
-#define TRANSLATE_EXTENDED_UNSAFE(emch) \
- (TRANSLATE_P (translate) && emch < 0x80 ? RE_TRANSLATE (emch) : (emch))
-
-#endif
-
/* Macros for outputting the compiled pattern into `buffer'. */
/* If the buffer isn't allocated when it comes in, use this. */
＠＠ -4114,10 +4107,12 ＠＠
{
#ifdef MULE
Emchar buf_ch;
+ Bufbyte str[MAX_EMCHAR_LEN];
buf_ch = charptr_emchar (d);
buf_ch = RE_TRANSLATE (buf_ch);
- if (buf_ch >= 0200 || fastmap[(unsigned char) buf_ch])
+ set_charptr_emchar (str, buf_ch);
+ if (buf_ch >= 0200 || fastmap[(unsigned char) *str])
break;
#else
if (fastmap[(unsigned char)RE_TRANSLATE (*d)])
＠＠ -4865,7 +4860,7 ＠＠
REGEX_PREFETCH ();
c = charptr_emchar ((const Bufbyte *) d);
- c = TRANSLATE_EXTENDED_UNSAFE (c); /* The character to match. */
+ c = TRANSLATE (c); /* The character to match. */
if (EQ (Qt, unified_range_table_lookup (p, c, Qnil)))
not_p = !not_p;
--
Yoshiki Hayashi
2:35 a.m.
New subject: [AC21.4] thank you for breaking latin1 re-searches
APPROVE COMMIT 21.4
Yoshiki, I don't have the knowledge to assess this patch, but if you
think it's good, I approve it. More tests in this area would be
appreciated.
Thanks,
Vin
Yoshiki Hayashi <yoshiki(a)xemacs.org> writes:
Hrvoje Niksic <hniksic(a)xemacs.org> writes:
>> Go to your *scratch* buffer. Insert "Ü" (that's \334, Uuml)
>>
>> (re-search-forward "Ü") => works
>> (re-search-forward "[Ü]") => fails
>
> I can repeat this.
>
> I think the problem is in the case conversions inside the regexp
> engine. If I search for "[ü]" instead, or if I set `case-fold-search'
> to nil, the regexp succeeds. I'm not sure if either of these
> workarounds would work for you, but that's the best I can come up with
> right now.
Yes, it's case conversion bug.
> Yoshiki Hayashi, who introduced non-ASCII case folding to regex.c,
> might be in the position to fix this one. Yoshiki, if you're still
> around, could you take a look at this one?
OK, here's the patch to fix it.
I take the blame for not implementing it correctly. I must
confess that I've never really understood charset_mule part
of regex engine.
We need more test cases to see fastmap and charset_mule are
working correctly but I don't have time to write thorough
test cases right now. Please wait until the weekend. I'll
probably be able to find some time to work on this again.
2003-09-12 Yoshiki Hayashi <yoshiki(a)xemacs.org>
* regex.c (TRANSLATE_EXTENDED_UNSAFE): Remove.
(re_search_2): Match the fisrt byte of Bufbyte, not Emchar.
(re_match_2_internal): Use TRANSLATE instead of
TRANSLATE_EXTENDED_UNSAFE. The latter was a hack to bypass
non-ASCII char case conversion.
Index: src/regex.c
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/regex.c,v
retrieving revision 1.25.2.4
diff -u -r1.25.2.4 regex.c
--- src/regex.c 2003/06/19 03:34:42 1.25.2.4
+++ src/regex.c 2003/09/12 18:01:01
＠＠ -1591,13 +1591,6 ＠＠
when we use a character as a subscript we must make it unsigned. */
#define TRANSLATE(d) (TRANSLATE_P (translate) ? RE_TRANSLATE (d) : (d))
-#ifdef MULE
-
-#define TRANSLATE_EXTENDED_UNSAFE(emch) \
- (TRANSLATE_P (translate) && emch < 0x80 ? RE_TRANSLATE (emch) : (emch))
-
-#endif
-
/* Macros for outputting the compiled pattern into `buffer'. */
/* If the buffer isn't allocated when it comes in, use this. */
＠＠ -4114,10 +4107,12 ＠＠
{
#ifdef MULE
Emchar buf_ch;
+ Bufbyte str[MAX_EMCHAR_LEN];
buf_ch = charptr_emchar (d);
buf_ch = RE_TRANSLATE (buf_ch);
- if (buf_ch >= 0200 || fastmap[(unsigned char) buf_ch])
+ set_charptr_emchar (str, buf_ch);
+ if (buf_ch >= 0200 || fastmap[(unsigned char) *str])
break;
#else
if (fastmap[(unsigned char)RE_TRANSLATE (*d)])
＠＠ -4865,7 +4860,7 ＠＠
REGEX_PREFETCH ();
c = charptr_emchar ((const Bufbyte *) d);
- c = TRANSLATE_EXTENDED_UNSAFE (c); /* The character to match. */
+ c = TRANSLATE (c); /* The character to match. */
if (EQ (Qt, unified_range_table_lookup (p, c, Qnil)))
not_p = !not_p;
--
Yoshiki Hayashi
10:07 p.m.
Hrvoje Niksic wrote:
if I set `case-fold-search' to nil, the regexp succeeds. I'm not sure
if either of these workarounds would work for you, but that's the best
I can come up with right now.
Yup, I can live with case-fold-search nil. Thanks!
--
Jamie Zawinski
jwz(a)jwz.org http://www.jwz.org/
jwz(a)dnalounge.com http://www.dnalounge.com/
7553
days inactive
7556
days old
7 comments
7 participants
participants (7)
-
Hrvoje Niksic -
Jamie Zawinski -
Michael Hill -
Stephen J. Turnbull -
Steve Youngs -
Vin Shelton -
Yoshiki Hayashi