1:02 p.m.
Save the following attached file:
(This file is only gzipped, because Gnus runs into an endless loop
when I attach it without gzipping. That is probably also a bug, but a
different one.).
Gunzip it, start XEmacs and let it load this file:
mfabian＠gregory:~/xemacs-bug$ LANG=ja_JP xemacs -q -vanilla kanji.euc-iso
Fatal error (11).
Your files have been auto-saved.
Use `M-x recover-session' to recover them.
It is important to use "LANG=ja_JP", it will not happen with
"LANG=C".
The file "kanji.euc-iso" contains only two lines. Both lines contain
only the Japanese word "漢字" (Kanji). The first line contains this
word in EUC-JP encoding, the second line in ISO-2022-JP encoding.
The order of these lines is also important, if the ISO-2022-JP line
comes first like in the gunzipped version of this file:
XEmacs will not crash (but of course the EUC-JP part is unreadable).
I encounter this problem often when opening Japanese *.patch files,
which often contain both EUC-JP in some hunks and ISO-2022-JP in other
hunks. If the EUC-JP comes first, XEmacs will crash.
I use the following XEmacs version:
In XEmacs 21.1 (patch 10) "Capitol Reef" [Lucid] (i686-pc-linux, Mule) of Mon
May 8 2000 on nozomi
configured using `configure --prefix=/opt/stow/xemacs-21.1.10 --with-mule=yes
--with-dialogs=athena3d --with-xface=yes --with-wnn=yes --with-canna=yes --with-xim=xlib
--with-xfs --with-menubars=lucid --site-includes=/opt/include --site-libraries=/opt/lib
'--cflags=-g -O4 -fno-strength-reduce''
(The same problem also occurs with the XEmacs 21.1.12 from the SuSE
rpm).
Here is the lisp backtrace:
And here is the C backtrace:
--
Mike Fabian <mfabian(a)suse.de>
4:22 a.m.
>>>> "Mike" == Mike Fabian
<mfabian(a)suse.de> writes:
Mike> mfabian＠gregory:~/xemacs-bug$ LANG=ja_JP xemacs -q -vanilla kanji.euc-iso
I've reproduced this up to the Lisp backtrace in 21.1 (patch 12)
"Channel Islands" plus some CVS updates. I should be able to take a
close look at it later today. It's not obvious to me what's happening
here though, so I can't promise a quick fix.
Vin, I haven't heard anything about a new release recently, but in
case you're thinking about it, it would be nice to get a fix for this
one out soon, OK? It's a bad bug, can get tickled easily.
--
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091
_________________ _________________ _________________ _________________
What are those straight lines for? "XEmacs rules."
10:25 a.m.
>>>> "Mike" == Mike Fabian
<mfabian(a)suse.de> writes:
Mike> LANG=ja_JP xemacs -q -vanilla kanji.euc-iso
>>>> "Stephen" == Stephen J Turnbull
<turnbull(a)sk.tsukuba.ac.jp> writes:
Stephen> I've reproduced this up to the Lisp backtrace in 21.1
Stephen> (patch 12) "Channel Islands" plus some CVS updates. I
Stephen> should be able to take a close look at it later today.
Stephen> It's not obvious to me what's happening here though, so I
Stephen> can't promise a quick fix.
Oh, boy, are things fxxked up here.
What is happening is that the presence of undesignated characters from
GR followed by the ISO-2022 escape sequences causes Mule to
auto-detect the coding category as 'iso-lock-shift, and then the
coding system itself is set to 'iso-2022-lock-unix. There are some
bugs in the implementation of this coding system, such that characters
which are represented as negative Emchars by MAKE_CHAR are generated by
decode-coding-region. This should not happen; I believe the range of
Emchars is still only 19 bits, so with a 30-bit character
representation there's no excuse for wrap-around. :-(
I don't understand why this happens yet. I will have some kind of
patch in about two days (heavy class load next two days); either I'll
make the "safe" coding-priority-list the default and document the
problem, or I'll have a real fix. If somebody else can do something
useful in the interim, I'd be much obliged!
A work-around is to put
(defun make-coding-priority-safe ()
"Give `no-conversion' higher priority than some buggy coding categories.
`iso-lock-shift' is known to cause crashes in a Japanese environment in
certain situations, and `iso-8-designate' is rare and perhaps also not
to be trusted."
(interactive)
(set-coding-priority-list
(let* ((buggy '(iso-lock-shift iso-8-designate))
(cpl (delq 'no-conversion (coding-priority-list)))
(ret cpl))
(while (and (cdr cpl)
(not (memq (car (cdr cpl)) buggy)))
(setq cpl (cdr cpl)))
(setcdr cpl (cons 'no-conversion (cdr cpl)))
ret)))
;; Not very safe; should check for current coding system and buffer
;; change status etc.
(defun convert-buffer-using-coding-system (coding-system)
"Convert the whole buffer according to CODING-SYSTEM.
Should only be called on an unnarrowed binary buffer with a known
external encoding. Any other use will have undefined results."
(interactive "SCoding system: ")
(if (coding-system-p coding-system)
(decode-coding-region (point-min) (point-max) coding-system)
(error "You bozo! Try again, with a REAL coding system this time.")))
(make-coding-priority-safe)
in ~/.emacs and to call it any time you change your language
environment or the function `set-coding-priority-list' by hand. Those
are the only ways I know of that coding-priority-list changes.
The file in question will be left in binary form.
`convert-buffer-using-coding-system' is a convenience function to do
the conversion at user request. In Mike's case, use of 'euc-jp gives
appropriate results. You can do this at any time, even after editing
the buffer, as long as you have not screwed up the external encoding
(eg, by altering an escape sequence or changing or deleting one of the
bytes in a multibyte character).
--
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091
_________________ _________________ _________________ _________________
What are those straight lines for? "XEmacs rules."
4:23 p.m.
Stephen,
Do you have a patch for this?
- vin
"Stephen J. Turnbull" <turnbull(a)sk.tsukuba.ac.jp> writes:
>>>>> "Mike" == Mike Fabian
<mfabian(a)suse.de> writes:
Mike> LANG=ja_JP xemacs -q -vanilla kanji.euc-iso
>>>>> "Stephen" == Stephen J Turnbull
<turnbull(a)sk.tsukuba.ac.jp> writes:
Stephen> I've reproduced this up to the Lisp backtrace in 21.1
Stephen> (patch 12) "Channel Islands" plus some CVS updates. I
Stephen> should be able to take a close look at it later today.
Stephen> It's not obvious to me what's happening here though, so I
Stephen> can't promise a quick fix.
Oh, boy, are things fxxked up here.
What is happening is that the presence of undesignated characters from
GR followed by the ISO-2022 escape sequences causes Mule to
auto-detect the coding category as 'iso-lock-shift, and then the
coding system itself is set to 'iso-2022-lock-unix. There are some
bugs in the implementation of this coding system, such that characters
which are represented as negative Emchars by MAKE_CHAR are generated by
decode-coding-region. This should not happen; I believe the range of
Emchars is still only 19 bits, so with a 30-bit character
representation there's no excuse for wrap-around. :-(
I don't understand why this happens yet. I will have some kind of
patch in about two days (heavy class load next two days); either I'll
make the "safe" coding-priority-list the default and document the
problem, or I'll have a real fix. If somebody else can do something
useful in the interim, I'd be much obliged!
A work-around is to put
(defun make-coding-priority-safe ()
"Give `no-conversion' higher priority than some buggy coding categories.
`iso-lock-shift' is known to cause crashes in a Japanese environment in
certain situations, and `iso-8-designate' is rare and perhaps also not
to be trusted."
(interactive)
(set-coding-priority-list
(let* ((buggy '(iso-lock-shift iso-8-designate))
(cpl (delq 'no-conversion (coding-priority-list)))
(ret cpl))
(while (and (cdr cpl)
(not (memq (car (cdr cpl)) buggy)))
(setq cpl (cdr cpl)))
(setcdr cpl (cons 'no-conversion (cdr cpl)))
ret)))
;; Not very safe; should check for current coding system and buffer
;; change status etc.
(defun convert-buffer-using-coding-system (coding-system)
"Convert the whole buffer according to CODING-SYSTEM.
Should only be called on an unnarrowed binary buffer with a known
external encoding. Any other use will have undefined results."
(interactive "SCoding system: ")
(if (coding-system-p coding-system)
(decode-coding-region (point-min) (point-max) coding-system)
(error "You bozo! Try again, with a REAL coding system this time.")))
(make-coding-priority-safe)
in ~/.emacs and to call it any time you change your language
environment or the function `set-coding-priority-list' by hand. Those
are the only ways I know of that coding-priority-list changes.
The file in question will be left in binary form.
`convert-buffer-using-coding-system' is a convenience function to do
the conversion at user request. In Mike's case, use of 'euc-jp gives
appropriate results. You can do this at any time, even after editing
the buffer, as long as you have not screwed up the external encoding
(eg, by altering an escape sequence or changing or deleting one of the
bytes in a multibyte character).
--
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091
_________________ _________________ _________________ _________________
What are those straight lines for? "XEmacs rules."
1:09 a.m.
>>>> "vin" == Vin Shelton <acs(a)xemacs.org>
writes:
vin> Stephen , Do you have a patch for this?
Aaargh. No, I took a further look at it, it was deeper than the time I
had.
I'll have a patch to paper things over for pretty much everybody but
Koreans, who are probably safe because although they use
'iso-lock-shift, that is not directed to 'iso-2022-lock-unix, but to
'euc-kr or 'iso-2022-kr, I forget which.
Don't wait on me, though, because I must verify that there is no code
which expects to use 'iso-lock-shift but fails to upgrade its
priority.
--
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091
_________________ _________________ _________________ _________________
What are those straight lines for? "XEmacs rules."
8541
days inactive
8606
days old
4 comments
3 participants
participants (3)
-
Mike FABIAN -
Stephen J. Turnbull -
Vin Shelton