4:56 p.m.
it seems I sent that to the
list, and not to you explicitly.
Then thanks for resending, since I am not on any of the lists.
Adding `ccl-program-definition' did not directly fix the broken
conversion. It was practically necessary to figure what C code to
look at, and will remain so until no CCL coding system is ever used.
Will post separate patches on request.
src/ChangeLog
2006-10-31
* mule-ccl.c (ccl_driver): Fix implementation of
`CCL_WriteMultibyteChar2' / `write-multibyte-character': when
passed valid representations of `control-1' charset characters,
write the desired emchar, not garbage.
2006-10-29
* mule-ccl.c (Fccl_program_definition): New.
ChangeLog
2006-10-31
* etc/NEWS: Document fix of writing `control-1' emchars in CCL.
2006-10-29
* etc/NEWS: Document adding `ccl-program-definition'.
--- src/mule-ccl.c
+++ src/mule-ccl.c
＠＠ -1372,8 +1372,6 ＠＠
i = reg[RRR]; /* charset */
if (i == LEADING_BYTE_ASCII)
i = reg[rrr] & 0xFF;
- else if (LEADING_BYTE_CONTROL_1 == i)
- i = ((reg[rrr] & 0xFF) - 0xA0);
else if (POSSIBLE_LEADING_BYTE_P(i) &&
!NILP(CHARSET_BY_LEADING_BYTE(i)))
{
＠＠ -2257,6 +2255,18 ＠＠
return idx;
}
+/* Unfortunately, can not avoid modifying core code with this
+ addition. `ccl_get_compiled_code' is `static', so can not put this
+ function in dynamic module. */
+DEFUN ("ccl-program-definition", Fccl_program_definition, 1, 1, 0, /*
+Return CCL program registered as NAME. Retrieves data set up with
+`register-ccl-program'.
+*/
+ (name))
+{
+ return ccl_get_compiled_code (name);
+}
+
void
syms_of_mule_ccl (void)
＠＠ -2266,6 +2276,7 ＠＠
DEFSUBR (Fccl_execute_on_string);
DEFSUBR (Fregister_ccl_program);
DEFSUBR (Fregister_code_conversion_map);
+ DEFSUBR (Fccl_program_definition);
}
void
--- etc/NEWS
+++ etc/NEWS
＠＠ -122,6 +122,8 ＠＠
- New file extensions recognized: .ss, .pdb, .psw.
-- Fixed ldap libraries configuration.
-- Fixed `LDAP_OPT_ON' libraries configuration.
+ -- Added `ccl-program-definition'.
+ -- Fixed writing `control-1' emchars in CCL.
** The delete key now deletes forward by default.
＠＠ -888,6 +890,22 ＠＠
evaluated expression. This makes it parallel with other descriptors,
which can also be expressions.
+** Added `ccl-program-definition'.
+
+It (along with `ccl-dump') is required to see what ccl code is
+actually used in current program state. Thus, a debugging feature.
+
+** Fixed writing `control-1' emchars in CCL.
+
+Previously, if CCL program passed valid representations of such
+characters to `write-multibyte-character', it would write garbage, not
+the desired emchar. Introduced in upstream revision 1.15.2.1
+(2005/11/28 02:10:19 +0), replaced with another garbage writing in
+upstream revision 1.15.2.2 (2005/12/07 01:17:03 +0). One CCL
+programming task affected by this is extending `mule-ucs' coding
+system decoding programs to decode unicode code points from 0x80 to
+0x9f to these emchars.
+
* Changes in XEmacs 21.0
========================
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
10:36 a.m.
New subject: decoders writing control-1 chars?
Ar an t-ochtú lá de mí na Samhain, scríobh Ilya N. Golubev:
> it seems I sent that to the
> list, and not to you explicitly.
Then thanks for resending, since I am not on any of the lists.
Adding `ccl-program-definition' did not directly fix the broken
conversion. It was practically necessary to figure what C code to
look at, and will remain so until no CCL coding system is ever used.
Will post separate patches on request.
[...]
--- src/mule-ccl.c
+++ src/mule-ccl.c
＠＠ -1372,8 +1372,6 ＠＠
i = reg[RRR]; /* charset */
if (i == LEADING_BYTE_ASCII)
i = reg[rrr] & 0xFF;
- else if (LEADING_BYTE_CONTROL_1 == i)
- i = ((reg[rrr] & 0xFF) - 0xA0);
else if (POSSIBLE_LEADING_BYTE_P(i) &&
!NILP(CHARSET_BY_LEADING_BYTE(i)))
{
＠＠ -2257,6 +2255,18 ＠＠
Your change crashes a debug build of 21.5 when the following code is
executed:
(let ((ccl-vector [0 0 0 0 0 0 0 0 0]))
(define-ccl-program ccl-write-two-control-1-chars
`(1
((r0 = 143)
(r1 = 0)
(write-multibyte-character r0 r1)
(r1 = 31)
(write-multibyte-character r0 r1)))
"CCL program to transform Mule characters to UCS-2.")
(assert (equal
(ccl-execute-on-string 'ccl-write-two-control-1-chars
ccl-vector "")
(format "%c%c" (make-char 'control-1 0)
(make-char 'control-1 31)))
nil
"CCL's write-multibyte-character breaks for control-1! "))
This works okay, however:
Index: src/mule-ccl.c
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/mule-ccl.c,v
retrieving revision 1.30
diff -u -u -r1.30 mule-ccl.c
--- src/mule-ccl.c 2006/06/14 06:10:10 1.30
+++ src/mule-ccl.c 2006/11/12 22:49:27
＠＠ -1390,7 +1390,7 ＠＠
if (i == LEADING_BYTE_ASCII)
i = reg[rrr] & 0xFF;
else if (LEADING_BYTE_CONTROL_1 == i)
- i = ((reg[rrr] & 0xFF) - 0xA0);
+ i = ((reg[rrr] & 0x1F) + 0x80);
else if (POSSIBLE_LEADING_BYTE_P(i) &&
!NILP(charset_by_leading_byte(i)))
{
There is a separate but similar problem with mule-to-unicode that I’m still
looking into.
--
Santa Maradona, priez pour moi!
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
5:18 p.m.
New subject: decoders writing control-1 chars?
debug build of 21.5
The patch was for 21.4, the branch the data mangling was reported for.
Never tried to merge it in 21.5. Neither have ever investigated the
behavior of ccl `write-multibyte-character' in 21.5, to decide that it
ever needs patch for this feature in the first place.
And in neither branch never called `ccl-execute-on-string'. Always
investigated behavior of decoding in ccl coding systems directly. If
in some branch some ccl operations do meaningful things inside ccl
coding system and crash when done by `ccl-execute-on-string',
considering it a bug in ccl implementation in that branch, and never
have hit it before, let alone tried to fix.
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
11:55 a.m.
New subject: decoders writing control-1 chars?
Ar an triú lá déag de mí na Samhain, scríobh Aidan Kehoe:
There is a separate but similar problem with mule-to-unicode that I’m
still
looking into.
Okay, the below eliminates both. I intend committing this evening.
src/ChangeLog addition:
2006-11-20 Aidan Kehoe <kehoea(a)parhasard.net>
* mule-ccl.c (CCL_MAKE_CHAR):
* mule-ccl.c (ccl_driver):
Eliminate a CCL bug with control-1 chars and
write-multibyte-character--thank you for the report, Ilya, and
eliminate a crash when a non-existent charset ID is specified for
a mule-to-unicode call.
tests/ChangeLog addition:
2006-11-20 Aidan Kehoe <kehoea(a)parhasard.net>
* automated/mule-tests.el:
Add tests to make sure the fixes to the CCL bugs I just checked in
don't regress.
XEmacs Trunk source patch:
Diff command: cvs -q diff -u
Files affected: tests/automated/mule-tests.el
===================================================================
RCS src/mule-ccl.c
===================================================================
RCS
Index: src/mule-ccl.c
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/mule-ccl.c,v
retrieving revision 1.30
diff -u -r1.30 mule-ccl.c
--- src/mule-ccl.c 2006/06/14 06:10:10 1.30
+++ src/mule-ccl.c 2006/11/20 11:47:01
＠＠ -850,13 +850,17 ＠＠
macro is only used in the MuleToUnicode transformation. */
#define CCL_MAKE_CHAR(charset, code, c) \
do { \
+ \
+ if (!POSSIBLE_LEADING_BYTE_P(charset)) \
+ CCL_INVALID_CMD; \
+ \
if ((charset) == LEADING_BYTE_ASCII) \
{ \
c = (code) & 0xFF; \
} \
else if ((charset) == LEADING_BYTE_CONTROL_1) \
{ \
- c = ((code) & 0xFF) - 0xA0; \
+ c = ((code) & 0x1F) + 0x80; \
} \
else if (!NILP(charset_by_leading_byte(charset)) \
&& ((code) >= 32) \
＠＠ -1390,7 +1394,7 ＠＠
if (i == LEADING_BYTE_ASCII)
i = reg[rrr] & 0xFF;
else if (LEADING_BYTE_CONTROL_1 == i)
- i = ((reg[rrr] & 0xFF) - 0xA0);
+ i = ((reg[rrr] & 0x1F) + 0x80);
else if (POSSIBLE_LEADING_BYTE_P(i) &&
!NILP(charset_by_leading_byte(i)))
{
Index: tests/automated/mule-tests.el
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/tests/automated/mule-tests.el,v
retrieving revision 1.10
diff -u -r1.10 mule-tests.el
--- tests/automated/mule-tests.el 2006/06/24 14:30:39 1.10
+++ tests/automated/mule-tests.el 2006/11/20 11:47:02
＠＠ -400,6 +400,48 ＠＠
(encode-coding-string xemacs-character 'ctext))))))
;;---------------------------------------------------------------
+ ;; Regression test for a couple of CCL-related bugs.
+ ;;---------------------------------------------------------------
+
+ (let ((ccl-vector [0 0 0 0 0 0 0 0 0]))
+ (define-ccl-program ccl-write-two-control-1-chars
+ `(1
+ ((r0 = ,(charset-id 'control-1))
+ (r1 = 0)
+ (write-multibyte-character r0 r1)
+ (r1 = 31)
+ (write-multibyte-character r0 r1)))
+ "CCL program that writes two control-1 multibyte characters.")
+
+ (Assert (equal
+ (ccl-execute-on-string 'ccl-write-two-control-1-chars
+ ccl-vector "")
+ (format "%c%c" (make-char 'control-1 0)
+ (make-char 'control-1 31))))
+
+ (define-ccl-program ccl-unicode-two-control-1-chars
+ `(1
+ ((r0 = ,(charset-id 'control-1))
+ (r1 = 31)
+ (mule-to-unicode r0 r1)
+ (r4 = r0)
+ (r3 = ,(charset-id 'control-1))
+ (r2 = 0)
+ (mule-to-unicode r3 r2)))
+ "CCL program that writes two control-1 UCS code points in r3 and r4")
+
+ ;; Re-initialise the vector, mainly to clear the instruction counter,
+ ;; which is its last element.
+ (setq ccl-vector [0 0 0 0 0 0 0 0 0])
+
+ (ccl-execute-on-string 'ccl-unicode-two-control-1-chars ccl-vector "")
+
+ (Assert (and (eq (aref ccl-vector 3)
+ (encode-char (make-char 'control-1 0) 'ucs))
+ (eq (aref ccl-vector 4)
+ (encode-char (make-char 'control-1 31) 'ucs)))))
+
+ ;;---------------------------------------------------------------
;; Test charset-in-* functions
;;---------------------------------------------------------------
(with-temp-buffer
--
Santa Maradona, priez pour moi!
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
7:24 p.m.
New subject: decoders writing control-1 chars?
APPROVE COMMIT
NOTE: This patch has been committed.
Ar an fichiú lá de mí na Samhain, scríobh Aidan Kehoe:
[...]
--
Santa Maradona, priez pour moi!
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
6606
days inactive
6618
days old
4 comments
2 participants
participants (2)
-
Aidan Kehoe -
Ilya N. Golubev