9:31 a.m.
Ar an chéad lá is fiche de mí Meitheamh, scríobh Stephen J. Turnbull:
>>>>> "Aidan" == Aidan Kehoe
<kehoea(a)parhasard.net> writes:
Aidan> The test case I gave didn’t involve Gnus, just
Aidan> latin-unity, which is why I asked you in particular.
Ah, sorry! What's even more sorry is I can't reproduce. :-( At least
if I understand you correctly; the designed result is to end up with a
buffer containing a bunch of ISO 646-IRV characters, with U+00F6,
U+0192, and U+5357 mixed in, and the form returns nil, right? That's
what I get.
Well, that’s encouraging, it works for someone :-) .
Japanese language environment (probably not directly relevant).
Coding system of buffer 5wDxDXtyU is Mule/ISO7. I would think this
doesn't matter since the internal encoding is a UCS, but.... Maybe
you can update your test case to set an appropriate
buffer-file-coding-system to force the problem?
To reproduce it here, XEmacs 21.5-b27 "fiddleheads" (+CVS-20060521)
configured for `i686-pc-linux'.,
1. Download the latest latin-euro-standards and latin-unity packages,
extract them to /tmp/aidan/
2. LANG=C xemacs -vanilla
3. Evaluate the following;
(push "/tmp/aidan/lisp/latin-euro-standards/" load-path)
(push "/tmp/aidan/lisp/latin-unity/" load-path)
4. Evaluate the test case;
(let* ((temp-buffer (generate-new-buffer "5wDxDXtyU"))
begin end csets psets)
(require 'latin-unity)
(set-buffer temp-buffer)
(insert (decode-coding-string "
> [...] m\xc3\xb6glich
\xc6\x92 \xe5\x8d\x97" 'utf-8))
(setq begin (point-min)
end (point-max)
csets (latin-unity-representations-feasible-region begin end)
psets (latin-unity-representations-present-region begin end))
(latin-unity-maybe-remap begin end 'iso-8859-1 csets psets))
That gives me a long list of warnings on “Obsolete key binding technique”,
and a result of iso-8859-1. It shouldn’t give me that result, because U+0192
cannot be, and was not, remapped to ISO 8859/1, and ditto for U+5357. The
coding system of the buffer is binary, but that shouldn’t be relevant, as I
understand it.
--
Santa Maradona, priez pour moi!
10:24 a.m.
New subject: setup_ccl_program in ccl_init_coding_stream
>>>> "Aidan" == Aidan Kehoe
<kehoea(a)parhasard.net> writes:
Aidan> To reproduce it here, XEmacs 21.5-b27 "fiddleheads"
Aidan> (+CVS-20060521) configured for `i686-pc-linux'.,
OK, I reproduced and I figured out what's happening. I suspect that
latin-unity is being stressed beyond its limits here. If you look at
the commentary on the related functions, you can see that I was
heading in the direction of getting them out of the coding-system
business as much as possible. Instead, I'd just unify (or
de-unificate, in the case of Han sets) *charsets*, and let some other
facility handle choosing an appropriate coding system. Ie, not caring
whether non-Latin charsets were present or not was more or less by
design.
Unfortunately, I don't recall how far that program proceeded (I didn't
have a clear idea at the time in any case). Whatever we do (or don't)
here is going to carry some risk. :-(
If you have any opinion, Aidan, I'd especially like to hear it, since
you've worked on the code a little bit, maybe.
The technical problem is that non-Latin charsets were inconceivable to
latin-unity. The following patch adds a signal that tells latin-unity
"whoa, hands off!" Unfortunately, this means that you can't remap a
region that contains non-Latin characters *at all*; you need to
restrict attention to Latin-only regions.
The recoding functions may need similar treatment, I haven't looked yet.
I'm almost sure this will cause annoyance to somebody, but most such
somebodies *probably* can just turn off latin-unity.
BTW, I apologize for the code ... that stuff is way too tricky. Well,
maybe not; it was gawdawful slow when I first coded it. I need to go
back and comment some of it, though.
Index: ChangeLog
===================================================================
RCS file: /pack/xemacscvs/XEmacs/packages/mule-packages/latin-unity/ChangeLog,v
retrieving revision 1.45
diff -u -r1.45 ChangeLog
--- ChangeLog 3 May 2006 12:04:44 -0000 1.45
+++ ChangeLog 22 Jun 2006 09:17:03 -0000
＠＠ -0,0 +1,8 ＠＠
+2006-06-22 Stephen J. Turnbull <stephen(a)xemacs.org>
+
+ * latin-unity-vars.el (latin-unity-non-latin-bit-flag): New constant.
+
+ * latin-unity.el (latin-unity-representations-present-region): Use
+ it to kludge around bug reported in
+ <m364iwq4yq.fsf(a)jerrypc.cs.usu.edu>.
+
Index: latin-unity-vars.el
===================================================================
RCS file: /pack/xemacscvs/XEmacs/packages/mule-packages/latin-unity/latin-unity-vars.el,v
retrieving revision 1.6
diff -u -r1.6 latin-unity-vars.el
--- latin-unity-vars.el 15 Feb 2005 22:22:48 -0000 1.6
+++ latin-unity-vars.el 22 Jun 2006 09:17:03 -0000
＠＠ -150,14 +150,17 ＠＠
"Bit vector representing the set of all Latin character sets.")
;; put the character set indicies and flag bits in reasonable places
-(let ((index 1) (bit 1))
- (if (> (length latin-unity-character-sets) 25)
- (error "representation too small to support so many charsets!"))
- (mapcar (lambda (cs)
- (put cs 'latin-unity-flag-bit bit)
- (put cs 'latin-unity-index index)
- (setq bit (lsh bit 1)
- index (1+ index)))
- latin-unity-character-sets))
+(defconst latin-unity-non-latin-bit-flag
+ (let ((index 1) (bit 1))
+ (if (> (length latin-unity-character-sets) 25)
+ (error "representation too small to support so many charsets!"))
+ (mapcar (lambda (cs)
+ (put cs 'latin-unity-flag-bit bit)
+ (put cs 'latin-unity-index index)
+ (setq bit (lsh bit 1)
+ index (1+ index)))
+ latin-unity-character-sets)
+ bit)
+ "A bit-flag indicating charsets not handled by latin-unity.")
;;; end of latin-unity-vars.
Index: latin-unity.el
===================================================================
RCS file: /pack/xemacscvs/XEmacs/packages/mule-packages/latin-unity/latin-unity.el,v
retrieving revision 1.14
diff -u -r1.14 latin-unity.el
--- latin-unity.el 2 May 2006 21:58:12 -0000 1.14
+++ latin-unity.el 22 Jun 2006 09:17:03 -0000
＠＠ -469,21 +469,23 ＠＠
(goto-char (point-min))
(while (not (eobp))
(let* ((ch (char-after))
- (cs (car (split-char ch))))
+ (cs (car (split-char ch)))
+ (flag (get cs 'latin-unity-flag-bit 0)))
(cond
((eq cs 'ascii)
(setq skipchars (concat "\000-\177" skipchars))
- (setq asets (logior (get cs 'latin-unity-flag-bit 0) asets)))
+ (setq asets (logior flag asets)))
((eq cs 'latin-jisx0201)
;; #### get this someday
;;(setq skipchars (concat skipchars latin-unity-latin-jisx0201))
(setq skipchars (concat skipchars (list ch)))
- (setq asets (logior (get cs 'latin-unity-flag-bit 0) asets)))
+ (setq asets (logior flag asets)))
(t
;; #### actually we can do the whole charset here
;; precompute and set a property on the cs symbol
(setq skipchars (concat skipchars (list ch)))
- (setq lsets (logior (get cs 'latin-unity-flag-bit 0) lsets)))))
+ (when (= flag 0) (setq lsets (logior latin-unity-non-latin-bit-flag lsets)))
+ (setq lsets (logior flag lsets)))))
;; The characters skipped here can't change asciisets
(skip-chars-forward skipchars))))
(cons lsets asets)))
--
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
1:39 p.m.
New subject: setup_ccl_program in ccl_init_coding_stream
Ar an dara lá is fiche de mí Meitheamh, scríobh Stephen J. Turnbull:
If you look at the commentary on the related functions, you can see
that
I was heading in the direction of getting them out of the coding-system
business as much as possible. Instead, I'd just unify (or de-unificate,
in the case of Han sets) *charsets*, and let some other facility handle
choosing an appropriate coding system. Ie, not caring whether non-Latin
charsets were present or not was more or less by design.
Well, “less,” because that wasn’t what you said in the docstrings :-) . Your
commentary is also full of things like:
;; #### I see nothing useful to be done with APPEND.
;; FILENAME, VISIT, or LOCKNAME could be used to default the coding system,
;; but this would conflict with the semantics of `write-region'.
;; #### The efficiency of this function can clearly be improved.
;; #### This function should be set up to call the check functions in a
;; condition-case, and call out to error handlers. Then tests could be
;; written more easily.
which is only readable in the event that you’re modifying the package and
have looked at the details of the implementation, rather than just the
interfaces. I didn’t read the commentary in detail as a result, since I was
interested in using the package.
Unfortunately, I don't recall how far that program proceeded (I
didn't
The technical problem is that non-Latin charsets were inconceivable to
latin-unity. The following patch adds a signal that tells latin-unity
"whoa, hands off!" Unfortunately, this means that you can't remap a
region that contains non-Latin characters *at all*; you need to
restrict attention to Latin-only regions.
That’s fine, I think; previously, remapping a region with non-Latin
characters would do something other than the docstring would lead you to
expect. And one can restrict attention to Latin-only regions with a
combination of skip-chars-forward, char tables, and the
latin-unity-character-sets variable.
The recoding functions may need similar treatment, I haven't
looked yet.
I'm almost sure this will cause annoyance to somebody, but most such
somebodies *probably* can just turn off latin-unity.
Hmm, I actually doubt it, but we’ll see.
--
Santa Maradona, priez pour moi!
7:18 p.m.
New subject: setup_ccl_program in ccl_init_coding_stream
>>>> "Aidan" == Aidan Kehoe
<kehoea(a)parhasard.net> writes:
Aidan> Well, “less,” because that wasn’t what you said in the
Aidan> docstrings :-) . Your commentary is also full of things
Aidan> like:
It's always acceptable to fix a bug by changing the documentation to
match the behavior. "More."
Aidan> I didn’t read the commentary in detail as a result, since
Aidan> I was interested in using the package.
Bad idea with half-baked code. The fact that it has worked with very
few complaints for as long as it has suggests I'm a much better
programmer than I think I am. :-)
--
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
9:21 p.m.
New subject: setup_ccl_program in ccl_init_coding_stream
Ar an tríú lá is fiche de mí Meitheamh, scríobh Stephen J. Turnbull:
Aidan> Well, “less,” because that wasn’t what you said in the
Aidan> docstrings :-) . Your commentary is also full of things like:
It's always acceptable to fix a bug by changing the documentation to
match the behavior. "More."
Anyway. Your patch you fixes the behaviour that meant that the code I wrote
(based on those functions as documented, to which there was and is no
reasonable alternative) for Gnus didn’t work. Thank you!
--
Santa Maradona, priez pour moi!
2:37 p.m.
New subject: setup_ccl_program in ccl_init_coding_stream
>>>> "Aidan" == Aidan Kehoe
<kehoea(a)parhasard.net> writes:
Aidan> Anyway. Your patch you fixes the behaviour that meant that
Aidan> the code I wrote (based on those functions as documented,
Aidan> to which there was and is no reasonable alternative) for
Aidan> Gnus didn’t work. Thank you!
You're welcome. I'll work up a formal patch and commit it within a
couple of days, unless you or Norbert sees a reason not to.
--
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
6513
days inactive
6516
days old
5 comments
2 participants
participants (2)
-
Aidan Kehoe -
Stephen J. Turnbull