Alexey Mahotkin <alexm(a)hsys.msk.ru&gt; writes: Does this work: (global-set-key 'Cyrillic_HardSIGN 'self-insert-command) (put 'Cyrillic_HardSIGN 'ascii-value ?\xff) If it works, it should also work with isearch and friends. The name might be misleading because: * AFAIK x_keysym_to_character is not xkb-specific. * It uses the term `charset' which *looks* like it refers to Mule, although the function has in fact nothing to do with Mule. * Other charsets might require similar hacks. How about: (x-set-keysym-translation 'cyrillic 'koi8-r) (x-set-keysym-translation 'latin-2 'windows-1250) ...

Hrvoje> Does this work: Hrvoje> (global-set-key 'Cyrillic_HardSIGN 'self-insert-command) Hrvoje> (put 'Cyrillic_HardSIGN 'ascii-value ?\xff) Hrvoje> If it works, it should also work with isearch and friends. No, it doesn't. 21.1 says last typed character has no ASCII equivalent: #<keypress-event Cyrillic_HARDSIGN> 21.4 ignores 'ascii-value and continues to use hardcoded 8859-5 chars. Hrvoje> The name might be misleading because: Hrvoje> * AFAIK x_keysym_to_character is not xkb-specific. I always thought that x11/keysymdef.h is defined only for xkb (IMHO, xmodmap is able only to specify numeric character codes and does not know about Cyrillic_XXX et al). Hrvoje> * It uses the term `charset' which *looks* like it refers to Hrvoje> Mule, although the function has in fact nothing to do with Hrvoje> Mule. Hrvoje> * Other charsets might require similar hacks. Hrvoje> How about: Hrvoje> (x-set-keysym-translation 'cyrillic 'koi8-r) Hrvoje> (x-set-keysym-translation 'latin-2 'windows-1250) ... I like style of this one, see below. SJT> Much better IMO, but as I understand it, conceptually there is SJT> only one keysym translation table. So how about the interface SJT> (x-set-keysym-translation 'Cyrillic_HARDSIGN ?\xff) SJT> (x-set-keysym-translation-subset 'cyrillic 'koi8-r) SJT> (setq koi8-r (x-get-keysym-translation-subset 'cyrillic)) SJT> etc? In this case first clause should be (it should modify cyrillic_koi8_r[] that is stored somewhere in some hashtable and used by x_keysym_to_character(): (x-set-keysym-translation 'koi8-r 'Cyrillic_HARDSIGN ?\xff) (x-set-keysym-translation 'iso8859-5 'Cyrillic_HARDSIGN ?\xca) (x-set-keysym-translation 'windows-1251 'Cyrillic_HARDSIGN ?\xda) (I just wanted to hardcode those tables into src/event-Xt.c, but do not object to implement modifying those tables). I think they should be predefined in stock XEmacs w/o any Lisp definitions (in style of cyrillic[] array that currently exists). Next we choose (x-set-keysym-translation-subset 'cyrillic 'koi8-r) I do not understand what you're trying to do with (setq koi8-r (x-get-keysym-translation-subset 'cyrillic)) I slightly feel like opening Pandora's box. :) --alexm

Hrvoje> I made two mistakes: I misnamed the property, and my `put' Hrvoje> line contains a case mismatch. It should be: Hrvoje> (global-set-key 'Cyrillic_HARDSIGN 'self-insert-command) Hrvoje> (put 'Cyrillic_HARDSIGN 'ascii-character ?\xff) Hrvoje> 21.1 should respect that. It does. isearch works. Thank you! Anyway it seems like we need to implement (x-set-keysym-translation/-subset) anyway, how do you think? Or just fix 21.4 (see below) and get done with that? Someone also suggested (global-set-key [(Cyrillic_HARDSIGN)] [Ъ]) It works too. (though isearch in gnus v5.6.45 does not work with "Keyboard macro terminated by a command ringing the bell" Hrvoje> If 21.4 doesn't, it's a bug. 21.4.1 doesn't. It keeps inserting ISO8859-5 code for HARDSIGN (0xca). I could test any patches for that matter. --alexm

mb> (for example, Alexey is working on adding another layer). ? His proposal seems to just make a layer that is already there (x_keysym_to_character) configurable. (BTW, despite my caution, it seems pretty straightforward at first glance.) Is that what you mean (another "layer of complexity") or am I missing something? mb> It should be possible to define a key as a character, e.g. mb> (define-key [(Cyrillic_hardsign)] ?�) I like this interface. mb> (define-key [(Cyrillic_hardsign)] 'self-insert-key) mb> (put 'Cyrillic_hardsign 'ascii-character ?�) This seems to work with extended (Mule) characters, too. (global-set-key [(f5)] 'self-insert-command) (put 'f5 'ascii-character ?字) F5 --| 字 -- University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091 _________________ _________________ _________________ _________________ What are those straight lines for? "XEmacs rules."

Hrvoje> "Stephen J. Turnbull" <turnbull(a)sk.tsukuba.ac.jp&gt; writes: mb> It should be possible to define a key as a character, e.g. mb> (define-key [(Cyrillic_hardsign)] ?ÿ) >> I like this interface. Hrvoje> I'm not so sure about it. What if ÿ itself is bound to something Hrvoje> other than self-insert-command? Do we still insert ÿ, or do we call Hrvoje> that other command? Historical note: keystrokes and characters have been confused, since they are somewhat indistinguishable on a tty. The default (fallback) binding of a character should be self-insert-key (except without the need to hack the ascii-character property). This should be overridable by the user, both to change the global fallback binding and to change a binding for a particular character. You need to be careful to make this work with isearch. Of course, lots more thinking should be done before anyone starts hacking on this.

Oh, Alexey, can't you do everything you need for koi8 without hacking the C code, by doing simply: ;;; koi8.el: (when (not (featurep 'mule)) (keyboard-translate 'Cyrillic_hardsign 'ydiaeresis) ... ) My apologies if this was already considered and rejected. It would be good if the following worked as expected. (keyboard-translate 'Cyrillic_hardsign ?ÿ) but this has different results from the above.

MB> Oh, Alexey, can't you do everything you need for koi8 without MB> hacking the C code, by doing simply: MB> ;;; koi8.el: (when (not (featurep 'mule)) (keyboard-translate MB> 'Cyrillic_hardsign 'ydiaeresis) ... ) A> I want to switch by whole arrays. A> First reason: there is already code `x_keysym_to_character' that works A> but just needs a clean well-defined hack to solve single simple A> problem (Stephen, I consider your concerns about overcyrillization). It would be easy to add some hack to allow the user to specify the translation from keysym to equivalent character in event-Xt.c. This can be implemented internally using a hash table. But I'm not all sure that doing this is a good idea, since it adds Yet Another Key Mapping Mechanism, when we already have so many. And I don't understand why the ones we have (specifically, function-key-map) don't do exactly what you want. You have to maintain a mapping table from keysyms to koi8 character codes in any case. Isn't it easier to have it in lisp code than C code? A> Second reason, more practical about why I want to deal with arrays: A> suppose we're setting cp1251 encoding for Cyrillic characters. We A> have A> (defun set-cyrillic-cp1251-encoding A> (set-keysym-translation 'Cyrillic_HARDSIGN ?\xDA) A> (set-keysym-translation 'Cyrillic_DZHE ?\0x8F)) A> After that we want to switch to koi8-r. We have: A> (defun set-cyrillic-koi8-r-encoding A> (set-keysym-translation 'Cyrillic_HARDSIGN ?\xFF) A> ;; because koi8-r does not have DZHE A> (set-keysym-translation 'Cyrillic_DZHE nil)) A> That is, we have to explicitly unset every keysym that could be A> supported by some other encoding and could not be supported by the A> encoding we're trying to set. This is very tedious IMHO and is better A> handled by A> char *cyrillic = cyrillic_koi8_r; I disagree about "tedious". The information about keysyms which should be mapped to nil has to be maintained in any case, so there is just as much code to maintain. And I don't think humans will notice the time taken by the computer to do the remappings. I disagree that we should have a specific C-level mechanism for koi8. The C code you are hacking in event-Xt.c is a fallback of last resort. It could actually have been implemented by simply populating global-map, but doing it in C the way we did was a small optimization of space and startup time, and reduces user confusion by not having global-map cluttered with key bindings for keys the user doesn't have.

On Thu, 10 May 2001 12:04:45 +0400, Alexey Mahotkin said: Huh? ;)

Hrvoje> I made two mistakes: I misnamed the property, and my `put' Hrvoje> line contains a case mismatch. It should be: Hrvoje> (global-set-key 'Cyrillic_HARDSIGN 'self-insert-command) Hrvoje> (put 'Cyrillic_HARDSIGN 'ascii-character ?\xff) By the way, there is no chapter "Character properties" in lispref.info (there is "String properties" chapter). Is there any other properties for characters (I could try to write up some summary documentation for that...)? --alexm

Hrvoje> I made two mistakes: I misnamed the property, and my `put' Hrvoje> line contains a case mismatch. It should be: Hrvoje> (global-set-key 'Cyrillic_HARDSIGN 'self-insert-command) Hrvoje> (put 'Cyrillic_HARDSIGN 'ascii-character ?\xff) A> By the way, there is no chapter "Character properties" in lispref.info A> (there is "String properties" chapter). Technically, character lisp objects themselves have no properties. In the above, it's a SYMBOL that has a property.

Hrvoje> * AFAIK x_keysym_to_character is not xkb-specific. A> I always thought that x11/keysymdef.h is defined only for xkb (IMHO, A> xmodmap is able only to specify numeric character codes and does not A> know about Cyrillic_XXX et al). keysymdef.h has been around for eons. I.e. before X11R6. Unfortunately, Russian has several possible encodings, mostly due to historical reasons I guess. Since XEmacs has coding systems, you would think if you simply told xemacs which encodings your system was using (or if it could guess) then everything should work. This is not quite true. Probably Koi8 is the dominant encoding in Russia, and it would be more useful for xemacs to conform to that. But most of the computers I have acccess to have iso-8859-5 fonts. Unfortunately, although it shouldn't matter what internal representation is used for characters internally, xemacs prefers to have the fonts it uses match the encoding in the internal representation. Alexey, I encourage you to try to be the Russian maintainer for XEmacs. Doing things properly might be hard, though. Perhaps Ben will Unicodeify XEmacs, and then we will need a general mechanism to map any characters to any fonts, and russian support should be simply a matter of writing the appropriate mapping tables. A> (I just wanted to hardcode those tables into src/event-Xt.c, but do A> not object to implement modifying those tables). I think they should A> be predefined in stock XEmacs w/o any Lisp definitions (in style of A> cyrillic[] array that currently exists). I wrote that code in event-Xt.c, but I did it for all character sets, without any special attention to Cyrillic. Or even any testing for Cyrillic. When designed properly, there should only be one internal representation for a given character - i.e. we should not have iso8859-5 and koi8 represented separately internally. This means that the event-Xt.c code should not know anything about koi8. A> I slightly feel like opening Pandora's box. :) Mule is hard. Good luck.

MB> keysymdef.h has been around for eons. I.e. before X11R6. A> By the way, while we're at it, probably you know: what is the proper A> function to get X keysym-code (0x6ff) from the string like A> "XK_Cyrillic_HARDSIGN"? If I'm going to implement A> (x-set-keysym-translation), I'll need that one.. I'm not sure what you want here. Check out XKeysymToString. Check out x-keysym-hash-table and x-keysym-on-keyboard-p and its implementation. (I wrote that many years ago) MB> Probably Koi8 is the dominant encoding in Russia, and it would be MB> more useful for xemacs to conform to that. But most of the MB> computers I have acccess to have iso-8859-5 fonts. A> If that was Solaris, then it's ok, because they know only of ISO I used to work at Sun, and can confirm Sun's zealous devotion to standards. For some things, like the C language, this was a good decision. But for other things, like `make' and iso8859-5, a reality check would be useful. In Sun's defence, iso8859-5 was the Soviet government's idea. I don't know if the Russian government has abandoned iso8859-5 officially, or recommended a new ISO standard for Cyrillic. I'm sure it's all very political.

SJT> That's what I wanted to avoid. OK, I see that it's probably SJT> unavoidable in no-mule. And it's probably right to do it inside SJT> that switch (this automatically makes it configurable according SJT> to character set, so that the iso-to-koi translation does NOT get SJT> applied in the Latin-2 buffer, which would really hinder a SJT> Polish/Russian translator). Oh oh oh. In large letters: I intend all those (x-set-keysym-translation/-subset) work _only_ in non-Mule XEmacsen. Those functions will not even exist when Mule is enabled. I am completely satisfied with how Mule does what it does. Maybe it'll clear someone's understanding of the situation. Rushing off home, so will answer (and read) to other several hours later. --alexm

Alexey> In this case first clause should be (it should modify Alexey> cyrillic_koi8_r[] that is stored somewhere in some hashtable Alexey> and used by x_keysym_to_character(): SJT> That's what I wanted to avoid. OK, I see that it's probably SJT> unavoidable in no-mule. And it's probably right to do it inside SJT> that switch (this automatically makes it configurable according SJT> to character set, SJT> so that the iso-to-koi translation does NOT get SJT> applied in the Latin-2 buffer, which would really hinder a SJT> Polish/Russian translator). Just for clarity: there is no "iso-to-koi" translation here and no concept of "Latin-2 buffer". I speak of non-Mule, and there is only one-to-one translation between internal representation (single byte), font encoding and external representation (bytes in files). It is very common kind of setup AFAIK. We just need to implement various translations between Cyrillic_XXX and those one-byte encodings. SJT> As far as I know, however, SJT> it's not Mule XEmacs that wants the to fonts match internal SJT> representation. It does not want. It just doesn't go into trouble of doing something like Mule's (set-charset-ccl-program). SJT> It's that XEmacs developers have never done SJT> anything about using the facilities provided by Mule to handle SJT> fonts that have different encoding from internal. Huh? What about ccl-programs? Alexey> I think they should be predefined in stock XEmacs w/o any Lisp Alexey> definitions (in style of cyrillic[] array that currently Alexey> exists). SJT> OK, I agree about the need for predefinition, especially in SJT> no-Mule. However, in practice, we don't know which fonts are SJT> important. There are two most important encodings to support for non-Mule: koi8-r and iso8859-5 (for compatibility with default). Then there is more seldom used cp1251 (for compatibility with Windows). Finally there is cp866 that needs to be implemented mostly for completeness. SJT> So there should be a Lisp-level way to create and SJT> install such arrays. If you hack only for Cyrillic, you will SJT> create a Russianized XEmacs. Why ever? Cyrillic != Russian. As far as encodings allow, I'll surely try to support every Cyrillic character. E. g. koi8-r just does not include NJE, KJE, DZHE, TSHE and friends, while cp1251 and 8859-5 does include those. SJT> And will you put in (eg) the extra SJT> characters the Ukrainians want? Ukrainians have some variance of koi8-r called something like koi8-u. I'm sure someone from Ukraine will implement this encoding when there'll be general support for that and Ukrainian users will do (x-set-keysym-translation-subset 'cyrillic 'koi8-u) (See interface definition below). Nothing prevents us from doing that. SJT> I disagree strongly about implementing such arrays for Mule, SJT> however. Mule will use only the same array it is using currently (it will be renamed to cyrillic_iso8859_5[] probably, to reduce confusion). SJT> As for predefinition in C vs Lisp: We try to think in terms of SJT> implementing _everything_ in the text editor in Lisp, for SJT> robustness and flexibility. We move down to the C level when we SJT> want the efficiency. We don't _need_ it here; users will never SJT> see the delay in installing such a table from Lisp rather than C, SJT> since it only happens once per session. We do _need_ the Lisp SJT> level definition, since it's probable that Greeks, at least, will SJT> need this too. And we don't know who else! We get robust SJT> predefinition from Lisp by "dumping" the Lisp into the SJT> executable, where we always know where to find it. I just want to switch whole arrays, not single keysyms. I think that interface will be something like that: ;; define new Cyrillic encoding (x-define-keysym-translation-subset 'cyrillic 'koi8-r) (x-set-keysym-translation 'koi8-r 'Cyrillic_HARDSIGN ?\xFF) ... repeat for every Cyrillic character ;; this is done by user interactively or in ~/.emacs (x-set-keysym-translation-subset 'cyrillic 'koi8-r) Most popular encodings will be predefined in src/event-Xt.c, but still modifiable via (x-set-keysym-translation), surely (though this will probably be used only on newly defined keysym translation subsets). If there are no objections or additional comments, I'll start coding that tomorrow. --alexm

Alexey> In this case first clause should be (it should modify Alexey> cyrillic_koi8_r[] that is stored somewhere in some hashtable Alexey> and used by x_keysym_to_character(): SJT> That's what I wanted to avoid. OK, I see that it's probably SJT> unavoidable in no-mule. And it's probably right to do it inside SJT> that switch (this automatically makes it configurable according SJT> to character set, SJT> so that the iso-to-koi translation does NOT get SJT> applied in the Latin-2 buffer, which would really hinder a SJT> Polish/Russian translator). A> Just for clarity: there is no "iso-to-koi" translation here and no A> concept of "Latin-2 buffer". I speak of non-Mule, and there is only A> one-to-one translation between internal representation (single byte), A> font encoding and external representation (bytes in files). It is A> very common kind of setup AFAIK. A> We just need to implement various translations between Cyrillic_XXX A> and those one-byte encodings. This model of computing is a very common one in the world of character sets smaller than 128. It's a very "low-tech" way of achieving a multi-lingual editor. The original developers of Multilingualization in Emacs targetted Japanese, for which the hack of using the 128-256 range of character codes doesn't work. So the low-tech way is not supported as well as it could be. But it's also so simple not much support is needed. You can have a function that sets the font for a default face for a buffer by using specifiers with (set-face-font 'default "*koi8" (current-buffer)) This allows you to implement the command (defun toggle-cyrillic-mode (&optional buffer) (interactive) ...) this technique, combined with the keyboard-translate or function-key-map mechanisms (and xmodmap, of course), allows you to have a complete russian (or ukranian, or hungarian, etc.) environment with very little lisp hacking. Mule does give you some extra benefits, but there's a price to pay.

MB> This model of computing is a very common one in the world of MB> character sets smaller than 128. MB> It's a very "low-tech" way of achieving a multi-lingual editor. A> That's exactly what we need. High-tech is already implemented and is A> called Mule :) MB> You can have a function that sets the font for a default face for MB> a buffer by using specifiers with MB> (set-face-font 'default "*koi8" (current-buffer)) A> That won't work because your buffer will have one-byte characters in A> 8859-5, and there is no way in non-Mule to have different font A> encoding and characters encoding. Moreover, your files will be in A> 8859-5 encoding. Moreover, you could not open koi8-r file and edit A> it. I mean that you would combine the set-face-font above with some method (explained in other messages) that would allows input to map keysyms to their koi8 code instead of their iso8859-5 code. A> Same shi^Wthing happens for some other languages I'm said (AFAIK, A> Czech have five commonly-used one-byte encodings, that's why Russian A> Apache is so popular there). Interesting. MB> This allows you to implement the command MB> (defun toggle-cyrillic-mode (&optional buffer) (interactive) ...) MB> this technique, combined with the keyboard-translate or MB> function-key-map mechanisms (and xmodmap, of course), allows you MB> to have a complete russian (or ukranian, or hungarian, etc.) MB> environment with very little lisp hacking. A> I just want to reuse code that's already there. Your code :) Although the key hacking code in C should be rewritten, there's enough functionality available to the lisp programmer that what you want can and should be implemented in lisp today. A .el file is much more shareable than a patch to the C code, that would require a potential user to rebuild their xemacs binary.

Alexey> I've already got some approval of this idea and Alexey> encouragement by some local XEmacs users. BTW, Alexey, the xemacs-mule(a)xemacs.org list is chartered to be multi-lingual. If for some reason it would be convenient, you and others are welcome to hold development-related discussions in Russian on that list. They need not be restricted specifically to Mule; any topic related to using XEmacs for non-English languages is welcome. I and other developers do monitor that list. I don't read Russian, but if a Russian thread should suddenly break into English, I would certainly consider that a request for my attention. This model has worked fairly well for Japanese (until now, the main non-Latin-X constituency for XEmacs). We could also create an xemacs-users-ru list to parallel c.e.x and xemacs-beta, but specifically chartered for Russian discussion. Again, this has worked for Japanese (although traffic has gone to zero on xemacs-users-ja recently). -- University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091 _________________ _________________ _________________ _________________ What are those straight lines for? "XEmacs rules."