9:09 p.m.
Salam,
[this proposal is a rethought and refined version of the "RFC:
(set-xkb-cyrillic-charset "koi8-r") for non-Mule XEmacs". Thanks to
everyone for their comments, especially to Hrvoje for
'ascii-character, and to Martin for ideas about not bothering with C
and the need of "many more thinking", you were right :) ]
I will speak of "single-byte encodings", using mostly Cyrillic ones as
an example.
It seems like 'ascii-character property could be renamed to
'single-byte-equivalent for clarity and formal correctness, if it
wasn't for the compatibility (although it could be considered to
declare 'ascii-character obsolete and consult it only if
'single-byte-equivalent is not defined. Renamed property should be
properly documented, as everything else discussed here, of course).
There are three primary things that should be modified when switching
single-byte encoding: mapping of X keysyms to single-byte codes;
case-tables; syntax-tables.
1. Mapping of keysyms was the primary subject of previous discussion
and the best practice seems to be
(put 'Keysym 'single-byte-equivalent ?\xXX)
All possible keysyms that are subject to this mapping are tracked and
bound to nil if they do not have an equivalent in the encoding that is
being switched to. E.g., when we are using "latin-1" encoding, we
have
(global-set-key 'ntilde 'self-insert-command)
(put 'ntilde 'single-byte-equivalent ?\xXX)
When we're switching to "koi8-r" encoding, we have to
(global-unset-key 'ntilde)
because it does not have koi8-r equivalent. (Or, otherwise, we could
bind it to some `no-single-byte-equivalent' function, that does just
(message "last typed character has no CURRENT-ENCODING equivalent")
(instead of "ASCII", which is not always true).
Sure, if user redefined 'ntilde to something like (insert "\~n"), it
will be left alone.
2. case-tables. It's a rather minor and straightforward issue, except
that it effectively does not work in 21.1.x. Is the change in 21.4.x
that enabled this feature small enough to be somehow backported to
21.1? It'd be good to be able to make a patch that brings it all
together to one more release of 21.1.
3. syntax-tables (mostly for "word constituent" syntax class). That's
as minor and straightforward issue as the previous one, and without
bugs. It just needs to be carefully checked for various major modes
that change syntax tables so that they all live together.
I propose that generic implementation of all this be put in
"single-byte.el", while particular encoding descriptions be put in
e.g. "cyrillic.el" and "x-cyrillic.el" (different from
"mule/cyrillic.el"), a-la "iso8859-1.el" and
"x-iso8859-1.el".
Previous implementations: koi8-r-only support for all of this based on
results of previous discussion in xemacs-beta is written by "Andrew
W. Nosenko" <awn(a)bcs.zp.ua> and seems to be successfully used around.
There is also a widely used .el-files by Ilya Perminov to support most
popular Cyrillic encodings: it also features automatic detection of
encoding and recoding of buffers.
There are also several more issues, like choosing the correct ispell
dictionary, but they probably could be done in hooks or just discussed
later.
Any comments? I mostly do not see any unresolved issues and plan to
start developing in couple more days.
--alexm
9:41 p.m.
Alexey Mahotkin <alexm(a)hsys.msk.ru> writes:
It seems like 'ascii-character property could be renamed to
'single-byte-equivalent for clarity and formal correctness,
That cannot by right because the value of that property is a
character, not a byte. For instance, under Mule, it works perfectly
for Japanese characters.
The name `ascii-character' sucks for the same reason -- it's not
necessarily ASCII. Historically it was ASCII, and that property was
used to communicate that TAB should insert ?\t, etc.
1. Mapping of keysyms was the primary subject of previous discussion
and the best practice seems to be
[...]
I take it that you would like to automatize these mappings according
to a new concept of "single-byte encodings", present in the absence of
Mule?
The one thing that bugs me about this is that it's introducing yet
another abstraction ("single-byte"), and one which has no equivalent
in Mule whatsoever. That's IMHO a very bad thing, which will
undoubtedly lead to maintenance and documentation problems. ("I
compiled XEmacs with Mule, and my single-byte stuff no longer works!")
Now that Mule has been hacked into something close to usability, I
must admit I doubt that what you propose it worth the maintenance
pain. Can you remind me why exactly Mule fails to work for you?
Also, all of this sounds like an idea for a first-class external
package. Write it and distribute it to all your friends. If it
becomes immensely popular and all of Russia starts using it, we'll
include it too. It's as simple as that.
2. case-tables. It's a rather minor and straightforward issue,
except
that it effectively does not work in 21.1.x. Is the change in 21.4.x
that enabled this feature small enough to be somehow backported to
21.1?
It's not, and even if it were, such a patch would be totally
inappropriate for 21.1. If you want the new features, switch to
21.4. That's why it's there!
3:17 a.m.
>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
Hrvoje> Alexey Mahotkin <alexm(a)hsys.msk.ru> writes:
> It seems like 'ascii-character property could be renamed to
> 'single-byte-equivalent for clarity and formal correctness,
Hrvoje> That cannot by right because the value of that property is a
Hrvoje> character, not a byte. For instance, under Mule, it works perfectly
Hrvoje> for Japanese characters.
Hrvoje> The name `ascii-character' sucks for the same reason -- it's not
Hrvoje> necessarily ASCII. Historically it was ASCII, and that property was
Hrvoje> used to communicate that TAB should insert ?\t, etc.
Agreed.
Hrvoje> The one thing that bugs me about this is that it's introducing yet
Hrvoje> another abstraction ("single-byte"), and one which has no equivalent
Hrvoje> in Mule whatsoever. That's IMHO a very bad thing, which will
Hrvoje> undoubtedly lead to maintenance and documentation problems. ("I
Hrvoje> compiled XEmacs with Mule, and my single-byte stuff no longer works!")
Agreed.
Hrvoje> Now that Mule has been hacked into something close to usability, I
Hrvoje> must admit I doubt that what you propose it worth the maintenance
Hrvoje> pain. Can you remind me why exactly Mule fails to work for you?
Agreeed.
Hrvoje> Also, all of this sounds like an idea for a first-class external
Hrvoje> package. Write it and distribute it to all your friends. If it
Hrvoje> becomes immensely popular and all of Russia starts using it, we'll
Hrvoje> include it too. It's as simple as that.
Alexey: In any case, it is not obvious exactly what the user will
want. Perhaps users will want to use koi8 in one buffer,
alternativnij in another. Then your `global-set-key's need to change
to `local-set-key'. (Unfortunately, the `ascii-character' property is
a global thing, which would tend to hinder implementing this kind of
functionality.) This again suggests that everything should be
implemented in elisp.
3:49 a.m.
>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
Hrvoje> Alexey Mahotkin <alexm(a)hsys.msk.ru> writes:
> It seems like 'ascii-character property could be renamed to
> 'single-byte-equivalent for clarity and formal correctness,
Hrvoje> That cannot by right because the value of that property is
Hrvoje> a character, not a byte. For instance, under Mule, it
Hrvoje> works perfectly for Japanese characters.
How about 'buffer-representation? Yes, the elements of a buffer are
always characters, but using "character" drags a lot of stuff (coded
character sets, in particular) with it.
> 1. Mapping of keysyms was the primary subject of previous
> discussion and the best practice seems to be
Hrvoje> [...]
Hrvoje> I take it that you would like to automatize these mappings
Hrvoje> according to a new concept of "single-byte encodings",
Hrvoje> present in the absence of Mule?
[...]
Hrvoje> Also, all of this sounds like an idea for a first-class
Hrvoje> external package. Write it and distribute it to all your
Hrvoje> friends. If it becomes immensely popular and all of
Hrvoje> Russia starts using it, we'll include it too. It's as
Hrvoje> simple as that.
I agree with this. I think that probably you can and should do all of
this with a package. The package could work with Mule by simply
forcing all file I/O to 'binary, I bet. If the package fails in some
boundary cases (which I think is highly likely, for the reasons Hrvoje
gives among others), well, that's the way packages (don't) work. The
core has to do better.
Except for one thing. There's no reason to wait for your package to
become immensely popular; we can include it immediately. :-)
The various changes that you are talking about are surely not going to
get in to 21.1 any other way, and not 21.4, either. They are not bug
fixes to implementation, they are design changes at least, and
arguably new features.
--
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091
_________________ _________________ _________________ _________________
What are those straight lines for? "XEmacs rules."
4:03 p.m.
"Stephen J. Turnbull" <turnbull(a)sk.tsukuba.ac.jp> writes:
How about 'buffer-representation? Yes, the elements of a buffer
are
always characters, but using "character" drags a lot of stuff (coded
character sets, in particular) with it.
The term "character" is pretty well defined in XEmacs, so
`character-representation' would probably do it.
buffer-representation sucks because characters are also applicable to
strings, streams, etc.
I agree with this. I think that probably you can and should do all
of
this with a package. The package could work with Mule by simply
forcing all file I/O to 'binary, I bet.
Or, if the package is general enough, it could simply turn on the
appropriate Mule magic behind the scenes in order to produce the same
effect as the same commands had under no-Mule.
Except for one thing. There's no reason to wait for your package
to
become immensely popular; we can include it immediately. :-)
I would prefer to wait until a degree of maturity, if not popularity,
is reached.
5:04 a.m.
>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
Hrvoje> "Stephen J. Turnbull" <turnbull(a)sk.tsukuba.ac.jp> writes:
> How about 'buffer-representation?
Hrvoje> The term "character" is pretty well defined in XEmacs, so
Hrvoje> `character-representation' would probably do it.
Much better, thanks! Who's the native speaker, here, anyway?!
An alternative would be `text-representation', as Ben is planning on
abstracting common aspects of character, string, and buffer
representation into "src/text.[ch]."
Hrvoje> I would prefer to wait until a degree of maturity, if not
Hrvoje> popularity, is reached.
That didn't stop the disgusting hack[1] known as "package-get.el" from
becoming part of the core. :-(
Footnotes:
[1] I'm referring only to the utter dependence on EFS. There are
parts of the explicit code in the library that aren't to my taste, but
no way are they "disgusting."
--
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091
_________________ _________________ _________________ _________________
What are those straight lines for? "XEmacs rules."
8369
days inactive
8371
days old
5 comments
4 participants
participants (4)
-
Alexey Mahotkin -
Hrvoje Niksic -
Martin Buchholz -
Stephen J. Turnbull