Re: Characters lost when saving UTF-8 file on win32 (21.5.16)

Vladimir> I'm a bit confused now - I was under impression that Vladimir> xemacs 21.5 uses Unicode internally. Not yet. Here's the plan: 0. Current Mule (ie, 21.4) has an explicit component of each character to indicate the charset, which is trivially translated to an X11 registry, and used to look up the font. There is no internal concept of Unicode. 1. Add functions to parse Unicode Consortium translation tables, and convert to internal tables. Provide unicode coding-systems which use these tables to translate to Mule code internally. Provide unicode<->Mule code functions for individual characters (unicodes are not of character type, they are integers). 2. Change internal code to Unicode. Provide unicode->X11 registry mapping tables for redisplay on X11. Maybe swap representations of integers and characters (currently integers are 31-bit, characters are 30-bit with several bits unused). This swap is trivial, but was vetoed 3 years ago on the grounds that we didn't support Unicode, but people do edit >1GB files (really!) Currently we have completed stage 1. The problem of not corrupting data is independent of the Unicode transition, and we *absolutely* have to do something about that soon (and I hope to get it backported to 21.4, too---it's a historic problem with Mule, nothing new). Vladimir> The characters that do get whacked are from different Vladimir> parts of Unicode - circled Kanas, The circled kana are corporate characters not part of JIS X 0201, 0208, or 0212, thus will not be supported by existing Mule charsets. There is no Devanagari charset in XEmacs. I would guess that the missing Hangul are similar to the circled kana, not part of the relevant standard. (I suspect that XEmacs Mule follows the JIS X 0208.1990, JIS X 0212.1990, and KSC 5601.1991 standards). Vladimir> The problem of a good Unicode editor is very much Vladimir> present. It would be great if xemacs could fill that Vladimir> space :) That's where I'd like to go! But unless we get more people who are willing to work on XEmacs in some way[1], it's going to take a fair amount of time. Footnotes: [1] Doesn't have to be directly on that---it's possible to "trade" among developers, or reduce the bug fixing burden on core developers, etc, and so allow them to work on the features you want. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.