SY> Let me or Adrian know if you have any problems with sending reports or SY> using the "build" package. Yes, I would like to hear about any gripes with build and build-report. SY> Footnotes: SY> [1] I should note here that I am not collecting any personal SY> information like names and email addresses etc. I guess that means you won't sell them either :-) I am willing to pay 100~ for them all. -- Adrian Aichner mailto:adrian＠xemacs.org http://www.xemacs.org/

|--==> "APA" == Adrian Aichner <adrian(a)xemacs.org&gt; writes: SY> [1] I should note here that I am not collecting any personal SY> information like names and email addresses etc. APA> I guess that means you won't sell them either :-) I don't remember saying /that/. :-P APA> I am willing to pay 100~ for them all. Well that all depends on how much 100 tildes actually is in $AUD. :-) BTW, exactly what is '100~'? -- |---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---| | XEmacs - The only _______ you'll ever need. | | Fill in the blank, yes, it's THAT good! | |------------------------------------&lt;youngs(a)xemacs.org&gt;---|

Aidan> "Around 100 dollars US," I would imagine :-) Surely a guy who can write Aidan> Ar an 25ú lá de mí 4, scríobh Steve Youngs : knows the value of €100 more accurately than that. ;-) (Now let's see if _my_ XEmacs is properly configured.) -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

|--==> "SJT" == Stephen J Turnbull <stephen(a)xemacs.org&gt; writes: SJT> knows the value of €100 more accurately than that. ;-) (Now SJT> let's see if _my_ XEmacs is properly configured.) Looks that way. (I wonder if I can charge another €100 for that answer...) -- |---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---| | XEmacs - The only _______ you'll ever need. | | Fill in the blank, yes, it's THAT good! | |------------------------------------&lt;youngs(a)xemacs.org&gt;---|

|--==> "APA" == Adrian Aichner <adrian(a)xemacs.org&gt; writes: Stephen> knows the value of ~100 more accurately than that. ;-) (Now let's Stephen> see if _my_ XEmacs is properly configured.) APA> I had APA> '(message-default-charset 'iso-8859-1) APA> in my c-s-v and just did a reset to standard setting. APA> Let's see if that change was worth more than 0~, I mean 0~. Nope. I'm seeing 0~ (zero tilde) instead of 0€ (zero Euro). And what's more, you've turned Steve's € into a ~ as well. -- |---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---| | XEmacs - The only _______ you'll ever need. | | Fill in the blank, yes, it's THAT good! | |------------------------------------&lt;youngs(a)xemacs.org&gt;---|

Jan> It did this to me once, when I finished editing a huge HTML Jan> file containing ISO-8859-2 characters, saved it and logged Jan> off. Much to my surprise, all ISO-8859-2 characters were Jan> replaced by tildes. Several hours of work gone. This still happens (in GNU Emacs as well) if you force the wrong coding system. It's much less likely to happen inadvertantly now, but still possible. Unfortunately, the intelligent strategy (default to Unicode for any multilingual document) was howled down by byte-pinching Europeans and cultural nationalists (primarily Japanese), and so today we have the embarrassing situation where there is no released Emacs (neither GNU nor XEmacs) that speaks Unicode natively. Jan> Now, surely there is an explanation -- I think I figured out Jan> why this happens, but I do not remember the exact reasons. If you force buffer-file-coding-system to ISO-8859-1 and insert Latin 2 characters into the buffer by hand, you will get this result. Technically speaking, Mule isn't doing any conversions. It's dropping characters that can't be represented in the coding system you (perhaps implicitly) requested. Jan> My XEmacs doesn't seem to do this now: it sometimes converts Jan> non-7bit characters I believe this is because XEmacs now defaults to preferring a coding system (iso-2022-7) that can encode all characters. See? You don't like the safe setup very much, do you? You want XEmacs to do what you want, even though the attempt to do so guaranteed to cause problems eventually. This is the problem we've always faced. People would like it to be safe, but even if they express this desire IN IMPERATIVE SYTLE, what they really insist on is having their minds read. Otherwise, they'd simply use Unicode. (Granted, Unicode alone will lose information for a few multilingual Asian users. However, that is not a problem for European languages.) Jan> into the MULE coding, I hope not; you should never ever see true Mule coding in a file. What you probably mean is iso-2022-7, which is trivial to convert: read it into a buffer with C-x C-f, then save it with C-u C-x C-w FILENAME RET TARGET-CODING RET. Of course if you choose an encoding that can't represent all the characters in the buffer, you'll lose data. Jan> If XEmacs still does this (loses data by doing a one-way Jan> conversion) for ANY REASON whatsoever, it has to be changed. Unfortunately, it cannot be, at least not yet. For ISO 8859 Latin users, things are made a lot more safe (not to mention convenient, in the sense of coalescing files that Emacs normally needs to save as ISO 2022 or Unicode into a single ISO 8859 set where possible) by use of the latin-unity package. However, it's still not perfect. If you really want safety, the answer is what it has always been: convert your working files to Unicode. (Or ISO 2022, but that's silly if Unicode is usable.) If they need to be distributed in some other encoding, use XEmacs or iconv to convert the distribution versions. Use Mule-UCS with existing Emacsen or the 21.5 branch of XEmacs or the emacs-unicode branch of GNU Emacs. Or you can use no-mule XEmacs and live with the one-charset-per-instance restriction. That's the only safe way to go; unfortunately that one charset can't be Unicode in a no-mule XEmacs. Jan> So, perhaps it is worth investigating why Adrian's XEmacs ate Jan> the euro. Gnus did it. ;-) Seriously, Gnus is far too smart for its own good about these things in some ways, and not smart enough in others. I'm willing to work to improve XEmacs's safety (viz latin-unity), but I'm not willing to worry about any problem that involves a hairball like Gnus. If somebody wants to trace through the Gnus code until they've made it back to XEmacs, I'll take a look at it then. But it's a better use of my time to just work directly on latin-unity and similar code. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Jan> Please notice that this is what people in 8859-2 countries Jan> get when they try XEmacs. No, in fact they don't. At least not if they have LANG set or use latin-unity. What is happening to you, I would guess, is that you have iso-8859-1 as default coding system. If you are using both that and iso-8859-2, you _will_ eventually lose data, even if you switch to vi or enable latin-unity. But iso-8859-1 is a very special case (for historical reasons). Jan> I therefore beg to differ with your assessment of the problem Jan> being related to "forcing" anything. Perhaps the problem lies Jan> in the 8859-1 coding being "forced" quietly, which I would Jan> consider a bug. Unfortunately, almost nobody in the LANG=*_*.ISO8859-1 locales agrees with you. And those in other locales feel equally strongly about forcing defaults for theirs (with the exception of the Cyrillics, but they want it forced, too, only to KOI8 instead of ISO 8859/5, the Mule default). Jan> Overall, the XEmacs experience is like walking through a Jan> minefield -- get file-coding-system-alist right for all names Jan> of files you'll be working with, or get your files quietly Jan> mashed to pieces behind your back. Or use latin-unity. (This doesn't apply to non-Latin users yet, but they mostly don't have these problems anyway.) Jan> paragraph. Except one: this procedure used to be broken (I've Jan> reported it on Jun 30 2002 in a message titled Jan> "set-buffer-file-coding-system doesn't") and it is still Jan> broken, having just checked: XEmacs 21.5? Quite possibly broken then and still broken now; Ben has been screwing with the coding stuff, and I only shifted to daily use of 21.5 when I passed 21.4 on to Vin. Let me check.... Nope, not broken for me. If I type in a few characters of Latin 2, save as ISO 7-bit, then read the file and save as ISO 8859/2, that's what I get in the file. 21.5.11, CVS 2003-04-22 (ie, just before the beta release of 21.5.12). M-x set-buffer-file-coding-system also works as I expect it to on my system. Maybe you're referring to the fact that if you set the b-f-c-s the buffer contents and display don't change? But that is correct behavior; the buffer and display representations are independent of b-f-c-s. b-f-c-s only affects the representation in the output file. If you want to change the characters in the buffer, you can either use the low-level APIs {en,de}code-coding-region or the more user-friendly UIs in latin-unity. Jan> new-filename.txt still contains 2022-7. Or am I doing Jan> something wrong? If the old file contained any Latin 1 characters, then they would continue to be coded using ISO 2022 escapes, at least through XEmacs 21.4; this is the old Mule safety mechanism. Only ASCII and Latin 2 characters would be encoded using ISO 8859/2. I believe this behavior is compatible with the definition of ISO 8859 (since ISO 8859 is a version of ISO 2022), but it's not what users expect. The reason this doesn't come into play in your data-loss case is due to historical lossage where iso-8859-1 and binary are considered identical. (I.e., XEmacs does not consider iso-8859-1 to be an ISO 8-bit coding system, in the implementation. I'm not sure if GNU Emacs has fixed this silliness, but we're still stuck with it.) Note that if instead of the POSIX or .iso8859-1 locale you use a .iso8859-15 locale, you'll get the "safe but confusing" behavior for Western European text with a few Central European characters. Obviously we don't want to change this until latin-unity is both considered reliable (I have yet to hear complaints of data loss or inconvenience even from latin-unity users, so that's probably OK) and has been ported to 21.5. But at that point we will remove the ISO 2022 escape stuff from the ISO 8859 coding systems. (Experimentally; if there's demand for GNU compatibility, we will implement the bogus-coding-with-esc coding systems. But my guess is that nobody except Japanese Mule programmers ever found those useful.) Jan> Can we at least make XEmacs less shy about reporting that Jan> it's about to lose data? In 21.4, put (latin-unity-install) in your init file. This not only will query before writing to disk if there's incompatible data in the buffer, but will also recommend a compatible ISO 8859 coding system (if available) as well as universal coding systems like utf-8 or iso-2022-7. In 21.5, I don't know offhand. In my init file, I currently have (unless (emacs-version>= 21.5) (latin-unity-install)), but I don't recall why. My intention is to integrate the code into 21.5. The problem is that although I'm reasonably happy with the UI (it gives you exactly the info you want AFAICT, mostly only when you really want it, and is fairly configurable to automatically default your personal usage cases reasonably), the API still sucks. So I'm not happy about mandating it for core yet. I guess a reasonable first step is to fix whatever caused me to disable it in my init file for 21.5, then release a new latin-unity package compatible with 21.5. I'll shoot for next weekend, and if not, I'll followup to let you know why I couldn't do it. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Jan> LANG! That is enlightening -- I somehow never thought XEmacs Jan> would use the LANG setting to enforce coding systems for Jan> files. In fact, I do have LANG set to en_US on this Jan> machine. I was somehow convinced that XEmacs tried to stay Jan> away from locales as much as possible. I know of only two places where it is used. In initialization of XEmacs, it is used as a default for set-language-environment, which sets the default buffer-file-coding-system, among others. Otherwise, the default coding system is iso8859-1. The other is to determine the apropriate X Input Method. Stephen> Or use latin-unity. (This doesn't apply to non-Latin Stephen> users yet, but they mostly don't have these problems Stephen> anyway.) Jan> Does this solve all cases? I mean, are you sure that this Jan> will trap all cases of data loss? If used properly and it isn't buggy, yes. What is done is to hang a function, latin-unity-sanity-check, on the hook that is used to determine the coding system for writing. The function goes through the buffer, determines all charsets used. If they are restricted to Latin sets, an auxiliary computation is done to see if they are all compatible with a single ISO 8859 character set. If that is the same as the buffer default, the save goes through immediately using that coding system. If they're not the same, the compatible coding system is checked for membership in a preapproved list, if so, the save goes through immediately using that coding system. Optionally (user-configurable) the default coding system for the buffer is updated. Finally, if no luck yet, the best available coding system is computed (it might be a Latin set but not preapproved, or it might be a universal coding system), and recommended to the user. At that point the user can choose any coding system (and thus shoot himself in the foot if some characters can't be encoded), but the recommended system is guaranteed not to lose data. What can go wrong: (1) Somebody hangs an unsafe procedure on the hook with higher priority. (This is why 21.5 needs to have this facility native, so we can make sure there's at least a warning.) (2) There's one corner case where you can escape and use the old Mule method for computing the coding system, but the user has to set latin-unity-like-to-live-dangerously non-nil or the save fails. ;-) Jan> Here's *exactly* what I did (XEmacs 21.5-b12): -- open a file Jan> containing 2022-7 with xemacs -vanilla using C-x C-f Jan> filename.txt -- C-u C-x C-w new-filename.txt RET iso-8859-2 Jan> RET Jan> new-filename.txt still contains ISO-2022-7 where the Jan> ISO-8859-2 characters should be. I'm not talking about Jan> on-screen representation, I'm talking about the file contents Jan> as viewed with less or vi (to be sure nothing messes with Jan> display). If you can, send me the file. I cannot reproduce this behavior. Jan> Perhaps this functionality is also influenced by my LANG Jan> setting? I don't think so; I can't reproduce it here, as I say. Jan> But one more question begs asking: what is the benefit of Jan> having your characters reduced to tildes? I mean, what Jan> purpose does it serve? Many terminals and applications choke on ISO 2022 escape sequences. I don't think there are many left, but I once fried a terminal inadvertantly. He sent me a mail with VT100 escape sequences in it that caused (ASCII graphic) fireworks on my display. I sent it back to him, not realizing that he'd ripped off the fancy console from the machine room and didn't have a VT-compatible. I don't know if he got literal fireworks, but the terminal did blow a fuse! In general, filtering illegal characters out of a text stream is a legitimate function. That's why ASCII has the 0x1A SUBSTITUTE character, and Unicode has the 0xFFFD REPLACEMENT CHARACTER. Mule was designed before Unicode, and you can't use SUB for its original purpose on DOS because somebody decided to use SUB to mean END OF TEXT. Thus the use of TILDE for this purpose. XEmacs is a general text-processing application; it should be able to do such filtering. Jan> So, while I understand your explanations about the complexity Jan> of the issues involved, I still don't understand the Jan> opposition to just changing or removing the evil piece of Jan> code that changes data to tildes. It's not a question of "just removing [one] evil piece of code", because you'd have to make sure that the characters that were left alone would not escape into a buffer or string where they weren't legal internal coding. That would lead to a crash. This is not easy to do, since the translation code is _not_ I/O code, and cannot know whether the internal object it is producing will ever be referenced inside of XEmacs again. So really, the filtering function should be available, it just should be disabled for most purposes. Unfortunately, without a fair amount of work, that's hard to do and still keep the internal data structures consistent. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Ilpo> One place to look in Gnus is mm-find-mime-charset-region in Ilpo> mm.util.el. I've tried looking at that code. Write-only. Ilpo> XEmacs doesn't have the find-coding-systems-region used Ilpo> there and I think that it is the reason why Gnus in XEmacs Ilpo> is not able to unify to utf-8 and sends as multipart Ilpo> instead. `find-coding-systems-region'? Crummy name. Coding systems are a file concept, not a buffer concept. Adrian could try (require 'latin-unity) ;; or (latin-unity-install), which has side effects (defun find-coding-systems-region (from to) (list (car (latin-unity-recommend-representation from to (latin-unity-representations-feasible-region from to) buffer-file-coding-system)))) This is not an exact solution, as l-u-r-r is designed to be interactive. OTOH, this is a situation that seems to call for interaction. Gnus could always try calling `latin-unity-sanity-check' before encoding a buffer. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

|--==> "APA" == Adrian Aichner <adrian(a)xemacs.org&gt; writes: APA> So how do I configure the Gnus XEmacs package under a Mule APA> XEmacs for money transfers in EUR currency? I haven't done anything special, I just: ,---- | ;:*======================= | ;:* Bind the Euro to a key | (define-key global-map "\C-ce" | (lambda () | (interactive) | (insert (make-char 'latin-iso8859-15 #x24)))) `---- -- |---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---| | XEmacs - The only _______ you'll ever need. | | Fill in the blank, yes, it's THAT good! | |------------------------------------&lt;youngs(a)xemacs.org&gt;---|