Re: Typos in GCC output

Nix writes: They do by default in most situations, such as reading from files. But processes are a different matter, because they're not rewindable. So basically the detector needs to make a decision based on the first bufferful, which is short and very likely to be pure ASCII. (Don't say "but ..." for a couple paragraphs, please.) In that situation, it's typically a bad idea to assume anything but binary. The codecs don't know that they're dealing with GCC; they could easily be dealing with XEmacs, which has a habit of dumping byte code in its Lisp backtraces. It turns out that in XEmacs 'binary is synonymous with 'iso-8859-1 (a backwards-compatibility hack), which had the misfeature of doing the right thing 90% of the time, keeping the Mule developers from working on pushing that to 99.4%. You could keep a list of which apps are well-behaved, but I suspect this would be unreliable. I don't know of any (working) resampling codecs that check to see if things have changed on the fly. It probably makes sense in the case of synchronous processes, such as *compile* buffers, to treat them as "slow" files. That is, buffer the whole output from the process, do detection on a large sample, and then convert the whole thing. (There's an obvious space optimization of buffering only until you have so much you won't use any more, convert the detection sample, and use that codec for the rest on-the-fly.) But it's not obvious how to make this work for asynchronous processes, such as *shell* buffers. Do you really want to reconvert the buffer every time the process spits out more output? (Maybe you do, but somebody's gonna have to code it and put it in front of users to see what they think.) And what if "real-gcc" is a lie, and in fact it's a script LANG=Etruscan exec really-real-gcc "$＠" ? People (at least in the Japanese subset) actually do stuff like that. Environment is for setting user defaults, but the user rarely interfaces directly with GCC these days; they interact with some kind of filter (aka IDE). "System-global" is incorrect as you point out, but LANG *is* process-global, which is inappropriate in multilingual applications. (Suppose iconv obeyed LANG? Then in your environment it would only be useful for converting UTF-8 to UTF-8! ;-) Nope; it's a chicken-and-egg problem. Sometimes you start with the chicken and sometimes with the egg. There is no general way to do this; it's AI stuff. You need a heavy-duty multilingual natural language processing system to do what even the famed 7-bit null-lingual American programmer can do "by eye". You're darn tootin' it isn't. That's basically the issue that killed XIM. setlocale() can take a noticeable amount of time in situations where you're switching back and forth between input modes all the time. And don't even mention "multi-threading"! Uh, isn't that what iconv and XEmacs are for? And what makes you think that the guy at the next desk will necessarily use Unicode for his name?---after all, your company only bought him and his code last week. It's a serious layering violation for GCC itself to be doing those translations. gcc (the gcc controller app itself) should assume that Java code is native-endian UTF-16 (that's the standard, right?). Users should invoke gcc via a wrapper that handles the translations for them. It's that in a multilingual environment the odds are very good that LANG inherited from a process's initialization doesn't match the encoding I'm using right now, yes. In Japan it is; UTF-8-encoded Japanese text is very much a minority taste even today. Of course you can change encodings back. The issue is, why should I have to pay that tax for a *compiler's error output*?? The `' convention is perfectly usable though ugly, and can easily be beautified (eg, with a nearly harmless comint filter in Emacsen). Because I'm a resident of Japan, which has *5* major encodings (ISO-2022-JP, EUC-JP, Shift-JIS, UTF-8, and roman transliteration) reachable with one keystroke from my mail summary buffer, not to mention GB2312 (aka the preferred encoding of Chinese spam), UTF-16, and other special-purpose encodings primarily used internally to various applications that sometimes leak into public (damn that broken Windows anyway, it's such a pane). I *can* stick to LANG=ja_JP.UTF-8, precisely because XEmacs ignores the "UTF-8". The important part of that to XEmacs is "ja_JP", because it tells XEmacs to prefer Japanese fonts and encodings where characters are shared with Chinese and/or Korean. Once I know it's Japanese, the statistical characteristics of the octet streams give "no defects in 10 years of heavy daily use" reliability in distinguishing the 4 "real" Japanese encodings from each other. And even if I'm in an environment where the text is more likely to be French than Japanese, XEmacs does better than 99%. XEmacs has no trouble decoding that, and even if it did, you could fix it with a simple comint filter. What bothers me is that a useful protocol was changed without warning, from something that is simple and robust and well-known even to legacy applications, to something that is less simple, demonstrably unreliable, and likely to cause bad interactions with smart applications that use code dating from before the protocol change. Since the protocol was never formalized, GCC is certainly within its rights to say "the joke's on you for trying to do something useful with our past very regular behavior". But I don't think that's very productive. Well, it's not specific to you. The basic tension is that for humans it's all quite obvious, right there in front of your nose in FG_COLOR and BG_COLOR. A machine, however, generally has no idea what text it's spewing at users. It has even less idea about the encodings it is being fed. It's just a variant on the Postel Principle for Internet clients: be catholic about what you accept, puritan about what you produce. In this case, "puritan" can, and IMO should, mean "use UTF-8, of course! but restrict yourself to the subset 0-127". ;-) _______________________________________________ XEmacs-Beta mailing list XEmacs-Beta(a)xemacs.org http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta