Well, evidently it's not my stupidity. It wasn't the call to Simply changing the call to `insert-file-contents-internal' to `insert-file-contents-literally' did the trick. I'm not going to submit a patch yet because I simply do not understand what the heck is going on. There is _all_ kinds of evil lurking in those functions. Not to mention confusing comments like "whatever happened to i-f-c-literally?" (in package-get.el), which makes no sense, as AFAIK it never went AWOL. I suspect that the `package-get-maybe-save-index' code could also be vulnerable to coding system silliness if the archive host and the local host have different line-ending conventions. Upgrading all my packages was a bad idea though, as I got hosed by the mule-base 1.32 bug. -- University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091 __________________________________________________________________________ __________________________________________________________________________ What are those two straight lines for? "Free software rules."

Darryl> I'll bet that it is the file-coding insanity. I've Darryl> been going through file-coding hell trying to get crypt.el Darryl> to work, and I'm coming to the conclusion that the design Darryl> of the coding system is to blame (it's not a bug -- it's Darryl> the design itself). Well, you can take that position if you like. It's not unreasonable. But ambiguous coding systems cannot be avoided in multilingual applications for the foreseeable future; there are just too many different encodings competing for 0x80-0xFF :-) Darryl> One basic issue is that, if you use the default Darryl> coding "undecided" and allow XEmacs to determine the Why is this happening? The point is that if users are loading in files, editing them as text, and then doing some crypt operation on them, you have to write them out encoded before crypting them anyway. (This can be short-circuited by using function like `encode-coding-region' on the buffer. But you can't do this if the data has been "corrupted", the encoding is not necessarily reversible. No-op sequences of charset designation escape sequences will simply disappear in the buffer representation, for example.) If the application doesn't permit you to forbid that, then users are going to make mistakes, either by themselves (forgetting the coding system argument) or by proxy (defaulting to 'undecided). Darryl> coding system, you must know beforehand if a file is Darryl> binary or not [...] a lovely kludge to crypt.el where Darryl> crypt reloads the file in binary mode if it detects that Darryl> the file was loaded in non-binary mode. Of course, this Darryl> is extremely user-hostile for large files. Yup. Your point is well taken; your alternative might be? As you point out, the buffer data has already been corrupted. What you really need to do is keep the user from reading in non-Unix-ASCII text as text in the first place. A crude attempt: Darryl> [ I'm not sure if I should submit my crypt.el patches, as Darryl> I'm wondering if it's just better to drop-kick file coding Darryl> and recompile XEmacs without it. ] This is not an alternative for Japanese-, Chinese-, and Korean- speaking users. Obviously, it should be possible to make XEmacs's default encoding 'binary. A command line switch might be satisfactory for non-Asian users. However, Asians would still like to use crypt.el I'm sure, and that default would not be satisfactory for them. We still need something like your patches. I hope you'll submit them. -- University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091 __________________________________________________________________________ __________________________________________________________________________ What are those two straight lines for? "Free software rules."

Darryl> Well, instead of file-coding, what about hooks, in Darryl> the grand sense of Emacs: As a matter of architecture, it's probably better. But since it is orthogonal to the ambiguity problem, I don't know if it's worth the effort enough to get done soon. Darryl> Also, are there any multilingual encodings that use Darryl> 0x00-0x1f (aside from the usual LF, CR, TAB, BS, & FF)? Unicode. Asians will use Unicode in files for the space savings over UTF-8 (all ideographs are 3 bytes, and all syllabic characters are 2 bytes). Darryl> Yes, and mangling crypt.el to properly handle writes Darryl> isn't too bad (as long as some symbol, other than `nil', Darryl> can be used to represent `binary' in file-coding). The Darryl> big problem is reading in encoded/encrypted files; the EOL Darryl> detection in file-coding often corrupts the binary file. Binding `coding-system-for-read' to `binary', using `insert-file-contents-literally', or passing `binary' as the optional CODING-SYSTEM argument to `insert-file-contents-internal' all should work. Darryl> Well, I first need your "explicit-binary-symbol" Darryl> file-coding fixes. My current patches assume that Darryl> call-process uses "binary" for subprocess output (instead Darryl> of the current "undecided"); I have to do this, because Yes. I think this should be made the default. This is one place where a hook should definitely be provided to do the guessing. You'd need an oracle to do this right, especially since a process can change its mind about which coding system it's going to spew without warning XEmacs. Darryl> there is currently no non-nil symbol I can bind to Darryl> coding-system-for-read to represent binary. Not documented outside of the source, I guess.... (coding-system-docstring 'binary) Darryl> [ Well, I suppose I could use `no-conversion-unix' to Darryl> represent binary. It's unclean, though. Does anyone know Darryl> if `no-conversion-unix' does any converting? ] 'binary is an alias for 'no-conversion-unix. Both do absolutely no conversion. (The file-format hooks etc may get run, but that's independent of the file-coding stuff.) This is documented somewhere.... Argh! (get-coding-system nil) (get-coding-system 'binary) (get-coding-system 'no-conversion-unix) The first two are `eq' because of the way `get-coding-system' is implemented. The last is not `equal' to the other two, but all its testable properties are `eq' to those of 'binary (they're memcpy's of each other, so they'd better be). -- University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091 __________________________________________________________________________ __________________________________________________________________________ What are those two straight lines for? "Free software rules."