Terje> If the signature you are refering to is the UNICODE Terje> Byte-Order Mark then XML 1.0 Second Edition contained an Thank you for clarifying the standard. But, uhm, ZERO-WIDTH NO-BREAK SPACE, if you please. ;-) There is no BOM in UTF-8; not even Microsoft could get away with advocating little-endian UTF-8. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Terje> It's there in UTF-8 to allow an heuristic parser Heuristic is the operative term here. Ah, the stench of Microsoft in mid-winter.... Terje> Then again, this all tends to give me a headache so I'm Terje> probably just hopelessly confused. :-) I'm sure you have a headache, and I'm sure you're confused, although far from hopeless. :-) This is precisely why Microsoft is guilty of criminal monopolization under the law. There never was a need for little-endian UTF-16, and there is no need for a UTF-8 encoding signature in XML. However, Microsoft has long since recognized that every dollar they spend dealing with gratuitous complexity (1) gives you a headache so you're less productive, and (2) thereby pays them back with $10 in monopoly power. "Embrace and Extend[tm]." Standards should be ruthlessly spare and cold. To err is human, of course, so software should be divinely forgiving and educational: "Dammit, Beavis, that's not XML, because it's not UTF-8 and you didn't say _what_ it is. But we'll let you go this time...." -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

For example, for some time I found such signatures in auto-autoload files in the core lisp. Doesn't seem to be there any more. Ben, do you know about this? There's also a complaint that on native Windows (XP) split-char is doing things like (split-char ?\351) => (chinese-sisheng 38) which may or may not be related. SJT> This needs to be fixed SY> OK, how? SY> Could I hang something off 'find-file-hooks' to change the SY> coding system if it's "MSW UTF8"? Not the right idea. Er ... try this: (set-coding-priority-list '(iso-8-1 iso-7 utf-16-little-endian-bom utf-16-bom utf-8-bom no-conversion)) or even (set-coding-priority-list '(iso-8-1 iso-7 no-conversion)) The current setting is (utf-16-little-endian-bom utf-16-bom utf-8-bom iso-7 no-conversion) which makes some theoretical sense in old Mule (unless a file begins with Unicode 0xFEFF, perhaps transformed into little-endian or UTF-8, it _cannot_ trigger any of the *-bom coding systems), but I'm not sure how it interacts with Ben's new coding detection algorithms (which are advertised as being able to choose the "highest probability", but I haven't yet studied what that actually means). SY> Is it my XEmacs that is adding the "UTF-8 signature" when I SY> visit a file edited by someone else? At one point 21.5 was doing this to my lisp/auto-autoloads, yes. I don't know why; maybe it makes sense on Ben's box. And it doesn't any more. It could very well have done that a couple of versions (of XEmacs) ago, and both CVS and later XEmacsen maintain "your" mods.... In which case it's now (possibly) fixed. Remove the signatures from the files and see if they come back. Wait for the definitive word from Ben. SY> Or is it coming from the person who edited the file before me? That could also be, in which case XEmacs is maintaining the mods as it should. SY> I usually run a Mule-enabled XEmacs and if I byte-compile SY> these files with a Mule-XEmacs I can't load them with a SY> non-Mule one. I get a "can't open load file: mule" error. Er, check those files for the coding system cookie `escape-quoted'. They are explicitly set to be encoded in raw-text-unix (binary) or escape-quoted. If they are getting the UTF-8 signature that is very bad indeed; it means the coding systems are leaking into each other. SY> What do we do about this, Steve? Scream to Ben, for starters. It's probably a one-line fix (if not already fixed)---if you're intimately familiar with the code. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

SY> Before rearranging the coding-priority-list these files had: SY> (require 'mule) SY> ;;;###coding system: escape-quoted That's bad. Would you mind setting the c-p-l back to default, and recompiling? I want to know if the bug is still present. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.