[Bug: 21.5-b24] Problems with coding systems autodetect

================================================================ Dear Bug Team! I have problems with autodetection of coding systems. I have attached a file that has two lines (at the end). If I open that file, I get the coding system big5. I would expect to get the coding system iso-8859-1 or similar. My coding categories are: ############################ ## LIST OF CODING CATEGORIES (ordered by priority) ## CATEGORY:CODING-SYSTEM ## utf-16-little-endian-bom:utf-16-little-endian-bom utf-16-bom:utf-16-bom utf-8-bom:utf-8-bom iso-7:iso-2022-7bit no-conversion:raw-text utf-8:utf-8 iso-8-1:iso-8859-1 iso-8-2:ctext iso-8-designate:ctext iso-lock-shift:iso-2022-lock shift-jis:shift-jis big5:big5 utf-16-little-endian:utf-16-little-endian utf-16:utf-16 ucs-4:ucs-4 I don't have much experience with XEmacs coding systems (in fact, today I read doc strings on that topic for the first time). Nevertheless, if I interpret that documentation correctly, iso-8-1 should be checked before big5; and since the file is encoded in Latin1, it should match. I would appreciate any hint what I'm doing wrong; or if this is XEmacs' fault, if there is a work-around to fix the autodetection. Incidentially, if I store the two lines of the attached file in two separate files and open them, the first line gets the coding system "Raw" and the second line gets the coding system "Noconv". I also don't understand why these different coding systems are selected. Btw, I'm working in a POSIX locale: $ locale LANG= LC_CTYPE="POSIX" LC_NUMERIC="POSIX" LC_TIME="POSIX" LC_COLLATE="POSIX" LC_MONETARY="POSIX" LC_MESSAGES="POSIX" LC_PAPER="POSIX" LC_NAME="POSIX" LC_ADDRESS="POSIX" LC_TELEPHONE="POSIX" LC_MEASUREMENT="POSIX" LC_IDENTIFICATION="POSIX" LC_ALL= Cheers, Joachim ================================================================ System Info to help track down your bug: --------------------------------------- uname -a: Linux n-pussy 2.6.13-15.7-smp #1 SMP Tue Nov 29 14:32:29 UTC 2005 i686 i686 i386 GNU/Linux ./configure '--with-ipv6-cname=no' '--with-pop' '--with-mule' '--with-athena=3d' '--prefix=/opt/xemacs' '--with-optimization' '--with-error-checking=no' XEmacs 21.5-b24 "dandelion" configured for `i686-pc-linux'. Compilation Environment and Installation Defaults: Source code location: /home/schrod/src/xemacs/xemacs-r21-5-current-beta Installation prefix: /opt/xemacs Operating system description file: `s/linux.h' Machine description file: `m/intel386.h' Compiler version: gcc (GCC) 4.0.2 20050901 (prerelease) (SUSE Linux) - GCC specs file: specs. - Compiler command: gcc -Wall -Wno-switch -Wundef -Wsign-compare -Wno-char-subscripts -Wpacked -Wunused-parameter -g -O3 libc version: 2.3.5 Relocating allocator for buffers: no GNU version of malloc: yes - Using Doug Lea's new malloc from the GNU C Library. Window System: Compiling in support for the X window system: - X Windows headers location: /usr/X11/include - X Windows libraries location: /usr/X11/lib - Handling WM_COMMAND properly. Compiling in support for the Athena widget set: - Athena headers location: X11/Xaw3d - Athena library to link: Xaw3d Using Lucid menubars. Using Lucid scrollbars. Using Athena dialog boxes. Using Athena native widgets. TTY: Compiling in support for ncurses. Compiling in support for GPM (General Purpose Mouse). Images: Compiling in support for GIF images (builtin). Compiling in support for XPM images. Compiling in support for PNG images. Compiling in support for JPEG images. Compiling in support for TIFF images. Sound: Compiling in support for sound (native). Databases: Internationalization: Compiling in support for Mule (multi-lingual Emacs). Compiling in support for XIM (X11R5+ I18N input method). - Using raw Xlib to provide XIM support. Mail: Compiling in support for POP mail retrieval. Compiling in support for "file" mail spool file locking method. Other Features: Inhibiting IPv6 canonicalization at startup. Compiling in support for dynamic shared object modules. Using the new GC algorithms. WARNING: --------------------------------------------------------- WARNING: The new algorithms are experimental. They are enabled by WARNING: default for this release. Use `--disable-kkcc' to WARNING: turn it off. WARNING: --------------------------------------------------------- Using the new allocator. WARNING: --------------------------------------------------------- WARNING: The new allocator is experimental. It is enabled by WARNING: default for this release. Use `--disable-mc-alloc' to WARNING: turn it off. WARNING: --------------------------------------------------------- Using the new portable dumper. Compiling in support for extra debugging code. Load-Path Lisp Shadows: ---------------------- (/home/schrod/lib/emacs-21.4/font-latex /opt/xemacs/lib/xemacs/xemacs-packages/lisp/auctex/font-latex /opt/xemacs/lib/xemacs/site-packages/lisp/packages/cl-indent /opt/xemacs/lib/xemacs/xemacs-packages/lisp/prog-modes/cl-indent /opt/xemacs/lib/xemacs/site-packages/lisp/packages/filladapt /opt/xemacs/lib/xemacs/xemacs-packages/lisp/text-modes/filladapt /opt/xemacs/lib/xemacs/mule-packages/lisp/mule-base/china-util /opt/xemacs/lib/xemacs-21.5-b24/lisp/mule/china-util /opt/xemacs/lib/xemacs/mule-packages/lisp/mule-base/cyril-util /opt/xemacs/lib/xemacs-21.5-b24/lisp/mule/cyril-util /opt/xemacs/lib/xemacs/mule-packages/lisp/mule-base/ethio-util /opt/xemacs/lib/xemacs-21.5-b24/lisp/mule/ethio-util /opt/xemacs/lib/xemacs/mule-packages/lisp/mule-base/japan-util /opt/xemacs/lib/xemacs-21.5-b24/lisp/mule/japan-util /opt/xemacs/lib/xemacs/mule-packages/lisp/mule-base/korea-util /opt/xemacs/lib/xemacs-21.5-b24/lisp/mule/korea-util /opt/xemacs/lib/xemacs/mule-packages/lisp/mule-base/viet-util /opt/xemacs/lib/xemacs-21.5-b24/lisp/mule/viet-util /opt/xemacs/lib/xemacs/xemacs-packages/lisp/build/build-report /opt/xemacs/lib/xemacs-21.5-b24/lisp/build-report /opt/xemacs/lib/xemacs/xemacs-packages/lisp/xemacs-base/regexp-opt /opt/xemacs/lib/xemacs-21.5-b24/lisp/regexp-opt /opt/xemacs/lib/xemacs/mule-packages/lisp/mule-ucs/unicode /opt/xemacs/lib/xemacs-21.5-b24/lisp/unicode /opt/xemacs/lib/xemacs/xemacs-packages/lisp/xemacs-base/easy-mmode /opt/xemacs/lib/xemacs-21.5-b24/lisp/easy-mmode) Installed XEmacs Packages: ------------------------- (zenirc ver: 1.16 upstream: 2.112) (xwem ver: 1.21 upstream: lg(a)xwem.org--2005/xwem--main--2.1--versionfix-1) (xslt-process ver: 1.12 upstream: 1.2.1) (xslide ver: 1.09 upstream: 0.2.2) (xlib ver: 1.14 upstream: lg(a)xwem.org--2005/xlib--main--2.1--version-0) (xetla ver: 1.01 upstream: steve(a)eicq.org--2005/xetla--main--1.1--version-0) (xemacs-devel ver: 1.72 upstream: No-Upstream-Ver) (xemacs-base ver: 2.01 upstream: No-Upstream-Ver) (x-symbol ver: 1.1 upstream: 4.5.1) (w3 ver: 1.32 upstream: 4.0pre47) (vm ver: 7.22 upstream: 7.17) (viper ver: 1.47 upstream: 3.09) (view-process ver: 1.13 upstream: 2.4) (vhdl ver: 1.2 upstream: 3.33.2) (vc-cc ver: 1.22 upstream: No-Upstream-Ver) (vc ver: 1.41 upstream: No-Upstream-Ver) (tramp ver: 1.28 upstream: 2.0.51) (tpu ver: 1.14 upstream: 4.2X) (tooltalk ver: 1.15 upstream: No-Upstream-Ver) (tm ver: 1.38 upstream: No-Upstream-Ver) (time ver: 1.14 upstream: 1.17) (textools ver: 1.15 upstream: No-Upstream-Ver) (text-modes ver: 1.9 upstream: No-Upstream-Ver) (texinfo ver: 1.3 upstream: No-Upstream-Ver) (supercite ver: 1.21 upstream: 3.55x3) (strokes ver: 1.1 upstream: No-Upstream-Ver) (speedbar ver: 1.28 upstream: 0.14beta4) (sounds-wav ver: 1.12 upstream: No-Upstream-Ver) (sounds-au ver: 1.12 upstream: No-Upstream-Ver) (sml-mode ver: 0.12 upstream: 3.9.5) (slider ver: 1.15 upstream: 0.3x1) (sieve ver: 1.18 upstream: No-Upstream-Ver) (sh-script ver: 1.21 upstream: 2.0f) (sgml ver: 1.11 upstream: No-Upstream-Ver) (semantic ver: 1.19 upstream: 1.4.2) (scheme ver: 1.14 upstream: No-Upstream-Ver) (sasl ver: 1.16 upstream: 1.14.4) (ruby-modes ver: 1.02 upstream: 1.6.8) (riece ver: 1.21 upstream: 2.0.2) (reftex ver: 1.34 upstream: 4.21) (re-builder ver: 1.05 upstream: 1.20) (python-modes ver: 1.07 upstream: No-Upstream-Ver) (psgml-dtds ver: 1.03 upstream: No-Upstream-Ver) (psgml ver: 1.44 upstream: 1.3.1) (ps-print ver: 1.11 upstream: 6.5.6) (prog-modes ver: 2.06 upstream: No-Upstream-Ver) (pgg ver: 1.06 upstream: 0.1) (perl-modes ver: 1.08 upstream: No-Upstream-Ver) (pcomplete ver: 1.04 upstream: 1.1.6) (pcl-cvs ver: 1.66 upstream: R-2_9_9) (pc ver: 1.28 upstream: No-Upstream-Ver) (os-utils ver: 1.36 upstream: No-Upstream-Ver) (oo-browser ver: 1.04 upstream: 4.08) (ocaml ver: 0.05 upstream: 3.06) (net-utils ver: 1.48 upstream: N/A) (mmm-mode ver: 1.02 upstream: 0.4.7) (misc-games ver: 1.18 upstream: No-Upstream-Ver) (mine ver: 1.16 upstream: 1.9) (mh-e ver: 1.29 upstream: 7.4.2) (mew ver: 1.19 upstream: 1.94.2) (mailcrypt ver: 2.14 upstream: 3.5.8) (mail-lib ver: 1.75 upstream: No-Upstream-Ver) (liece ver: 1.14 upstream: 1.4.9) (jde ver: 1.48 upstream: 2.3.2) (ispell ver: 1.32 upstream: 3.6) (ilisp ver: 1.33 upstream: 5.12.0) (igrep ver: 1.14 upstream: 2.111) (idlwave ver: 1.32 upstream: 5.1) (ibuffer ver: 1.09 upstream: No-Upstream-Ver) (hyperbole ver: 1.16 upstream: 4.18) (hm--html-menus ver: 1.23 upstream: 5.9) (haskell-mode ver: 1.07 upstream: 1.45) (gnus ver: 1.89 upstream: 5.10.7) (gnats ver: 1.17 upstream: 3.101) (general-docs ver: 1.04 upstream: No-Upstream-Ver) (games ver: 1.17 upstream: 1.04) (fsf-compat ver: 1.15 upstream: No-Upstream-Ver) (frame-icon ver: 1.11 upstream: No-Upstream-Ver) (fortran-modes ver: 1.05 upstream: No-Upstream-Ver) (forms ver: 1.15 upstream: 2.37) (footnote ver: 1.16 upstream: 0.18x) (eudc ver: 1.39 upstream: 1.32) (eterm ver: 1.17 upstream: No-Upstream-Ver) (ess ver: 1.16 upstream: 5.2.7) (eshell ver: 1.1 upstream: 2.4.1) (escreen ver: 1.01 upstream: 1.16) (erc ver: 0.19 upstream: Version 5.0 Revision: 1.776) (emerge ver: 1.11 upstream: No-Upstream-Ver) (elib ver: 1.11 upstream: 1.0) (eieio ver: 1.05 upstream: 0.17) (efs ver: 1.33 upstream: 1.23) (edt ver: 1.13 upstream: No-Upstream-Ver) (edit-utils ver: 2.32 upstream: No-Upstream-Ver) (ediff ver: 1.6 upstream: 2.75) (edebug ver: 1.21 upstream: No-Upstream-Ver) (ecrypto ver: 0.2 upstream: 2.0) (ecb ver: 1.22 upstream: 2.31) (docbookide ver: 0.07000000000000001 upstream: 0.1) (dired ver: 1.17 upstream: 7.13) (dictionary ver: 1.16 upstream: 1.8) (debug ver: 1.17 upstream: No-Upstream-Ver) (crisp ver: 1.15 upstream: 1.34) (cookie ver: 1.15 upstream: No-Upstream-Ver) (clearcase ver: 1.1 upstream: /main/laptop/165) (cc-mode ver: 1.45 upstream: 5.30.10) (calendar ver: 1.23 upstream: No-Upstream-Ver) (calc ver: 1.26 upstream: 2.02fX3) (c-support ver: 1.22 upstream: No-Upstream-Ver) (build ver: 1.14 upstream: 2.02) (bbdb ver: 1.29 upstream: 2.34) (auctex ver: 1.46 upstream: 11.55) (apel ver: 1.32 upstream: 10.6) (ada ver: 1.14 upstream: 2.27) (Sun ver: 1.16 upstream: No-Upstream-Ver) (mule-ucs ver: 1.14 upstream: 0.84) (mule-base ver: 1.47 upstream: No-Upstream-Ver) (lookup ver: 1.15 upstream: 1.0) (locale ver: 1.22 upstream: No-Upstream-Ver) (leim ver: 1.22 upstream: No-Upstream-Ver) (latin-unity ver: 1.16 upstream: 1.16) (latin-euro-standards ver: 1.07 upstream: 1.07) (egg-its ver: 1.27 upstream: No-Upstream-Ver) (edict ver: 1.16 upstream: 0.9.9) Installed Modules: ----------------- Features: -------- (vm-folder mail-abbrevs vm-menu vm-reply vm-misc vm-summary vm-mouse vm-window alist vm-autoload vm-vars highlight-headers vm-version vm vm-startup xemacsbug shadow vm-auto-fcc sendmail rfc822 trim-lines template efs-cu dired-faces dired-xemacs dired gnuserv sp-ccmode sp-vmsetup sp-toolbar sp-menubar font sp-look x-compose sp-bindings tex-site font-lock byte-optimize bytecomp byte-compile disp-table jka-compr sp-funs sp-setup frame-icon filladapt advice advice-preload uniquify func-menu cus-face zenirc-autoloads xwem-autoloads xslt-process-autoloads xslide-autoloads xlib-autoloads xetla-autoloads xemacs-devel-autoloads xemacs-base-autoloads x-symbol-autoloads w3-autoloads vm-autoloads viper-autoloads view-process-autoloads vhdl-autoloads vc-cc-autoloads vc-autoloads tramp-autoloads tpu-autoloads tooltalk-autoloads tm-autoloads time-autoloads textools-autoloads text-modes-autoloads texinfo-autoloads supercite-autoloads strokes-autoloads speedbar-autoloads sounds-wav-autoloads sounds-au-autoloads sml-mode-autoloads slider-autoloads sieve-autoloads sh-script-autoloads sgml-autoloads semantic-autoloads scheme-autoloads sasl-autoloads ruby-modes-autoloads riece-autoloads reftex-autoloads re-builder-autoloads python-modes-autoloads psgml-dtds-autoloads psgml-autoloads ps-print-autoloads prog-modes-autoloads pgg-autoloads perl-modes-autoloads pcomplete-autoloads pcl-cvs-autoloads pc-autoloads os-utils-autoloads oo-browser-autoloads ocaml-autoloads net-utils-autoloads mmm-mode-autoloads misc-games-autoloads mine-autoloads mh-e-autoloads mew-autoloads mailcrypt-autoloads mail-lib-autoloads liece-autoloads jde-autoloads ispell-autoloads ilisp-autoloads igrep-autoloads idlwave-autoloads ibuffer-autoloads hyperbole-autoloads hm--html-menus-autoloads haskell-mode-autoloads gnus-autoloads gnats-autoloads general-docs-autoloads games-autoloads fsf-compat-autoloads frame-icon-autoloads fortran-modes-autoloads forms-autoloads footnote-autoloads eudc-autoloads eterm-autoloads ess-autoloads eshell-autoloads escreen-autoloads erc-autoloads emerge-autoloads elib-autoloads eieio-autoloads efs-autoloads edt-autoloads edit-utils-autoloads ediff-autoloads edebug-autoloads ecrypto-autoloads ecb-autoloads docbookide-autoloads dired-autoloads dictionary-autoloads debug-autoloads crisp-autoloads cookie-autoloads clearcase-autoloads cc-mode-autoloads calendar-autoloads calc-autoloads c-support-autoloads build-autoloads bbdb-autoloads auctex-autoloads apel-autoloads ada-autoloads Sun-autoloads mule-ucs-autoloads mule-base-autoloads lookup-autoloads locale-autoloads leim-autoloads latin-unity-autoloads latin-euro-standards-autoloads egg-its-autoloads edict-autoloads npc-sp-utils-autoloads npc-packages-autoloads npc-modes-autoloads modules-autoloads mule-autoloads auto-autoloads loadhist rsz-minibuf auto-show fontl-hooks canna-leim tibetan lao devanagari indian slovenian czech romanian ccl code-cmds gutter-items menubar-items x-menubar mode-motion mouse behavior itimer auto-save lisp-mode easymenu easy-mmode iso8859-1 page buff-menu lib-complete cus-file derived newcomment env text-props fontconfig frame obsolete cus-start custom widget cl-extra cl cl-19 packages backquote unicode lucid-scrollbars cut-buffer lucid-menubars athena-dialogs x c-balloon-help tty-frames tty toolbar native-sound scrollbar unix-processes multicast network-streams subprocesses modules menu-accelerator-support menubar md5 xemacs xim mule gutter tiff png gif jpeg xpm xbm lisp-float-type file-coding linux dialog devices window-system base64) Recent keystrokes: ----------------- SPC SPC f C-f1 misc-user Recent messages (most recent first): ----------------------------------- Loading vm-folder...done Loading vm-folder... Parsing /home/schrod/Mail/Alias... done Parsing /home/schrod/Mail/Alias... Loading mail-abbrevs...done Loading mail-abbrevs... Loading vm-menu...done Loading vm-menu... Loading vm-reply...done Loading vm-reply... Loading vm-misc...done Loading vm-misc... Loading vm-summary...done Loading vm-summary... Loading vm-mouse...done Loading vm-mouse... Loading vm-window...done Loading vm-window... Loading alist...done Loading alist...