[Bug: 21.4.19] utf-8 encoding

================================================================ Dear Bug Team! As a historian of science and medicine I'm definitely not a software or computer expert, nevertheless, I mainly use (and like) xemacs to edit large latex-files for my books (texts in German and other languages with a lot of diacritic symbols). A few days ago I changed my gentoo linux system from latin-1 to utf-8 encoding. Everything went fine, however, I noticed the following bug: If I edit an ordinary plain text *.tex-file with xemacs 21.4.19 one or two - not more! - out of about 15,000 German Umlauts (ä, ö, ü etc.) in the file are not saved with their correct utf-8 two-byte hex codes (e. g. "ü" = "c3 b3") but with a 3 (!) byte garbled hex code (e.g. "ü" = "ef bf bd"), which, of course, causes a stop of the latex interpreter ("no \u8 symbol"). I checked the hex codes with a hexadecimal editor and could reproduce the bug just by directly replacing the correct code and saving the file once more with xemacs. I couldn't reproduce the bug with other editors (vim). I don't have any idea about the reason but I noticed that the first byte (c3) was saved at the offset address 0007:fff0 and the second (b3) at 0008:0000. After I had moved the character two bytes forward (just by adding twa blanks) the bug didn't happen any longer. Yours, Matthias M. Weber ================================================================ System Info to help track down your bug: --------------------------------------- uname -a: Linux pcarch 2.6.17-gentoo #1 Sat Jul 1 12:55:45 CEST 2006 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux ./configure '--prefix=/usr/local/xemacs' '--with-mule' XEmacs 21.4.19 "Constant Variable" configured for `x86_64-unknown-linux'. Compilation / Installation: Source code location: /home/mmw/tmp/xemacs-21.4.19 Installation prefix: /usr/local/xemacs Operating system description file: `s/linux.h' Not using any machine description file Compiler: gcc -g -O3 -Wall -Wno-switch -Winline -Wmissing-prototypes -Wsign-compare -fno-strict-aliasing -Wshadow Compiler version: x86_64-pc-linux-gnu-gcc (GCC) 4.1.1 (Gentoo 4.1.1) Compiler specs file: specs. Relocating allocator for buffers: no GNU version of malloc: yes - Using Doug Lea's new malloc from the GNU C Library. Need to guess glibc1/2/etc here Window System: Compiling in support for the X window system: - X Windows headers location: /usr/X11R6/include - X Windows libraries location: /usr/X11R6/lib - Handling WM_COMMAND properly. Compiling in support for the Athena widget set: - Athena headers location: X11/Xaw - Athena library to link: Xaw Using Lucid menubars. Using Lucid scrollbars. Using Athena dialog boxes. TTY: Compiling in support for ncurses. Compiling in support for GPM (General Purpose Mouse). Images: Compiling in support for GIF images (builtin). Compiling in support for XPM images. Compiling in support for PNG images. Compiling in support for JPEG images. Compiling in support for TIFF images. Sound: Compiling in support for sound (native). Databases: Compiling in support for Berkeley database. Compiling in support for LDAP. Compiling in support for PostgreSQL. - Using PostgreSQL header file: libpq-fe.h - Using PostgreSQL V7 bindings. Internationalization: Compiling in support for Mule (multi-lingual Emacs). Compiling in support for XIM (X11R5+ I18N input method). - Using raw Xlib to provide XIM support. Mail: Compiling in support for "dot-locking" mail spool file locking method. Other Features: Inhibiting IPv6 canonicalization at startup. Compiling in support for dynamic shared object modules. Using the new portable dumper. Load-Path Lisp Shadows: ---------------------- (/usr/local/xemacs/lib/xemacs/xemacs-packages/lisp/build/build-report /usr/local/xemacs/lib/xemacs-21.4.19/lisp/build-report) Installed XEmacs Packages: ------------------------- (zenirc ver: 1.16 upstream: 2.112) (xslt-process ver: 1.12 upstream: 1.2.1) (xslide ver: 1.09 upstream: 0.2.2) (xemacs-devel ver: 1.5 upstream: single-file) (xemacs-base ver: 1.75 upstream: regular) (x-symbol ver: 1.1 upstream: 4.5.1) (w3 ver: 1.32 upstream: 4.0pre47) (vm ver: 7.22 upstream: 7.17) (viper ver: 1.48 upstream: 3.09) (view-process ver: 1.13 upstream: 2.4) (vhdl ver: 1.21 upstream: 3.33.8) (vc-cc ver: 1.22 upstream: No-Upstream-Ver) (vc ver: 1.41 upstream: No-Upstream-Ver) (tramp ver: 1.32 upstream: 2.0.53) (tpu ver: 1.14 upstream: 4.2X) (tooltalk ver: 1.15 upstream: No-Upstream-Ver) (tm ver: 1.38 upstream: No-Upstream-Ver) (time ver: 1.14 upstream: 1.17) (textools ver: 1.15 upstream: No-Upstream-Ver) (text-modes ver: 1.9 upstream: No-Upstream-Ver) (texinfo ver: 1.3 upstream: No-Upstream-Ver) (supercite ver: 1.21 upstream: 3.55x3) (strokes ver: 1.1 upstream: No-Upstream-Ver) (speedbar ver: 1.28 upstream: 0.14beta4) (sounds-wav ver: 1.12 upstream: No-Upstream-Ver) (sounds-au ver: 1.12 upstream: No-Upstream-Ver) (sml-mode ver: 0.12 upstream: 3.9.5) (slider ver: 1.15 upstream: 0.3x1) (sieve ver: 1.18 upstream: No-Upstream-Ver) (sh-script ver: 1.21 upstream: 2.0f) (sgml ver: 1.11 upstream: No-Upstream-Ver) (semantic ver: 1.2 upstream: 1.4.4) (scheme ver: 1.14 upstream: No-Upstream-Ver) (sasl ver: 1.16 upstream: 1.14.4) (ruby-modes ver: 1.02 upstream: 1.6.8) (rmail ver: 1.14 upstream: No-Upstream-Ver) (riece ver: 1.22 upstream: 2.0.2) (reftex ver: 1.34 upstream: 4.21) (python-modes ver: 1.07 upstream: No-Upstream-Ver) (psgml-dtds ver: 1.03 upstream: No-Upstream-Ver) (psgml ver: 1.44 upstream: 1.3.1) (ps-print ver: 1.11 upstream: 6.5.6) (prog-modes ver: 2.07 upstream: No-Upstream-Ver) (pgg ver: 1.06 upstream: 0.1) (perl-modes ver: 1.09 upstream: No-Upstream-Ver) (pcomplete ver: 1.04 upstream: 1.1.6) (pcl-cvs ver: 1.66 upstream: R-2_9_9) (pc ver: 1.28 upstream: No-Upstream-Ver) (os-utils ver: 1.37 upstream: No-Upstream-Ver) (oo-browser ver: 1.04 upstream: 4.08) (ocaml ver: 0.05 upstream: 3.06) (net-utils ver: 1.48 upstream: N/A) (mmm-mode ver: 1.02 upstream: 0.4.7) (misc-games ver: 1.18 upstream: No-Upstream-Ver) (mine ver: 1.16 upstream: 1.9) (mh-e ver: 1.29 upstream: 7.4.2) (mew ver: 1.19 upstream: 1.94.2) (mailcrypt ver: 2.14 upstream: 3.5.8) (mail-lib ver: 1.76 upstream: No-Upstream-Ver) (liece ver: 1.12 upstream: regular) (jde ver: 1.46 upstream: regular) (ispell ver: 1.24 upstream: regular) (ilisp ver: 1.29 upstream: regular) (igrep ver: 1.12 upstream: 2.95) (idlwave ver: 1.31 upstream: 5.1) (ibuffer ver: 1.09 upstream: No-Upstream-Ver) (hyperbole ver: 1.13 upstream: 4.18) (hm--html-menus ver: 1.23 upstream: 5.9) (haskell-mode ver: 1.06 upstream: 1.44) (gnus ver: 1.71 upstream: regular) (gnats ver: 1.16 upstream: 3.101) (games ver: 1.15 upstream: 1.04) (fsf-compat ver: 1.13 upstream: No-Upstream-Ver) (frame-icon ver: 1.11 upstream: No-Upstream-Ver) (fortran-modes ver: 1.03 upstream: No-Upstream-Ver) (forms ver: 1.15 upstream: 2.37) (footnote ver: 1.16 upstream: 0.18x) (eudc ver: 1.39 upstream: 1.32) (eterm ver: 1.13 upstream: regular) (ess ver: 1.06 upstream: 5.1.21) (eshell ver: 1.06 upstream: 2.4.1) (emerge ver: 1.09 upstream: regular) (elib ver: 1.1 upstream: single-file) (eieio ver: 1.05 upstream: 0.17) (efs ver: 1.29 upstream: regular) (edt ver: 1.13 upstream: No-Upstream-Ver) (edit-utils ver: 1.98 upstream: single-file) (ediff ver: 1.46 upstream: regular) (edebug ver: 1.15 upstream: regular) (ecrypto ver: 0.12 upstream: regular) (ecb ver: 1.13 upstream: 1.96) (docbookide ver: 0.07000000000000001 upstream: 0.1) (dired ver: 1.13 upstream: regular) (dictionary ver: 1.12 upstream: regular) (debug ver: 1.17 upstream: No-Upstream-Ver) (crisp ver: 1.14 upstream: 1.34) (cookie ver: 1.14 upstream: regular) (clearcase ver: 1.08 upstream: No-Upstream-Ver) (cc-mode ver: 1.32 upstream: regular) (calendar ver: 1.22 upstream: No-Upstream-Ver) (calc ver: 1.24 upstream: 2.02fX3) (c-support ver: 1.18 upstream: No-Upstream-Ver) (build ver: 1.14 upstream: 2.02) (bbdb ver: 1.21 upstream: regular) (auctex ver: 1.35 upstream: regular) (apel ver: 1.27 upstream: 10.2) (ada ver: 1.14 upstream: 2.27) (skk ver: 1.23 upstream: regular) (mule-ucs ver: 1.05 upstream: 0.84) (mule-base ver: 1.44 upstream: No-Upstream-Ver) (lookup ver: 1.14 upstream: 1.0) (locale ver: 1.2 upstream: regular) (leim ver: 1.22 upstream: No-Upstream-Ver) (latin-unity ver: 1.09 upstream: 1.09) (egg-its ver: 1.27 upstream: No-Upstream-Ver) (edict ver: 1.16 upstream: 0.9.9) Installed Modules: ----------------- Features: -------- (xemacsbug shadow tex-mode mail-utils mail-abbrevs sendmail vm-reply cal-move cal-xemacs cal-china lunar solar cal-dst cal-persia cal-islam cal-hebrew cal-julian holidays diary-lib calendar ethio-util w3-hot url-file url-cache w3-xemac w3-xemacs w3-toolbar xbm-button xpm-button w3-forms images url-http url-cookie timezone urlauth w3 w3-menu url-news url-gw w3-script w3-jscript w3-elisp w3-java cus-edit cus-load w3-emulate w3-auto w3-parse url mm url-vars w3-cfg w3-sysdp w3-display w3-mouse advice advice-preload w3-imap w3-vars wid-edit w3-cus mule-sysdp w3-widget url-parse css w3-keyword vm-page vm-mime vm-save vm-toolbar vm-motion vm-summary tapestry vm-undo vm-message vm-menu vm-folder efs-cu font-lock vm-misc vm-mouse vm-window vm-autoload vm-vars highlight-headers vm-version vm vm-startup mailcrypt rfc822 comint ring un-define mule-uni font disp-table cus-face zenirc-autoloads xslt-process-autoloads xslide-autoloads xemacs-devel-autoloads xemacs-base-autoloads x-symbol-autoloads w3-autoloads vm-autoloads viper-autoloads view-process-autoloads vhdl-autoloads vc-cc-autoloads vc-autoloads tramp-autoloads tpu-autoloads tooltalk-autoloads tm-autoloads time-autoloads textools-autoloads text-modes-autoloads texinfo-autoloads supercite-autoloads strokes-autoloads speedbar-autoloads sounds-wav-autoloads sounds-au-autoloads sml-mode-autoloads slider-autoloads sieve-autoloads sh-script-autoloads sgml-autoloads semantic-autoloads scheme-autoloads sasl-autoloads ruby-modes-autoloads rmail-autoloads riece-autoloads reftex-autoloads python-modes-autoloads psgml-dtds-autoloads psgml-autoloads ps-print-autoloads prog-modes-autoloads pgg-autoloads perl-modes-autoloads pcomplete-autoloads pcl-cvs-autoloads pc-autoloads os-utils-autoloads oo-browser-autoloads ocaml-autoloads net-utils-autoloads mmm-mode-autoloads misc-games-autoloads mine-autoloads mh-e-autoloads mew-autoloads mailcrypt-autoloads mail-lib-autoloads liece-autoloads jde-autoloads ispell-autoloads ilisp-autoloads igrep-autoloads idlwave-autoloads ibuffer-autoloads hyperbole-autoloads hm--html-menus-autoloads haskell-mode-autoloads gnus-autoloads gnats-autoloads games-autoloads fsf-compat-autoloads frame-icon-autoloads fortran-modes-autoloads forms-autoloads footnote-autoloads eudc-autoloads eterm-autoloads ess-autoloads eshell-autoloads emerge-autoloads elib-autoloads eieio-autoloads efs-autoloads edt-autoloads edit-utils-autoloads ediff-autoloads edebug-autoloads ecrypto-autoloads ecb-autoloads docbookide-autoloads dired-autoloads dictionary-autoloads debug-autoloads crisp-autoloads cookie-autoloads clearcase-autoloads cc-mode-autoloads calendar-autoloads calc-autoloads c-support-autoloads build-autoloads bbdb-autoloads auctex-autoloads apel-autoloads ada-autoloads skk-autoloads mule-ucs-autoloads mule-base-autoloads lookup-autoloads locale-autoloads leim-autoloads latin-unity-autoloads egg-its-autoloads edict-autoloads lisp-autoloads loadhist auto-show fontl-hooks x-iso8859-1 slovenian czech romanian ccl mule-help code-cmds gutter-items menubar-items x-menubar mode-motion mouse itimer auto-save lisp-mode easymenu iso8859-1 page buff-menu lib-complete cus-file derived frame text-props obsolete cus-start custom widget cl-extra mini-cl cl cl-19 packages backquote very-early-lisp postgresqlv7 postgresql file-coding mule lucid-scrollbars cut-buffer lucid-menubars athena-dialogs x c-balloon-help tty-frames tty toolbar native-sound scrollbar unix-processes multicast network-streams subprocesses modules menu-accelerator-support menubar berkeley-db md5 xemacs xim gutter tiff png gif jpeg xpm xbm lisp-float-type linux dialog devices window-system base64) Recent keystrokes: ----------------- b 6 / k b 6 . t e x RET ESC > up up up SPC BS C-x C-s C-x k RET C-x C-f up RET C-s C-s up down C-s D e z e r n e n t e n C-s down down down down down up left left left left left left left left left left left left SPC right right right right right right right SPC C-x C-s C-x k RET ESC x ESC x C-g C-g button1 button1up C-g C-g ESC x r e p o TAB r t TAB x e TAB RET Recent messages (most recent first): ----------------------------------- Parsing /home/mmw/.mailrc... Fontifying *mail*... Loading xemacsbug...done Loading xemacsbug... Making completion list... Quit Quit Not over a window. Quit Quit Command attempted to use minibuffer while in minibuffer Wrote /home/mmw/texkrp/kb6/kb6.tex Fontifying kb6.tex... done. Fontifying kb6.tex... (regexps)................. Fontifying kb6.tex... (regexps)................ Fontifying kb6.tex... (regexps)................ Fontifying kb6.tex... (regexps)................ Fontifying kb6.tex... (regexps)................ Fontifying kb6.tex... (regexps)................ Fontifying kb6.tex... (regexps)............... -- Prof. Dr. med. Matthias M. Weber Geschäftsführender Oberarzt Leiter des Historischen Archivs Max-Planck-Institut für Psychiatrie (Deutsche Forschungsanstalt für Psychiatrie) Kraepelinstraße 2-10, D-80804 München Tel +49 (0)89 30622 485 Fax +49 (0)89 30622 621 mmw(a)mpipsykl.mpg.de