Re: [21.5 AC sjt-xft] Convert Canna to ELL, but crashes with MC_ALLOC

SJT> The buggy version can be checked out from CVS with -r SJT> canna-mc-alloc-crash. Although this tag is on the sjt-xft SJT> branch, you can build a very close facsimile of current CVS SJT> simply by configuring --with-xft=none. When I try to compile with Xft, I get a couple of errors in lwlib/xlwmenu.c: gcc -c -Wall -Wno-switch -Wundef -Wsign-compare -Wno-char-subscripts -Wpacked -Wunused-parameter -g -I. -I/home/crestani/src/xemacs/canna-mc-xemacs-21.5/lwlib/../src -DHAVE_CONFIG_H -I/usr/include/freetype2 -I/usr/X11/include xlwmenu.c xlwmenu.c: In function `string_draw': xlwmenu.c:785: error: `gc' undeclared (first use in this function) xlwmenu.c:785: error: (Each undeclared identifier is reported only once xlwmenu.c:785: error: for each function it appears in.) xlwmenu.c:770: warning: unused parameter `color' xlwmenu.c:771: warning: unused parameter `colorBg' xlwmenu.c: In function `string_draw_range': xlwmenu.c:848: error: `gc' undeclared (first use in this function) xlwmenu.c:825: warning: unused parameter `color' xlwmenu.c:826: warning: unused parameter `colorBg' xlwmenu.c: In function `binding_draw': xlwmenu.c:989: error: `gc' undeclared (first use in this function) xlwmenu.c:989: warning: passing arg 6 of `string_draw' from incompatible pointer type xlwmenu.c:989: error: too few arguments to function `string_draw' xlwmenu.c:980: warning: unused parameter `color' xlwmenu.c:981: warning: unused parameter `colorBg' xlwmenu.c: In function `XlwMenuInitialize': xlwmenu.c:3242: error: structure has no member named `renderFont' xlwmenu.c:3243: error: structure has no member named `renderFontSpec' xlwmenu.c: At top level: xlwmenu.c:754: warning: `x_xft_text_width' defined but not used This is the Installation file of my configuration: uname -a: Linux zoiby 2.6.11.4-21.9-smp #1 SMP Fri Aug 19 11:58:59 UTC 2005 i686 i686 i386 GNU/Linux ./configure '--enable-mule' '--with-canna' XEmacs 21.5-b22 "cucumber" (+Xft+CVS-20050922) configured for `i686-pc-linux'. Compilation Environment and Installation Defaults: Source code location: /home/crestani/src/xemacs/canna-mc-xemacs-21.5 Installation prefix: /usr/local Operating system description file: `s/linux.h' Machine description file: `m/intel386.h' Compiler version: gcc (GCC) 3.3.5 20050117 (prerelease) (SUSE Linux) - GCC specs file: /usr/lib/gcc-lib/i586-suse-linux/3.3.5/specs - Compiler command: gcc -Wall -Wno-switch -Wundef -Wsign-compare -Wno-char-subscripts -Wpacked -Wunused-parameter -g libc version: Relocating allocator for buffers: no GNU version of malloc: yes - Using Doug Lea's new malloc from the GNU C Library. Window System: Compiling in support for the X window system: - X Windows headers location: /usr/X11/include - X Windows libraries location: /usr/X11/lib - Handling WM_COMMAND properly. Using Lucid menubars. - Using Xft to render antialiased fonts in menubars. WARNING: This feature will be replaced with a face. Using Lucid scrollbars. Using Motif dialog boxes. Using Motif native widgets. - Using Xft to render antialiased fonts in tab controls. WARNING: This feature will be replaced with a face. - Using Xft to render antialiased fonts in progress bars. WARNING: This feature will be replaced with a face. WARNING: This feature not yet implemented; setting ignored. TTY: Compiling in support for ncurses. Compiling in support for GPM (General Purpose Mouse). Images: Compiling in support for GIF images (builtin). Compiling in support for XPM images. Compiling in support for PNG images. Compiling in support for JPEG images. Compiling in support for TIFF images. Sound: Compiling in support for sound (native). Databases: Compiling in support for Berkeley database. Compiling in support for LDAP. Compiling in support for PostgreSQL. - Using PostgreSQL header file: pgsql/libpq-fe.h - Using PostgreSQL V7 bindings. Internationalization: Compiling in support for Mule (multi-lingual Emacs). Compiling in support for XIM (X11R5+ I18N input method). - Using Motif to provide XIM support. Compiling in support for Canna on Mule. Mail: Compiling in support for "file" mail spool file locking method. Other Features: Inhibiting IPv6 canonicalization at startup. Compiling in support for dynamic shared object modules. Using the new GC algorithms. WARNING: --------------------------------------------------------- WARNING: The new algorithms are experimental. They are enabled by WARNING: default for this release. Use `--disable-kkcc' to WARNING: turn it off. WARNING: --------------------------------------------------------- Using the new allocator. WARNING: --------------------------------------------------------- WARNING: The new allocator is experimental. It is enabled by WARNING: default for this release. Use `--disable-mc-alloc' to WARNING: turn it off. WARNING: --------------------------------------------------------- Using the new portable dumper. Compiling in support for extra debugging code. Compiling in support for runtime error checking. WARNING: --------------------------------------------------------- WARNING: XEmacs will run noticeably more slowly as a result. WARNING: Error checking is on by default for XEmacs beta releases. WARNING: --------------------------------------------------------- This seems to be some mixup of the preprocessor symbols USE_XFT_MENUBARS and NEED_MOTIF. Sorry, currently I do not have the time to take a closer look at this. Using --with-xft=none only worked after I patched src/glyphs-x.c: --- src/glyphs-x.c 15 Mar 2005 05:24:21 -0000 1.80.2.6 +++ src/glyphs-x.c 25 Sep 2005 16:42:18 -0000 ＠＠ -2351,12 +2351,14 ＠＠ lw_add_widget_value_arg (wv, XtNfont, (XtArgVal) fs); } +#ifdef USE_XFT /* #### sanity check, should wrap in appropriate ERROR_CHECK macro */ if (!rf && !fs) warn_when_safe_lispobj (intern ("xft"), Qdebug, Fcons (build_string ("missing font in update_widget_face"), Fface_name (face))); +#endif } wv->change = VISIBLE_CHANGE; /* #### Megahack - but its just getting too complicated to do this Without this patch, rf is undefined. And I stumbled about another thing: Without the following patch, I got "Symbol's value as variable is void: file-error" while I tried to figure out what canna stuff I needed to install. --- lisp/mule/canna-leim.el 24 Sep 2005 14:02:22 -0000 1.7.8.1 +++ lisp/mule/canna-leim.el 25 Sep 2005 17:51:58 -0000 ＠＠ -46,10 +46,10 ＠＠ (defun canna-activate (&optional name) ;; XEmacs 21.5.10 and later have 3-argument require. (unless (require 'CANNA "canna/canna_api" 'no-error) - (error file-error + (error 'file-error "Neither built-in nor ELL support for the Canna API was found.")) (unless (require 'canna nil 'no-error) - (error file-error + (error 'file-error "Canna LISP support not found. Is the mule-base package missing?")) (setq inactivate-current-input-method-function 'canna-inactivate) (unless (featurep 'leim-canna-initialized) SJT> Marcus, any help you can give here as to what to look for? I'll SJT> be working on getting a backtrace, but it may take some time. SJT> No, I can't set a breakpoint there, either, as the Mac's gdb SJT> doesn't handle keyboard interrupt correctly. Sorry. I would not SJT> assume that the bug is in MC_ALLOC, the Canna code is over ten SJT> years old. The bug hits during the next garbage collection after the user has set the input method to japanese-canna. The backtrace shows: [...] #7 0x080e0be5 in assert_failed (file=0x82d90ff "mc-alloc.c", line=579, expr=0x82d910a "LEVEL2 (l2, L2_INDEX (ptr))") at emacs.c:3941 #8 0x081bddd5 in get_page_header (ptr=0x8417ff0) at mc-alloc.c:579 #9 0x081be136 in get_mark_bit (ptr=0x8417ff0) at mc-alloc.c:803 #10 0x0808f4bd in kkcc_gc_stack_push_lisp_object_1 (obj=138510320, level=621, pos=3) at alloc.c:3989 #11 0x0808f546 in mark_object_maybe_checking_free_1 (obj=138510320, allow_free=0, level=621, pos=3) at alloc.c:4030 #12 0x0808f6fc in kkcc_marking () at alloc.c:4144 #13 0x080905b3 in garbage_collect_1 () at alloc.c:5565 [...] The traversal of the live objects reaches an object that is not correctly allocated with the new allocator, thus the mapping of the heap address to the page header does not work. The object not correctly allocated is: (gdb) up #10 0x0808f4bd in kkcc_gc_stack_push_lisp_object_1 (obj=138510320, level=621, pos=3) at alloc.c:3989 3989 if (! MARKED_RECORD_HEADER_P (lheader)) (gdb) print *(struct lrecord_header *) obj $15 = {type = 5, lisp_readonly = 1, free = 0, uid = 0} The lrecord type number 5 stands for Lisp_Subr: (gdb) print *(Lisp_Subr *) obj $16 = {lheader = {type = 5, lisp_readonly = 1, free = 0, uid = 0}, min_args = 0, max_args = 0, prompt = 0x0, doc = 0x8827538 "Get current mode string.\n\narguments: ()\n", name = 0x40a51c30 "canna-query-mode", subr_fn = 0x40a4c24a <Fcanna_query_mode>} The above Lisp_Subr is a DEFUN of the new canna module and is apparently not allocated in valid Lisp-accessible memory, at least from the new allocator's point of view. I took a look at the defsubr code in symbol.c and found that the problem is in check_module_subr. Check_module_subr wants to put the subr in Lisp-accessible memory. This is correct for the old allocator, but not needed for the new allocator and it actually breaks module support with the new allocator. Check_module_subr copies the subr to newly allocated memory that is not managed by the new allocator: newsubr = xnew (Lisp_Subr); \ memcpy (newsubr, subr, sizeof (Lisp_Subr)); \ subr->doc = (const char *)newsubr; \ subr = newsubr; \ This patch removes this part for the new allocator and makes the canna module work: --- src/symbols.c 13 Jul 2005 04:47:01 -0000 1.46.2.3 +++ src/symbols.c 26 Sep 2005 06:02:29 -0000 ＠＠ -3474,6 +3474,7 ＠＠ #endif #ifdef HAVE_SHLIB +#ifndef MC_ALLOC /* * If we are not in a pure undumped Emacs, we need to make a duplicate of * the subr. This is because the only time this function will be called ＠＠ -3533,6 +3534,45 ＠＠ subr = newsubr; \ } \ } while (0) +#else /* MC_ALLOC */ +/* + * If we have the new allocator enabled, we do not need to make a + * duplicate of the subr. The new allocator already does allocate all + * subrs in Lisp-accessible memory rather than have it in the static + * subr struct. + * + * NOTE: The DOC pointer is not set here as described above. + */ +#define check_module_subr(subr) \ +do { \ + if (initialized) { \ + Lisp_Object f; \ + \ + if (subr->min_args < 0) \ + signal_ferror (Qdll_error, "%s min_args (%hd) too small", \ + subr_name (subr), subr->min_args); \ + if (subr->min_args > SUBR_MAX_ARGS) \ + signal_ferror (Qdll_error, "%s min_args (%hd) too big (max = %d)", \ + subr_name (subr), subr->min_args, SUBR_MAX_ARGS); \ + \ + if (subr->max_args != MANY && \ + subr->max_args != UNEVALLED) \ + { \ + /* Need to fix lisp.h and eval.c if SUBR_MAX_ARGS too small */ \ + if (subr->max_args > SUBR_MAX_ARGS) \ + signal_ferror (Qdll_error, "%s max_args (%hd) too big (max = %d)", \ + subr_name (subr), subr->max_args, SUBR_MAX_ARGS); \ + if (subr->min_args > subr->max_args) \ + signal_ferror (Qdll_error, "%s min_args (%hd) > max_args (%hd)", \ + subr_name (subr), subr->min_args, subr->max_args); \ + } \ + \ + f = XSYMBOL (sym)->function; \ + if (!UNBOUNDP (f) && (!CONSP (f) || !EQ (XCAR (f), Qautoload))) \ + signal_ferror (Qdll_error, "Attempt to redefine %s", subr_name (subr)); \ + } \ +} while (0) +#endif /* MC_ALLOC */ #else /* ! HAVE_SHLIB */ #define check_module_subr(subr) #endif I will send this patch for the mc-alloc module support to the mainline, shall I also commit it to sjt-xft? Would you take care of the Xft- and canna-related patches I included in this post? -- Marcus