2:08 p.m.
After upgrading from glibc 2.2.5 to 2.3.1 in Debian `sid', the unexec
version of XEmacs builds successfully, but crashes immediately when
invoked. Are there reasons to believe that the new glibc might cause
problems with the unexec'd binary? It may be related to changes in
malloc, see below.
N.B. I don't consider this a glibc bug, I just want to try to confirm
that it is related to the unexec process or malloc (we have an
alternative way to "dump" Lisp now, the so-called "portable dumper").
If it's not, it's a Heisenbug waiting to bite....
Debian version: testing/unstable (updated Mon Nov 4 19:41:15 2002)
Kernel version: Linux tleepslib 2.2.18 #1 SMP Tue Dec 26 11:36:10 JST 2000 i686 unknown
unknown GNU/Linux
glibc version: 2.3.1-3
binutils version: 2.13.90.0.10-2
gcc version: 2.95.4-12
libgcc1: 3.2.1-0pre5
(The version parts after the hyphen are Debian release numbers for the
benefit of the Debian maintainers.) The kernel is locally compiled,
all the other packages mentioned are stock Debian sid.
XEmacs sources (for completeness):
CVSROOT=:pserver:cvs＠cvs.xemacs.org:/pack/xemacscvs
(password = cvs)
cvs checkout -r r21-4-10 xemacs
To reproduce the crash (my comments are in square brackets []):
------------------------------------------------------------------------
./configure --without-x11 --without-postgresql --with-database=no \
--with-sound=none --without-modules --with-ncurses=no --with-gpm=no
[This gives pretty close to the minimal number of other libraries:
bash-2.05b$ ldd xemacs
libncurses.so.5 => /lib/libncurses.so.5 (0x40022000)
libm.so.6 => /lib/libm.so.6 (0x40061000)
libutil.so.1 => /lib/libutil.so.1 (0x40082000)
libc.so.6 => /lib/libc.so.6 (0x40085000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
and:]
make
[A couple thousand lines of normal output, then:]
Dumping under the name xemacs
[So temacs works fine, and produces an xemacs by unexec.
The following invocation of ./xemacs does NOT appear in normal make
output due to an `＠' in the Makefile.]
Testing for Lisp shadows ...
./xemacs -batch -vanilla -f list-load-path-shadows
Wrong type argument: vectorp, #<INTERNAL OBJECT (XEmacs bug?)
(symbol-value-forward type 7444) 0x40198008>
xemacs exiting
.
make[1]: *** [xemacs] Error 255
make[1]: Leaving directory `/playpen/Projects/XEmacs/staging/xemacs-21.4.10/src'
make: *** [src] Error 2
[This doesn't leave a core. gdb tells me:]
bash-2.05b$ gdb xemacs
GNU gdb 2002-08-18-cvs
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-linux"...
(gdb) run
Starting program: /playpen/Projects/XEmacs/staging/xemacs-21.4.10/src/xemacs
Wrong type argument: vectorp, #<INTERNAL OBJECT (XEmacs bug?) (symbol-value-forward
type 7444) 0x40198008>
Program exited with code 0377.
(gdb) bt
No stack.
------------------------------------------------------------------------
Add --with-system-malloc to the configure, and things proceed
normally. This seems strange, though, because the link list is the
same except for some debugging code (vm-limits.c), while configure
says
GNU version of malloc: no
- User chose not to use GNU allocators.
for the --with-system-malloc=yes case (this short-circuits all
attempts to check for special malloc features and simply uses the
<malloc.h> interface, so configure doesn't recognize GNU malloc in GNU
libc, either), while
GNU version of malloc: yes
- Using Doug Lea's new malloc from the GNU C Library.
for --with-system-malloc=no. There are a number of places where the
relevant conditional compilation (SYSTEM_MALLOC, defined in the first
case, and GNU_MALLOC and DOUG_LEA_MALLOC, defined in the second)
change usage of what look like defined malloc APIs (malloc_get_state
and malloc_set_state, before and after the unexec, it looks like).
They are used for GNU_MALLOC and DOUG_LEA_MALLOC, not used with
SYSTEM_MALLOC. The only place where implementation details seem to be
involved is in the snippet included below.
OK, I'm stuck. Any further information I should provide?
------------------------------------------------------------------------
#ifdef GNU_MALLOC
if (claimed_size < 2 * sizeof (void *))
claimed_size = 2 * sizeof (void *);
# ifdef SUNOS_LOCALTIME_BUG
if (claimed_size < 16)
claimed_size = 16;
# endif
if (claimed_size < 4096)
{
int log = 1;
/* compute the log base two, more or less, then use it to compute
the block size needed. */
claimed_size--;
/* It's big, it's heavy, it's wood! */
while ((claimed_size /= 2) != 0)
++log;
claimed_size = 1;
/* It's better than bad, it's good! */
while (log > 0)
{
claimed_size *= 2;
log--;
}
/* We have to come up with some average about the amount of
blocks used. */
if ((size_t) (rand () & 4095) < claimed_size)
claimed_size += 3 * sizeof (void *);
}
else
{
claimed_size += 4095;
claimed_size &= ~4095;
claimed_size += (claimed_size / 4096) * 3 * sizeof (size_t);
}
#elif defined (SYSTEM_MALLOC)
if (claimed_size < 16)
claimed_size = 16;
claimed_size += 2 * sizeof (void *);
#else
------------------------------------------------------------------------
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
12:01 a.m.
[unrelated to the actual problem, hence Cc's snipped]
On Mon, 2002-11-04 at 16:08, Stephen J. Turnbull wrote:
./configure --without-x11 --without-postgresql --with-database=no
\
--with-sound=none --without-modules --with-ncurses=no --with-gpm=no
[This gives pretty close to the minimal number of other libraries:
bash-2.05b$ ldd xemacs
libncurses.so.5 => /lib/libncurses.so.5 (0x40022000)
libm.so.6 => /lib/libm.so.6 (0x40061000)
libutil.so.1 => /lib/libutil.so.1 (0x40082000)
libc.so.6 => /lib/libc.so.6 (0x40085000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
and:]
Uh, --with-ncurses=no results in libncurses.so.5 dependency?
--
\/ille Skyttä
scop at xemacs.org
5:24 a.m.
>>>> "Ville" == Ville Skytt <Ville>
writes:
Ville> Uh, --with-ncurses=no results in libncurses.so.5
Ville> dependency?
Yes. On Linux, termcap and "old" curses are implemented as
compatibility APIs to libncurses. You'd get a different result on
*BSD or Solaris, I suspect.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
11:44 p.m.
Wrong type argument: vectorp, #<INTERNAL OBJECT (XEmacs bug?)
(symbol-value-forward type 7444) 0x40198008>
xemacs exiting
Sigh, it looks like support for glibc's / "Doug Lea"'s malloc
wasn't
properly merged in XEmacs. You really want to disable mmapped chunks
_only_ for allocation of Lisp Objects, i.e. where pointers become
tagged. As a workaround, I've disabled use of mmap completely in the
patch below.
N.B. the exact same problem should/could show up with earlier glibc
releases, but maybe the allocation pattern was slightly different.
If the patch below works for you, I can try to come up with something
less drastic later.
Regards,
Wolfram.
--- src/alloc.c.orig Mon Dec 17 06:49:22 2001
+++ src/alloc.c Thu Nov 7 00:27:31 2002
＠＠ -3882,8 +3890,8 ＠＠
#ifdef DOUG_LEA_MALLOC
mallopt (M_TRIM_THRESHOLD, 128*1024); /* trim threshold */
mallopt (M_MMAP_THRESHOLD, 64*1024); /* mmap threshold */
-#if 0 /* Moved to emacs.c */
- mallopt (M_MMAP_MAX, 64); /* max. number of mmap'ed areas */
+#if 1 /* Moved to emacs.c */
+ mallopt (M_MMAP_MAX, 0); /* max. number of mmap'ed areas */
#endif
#endif
init_string_alloc ();
--- src/emacs.c.orig Thu Oct 31 16:07:36 2002
+++ src/emacs.c Thu Nov 7 00:25:47 2002
＠＠ -2713,7 +2713,8 ＠＠
if (!initialized)
{
#ifdef DOUG_LEA_MALLOC
- mallopt (M_MMAP_MAX, 0);
+ if (mallopt (M_MMAP_MAX, 0) != 1)
+ abort();
#endif
run_temacs_argc = 0;
if (! SETJMP (run_temacs_catch))
＠＠ -2770,7 +2771,8 ＠＠
defined(_NO_MALLOC_WARNING_) || \
(defined(__GLIBC__) && __GLIBC_MINOR__ < 1 && !defined(MULE)) ||
\
defined(DEBUG_DOUG_LEA_MALLOC)
- mallopt (M_MMAP_MAX, 64);
+ if(mallopt (M_MMAP_MAX, 0) != 1)
+ abort();
#endif
#ifdef REL_ALLOC
r_alloc_reinit ();
5:44 a.m.
>>>> "Wolfram" == Wolfram Gloger
<wmglo(a)dent.med.uni-muenchen.de> writes:
Wrong type argument: vectorp, #<INTERNAL OBJECT (XEmacs bug?)
(symbol-value-forward type 7444) 0x40198008>
xemacs exiting
Wolfram> Sigh, it looks like support for glibc's / "Doug Lea"'s
Wolfram> malloc wasn't properly merged in XEmacs. You really want
Wolfram> to disable mmapped chunks _only_ for allocation of Lisp
Wolfram> Objects, i.e. where pointers become tagged. As a
Wolfram> workaround, I've disabled use of mmap completely in the
Wolfram> patch below.
Thanks you very much for the review and the patch! The patch "works"
in the sense that I can now build.
I'll run with it for a while, put it in the next release candidate,
and see how it affects XEmacs's footprint. Probably it will go in the
next release of 21.4, safety over efficiency. But I would appreciate
it if you could help us recover the mmap capabilities.
Wolfram> N.B. the exact same problem should/could show up with
Wolfram> earlier glibc releases, but maybe the allocation pattern
Wolfram> was slightly different.
I don't understand the problem so I'm not sure what you're saying.
First, your patch also affects "portable dumper" builds which build
and (mostly) run fine. Is this intentional? Ie, is this a generic
problem with our allocator implementation, which "just happened" to
manifest dramatically only in unexec builds on very recent glibcs?
Second, if it's a generic problem, is it possible that it would
generate GCPRO-bug-like symptoms (ie, weird crashes in "obviously
correct" code because data that we know is correctly initialized
mysteriously changes)? For example, we fixed a couple of GCPRO bugs
recently, but we're still seeing mysterious "illegal bytecode" crashes
(especially in Gnus), although fewer of them :-). We're pretty sure
the bytecompiler isn't responsible for this, because we've checked the
code in memory.
Third, is there still a possible problem if we use --with-system-malloc?
Ie, we use the Doug Lea malloc from glibc, but do no mallopt tweaking.
Thanks for your help.
(BTW, I've added references to your home page, and Doug Lea's,
conveniently linked from there, to our Internals docs. Great URL!)
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
4:24 p.m.
Hi,
Thanks you very much for the review and the patch! The patch
"works"
in the sense that I can now build.
Ok, good.
I'll run with it for a while, put it in the next release
candidate,
and see how it affects XEmacs's footprint. Probably it will go in the
next release of 21.4, safety over efficiency. But I would appreciate
it if you could help us recover the mmap capabilities.
Yes, I'll give it a try when I get some time.
Wolfram> N.B. the exact same problem should/could show up
with
Wolfram> earlier glibc releases, but maybe the allocation pattern
Wolfram> was slightly different.
I don't understand the problem so I'm not sure what you're saying.
First, your patch also affects "portable dumper" builds which build
and (mostly) run fine. Is this intentional? Ie, is this a generic
problem with our allocator implementation, which "just happened" to
manifest dramatically only in unexec builds on very recent glibcs?
That's quite probable. You see, the problem is Lisp's tagged
pointers. Pointers to Lisp objects are "coloured" in their most
significant bits (I think 3 bits) with type information. Therefore,
when the malloc implementation hands out chunks with one of those high
bits already set, you get a clash. This can indeed happen with
glibc's malloc on Linux, because it hands out mmapped chunks (they
start near 0x40000000 on ix86-linux), but generally only for "large"
allocations. (You had the M_MMAP_THRESHOLD set to 64k, a reasonable
choice IMHO; one Lisp vector allocation from the "temacs -dump" run
just exceeded that threshold.) By setting M_MMAP_MAX to 0 I've
disabled all use of mmap; glibc's malloc behaves more like a classic
malloc then.
For GNU Emacs, I've added temporary switches of M_MMAP_MAX to 0 and
back _only_ in the Lisp object allocation paths (there were about half
a dozen places); I'll try to do the same for XEmacs when I find the
time.
Second, if it's a generic problem, is it possible that it would
generate GCPRO-bug-like symptoms (ie, weird crashes in "obviously
correct" code because data that we know is correctly initialized
mysteriously changes)? For example, we fixed a couple of GCPRO bugs
recently, but we're still seeing mysterious "illegal bytecode" crashes
(especially in Gnus), although fewer of them :-). We're pretty sure
the bytecompiler isn't responsible for this, because we've checked the
code in memory.
Not sure about this, but I suspect that "mysteriously changing" memory
cannot occur due to this. The chance that a coloured pointer is
masked (by stripping off the top bits) into a valid memory region
seems quite small to me. But it's certainly not impossible.
Third, is there still a possible problem if we use
--with-system-malloc? Ie, we use the Doug Lea malloc from glibc,
but do no mallopt tweaking.
Yes, there is the same problem, AFAICS. IMHO the autoconf test should
check whether mallopt(M_MMAP_MAX) is available, and deduce that it's a
variant of Doug Lea malloc.
Regards,
Wolfram.
5:49 p.m.
>>>> "Wolfram" == Wolfram Gloger
<Wolfram.Gloger(a)dent.med.uni-muenchen.de> writes:
> First, your patch also affects "portable dumper" builds
which
> build and (mostly) run fine. Is this intentional? Ie, is this
> a generic problem with our allocator implementation, which
> "just happened" to manifest dramatically only in unexec builds
> on very recent glibcs?
Wolfram> That's quite probable. You see, the problem is Lisp's
Wolfram> tagged pointers. Pointers to Lisp objects are "coloured"
Wolfram> in their most significant bits (I think 3 bits) with type
Wolfram> information.
Not in XEmacs. In XEmacs >= 21.4, there are 3 kinds of immediate Lisp
Objects and a pointer Lisp Object, and the tags are in the _low_ two
bits. If the low bit is 1, it is a 31-bit Lisp integer. If the two
low bits are 10, the object is a 30-bit Lisp character.
If the two low bits are 00, then the object is an lrecord type, and
the bit pattern is interpreted as (struct lrecord *). Note that there
is no difference between the bit pattern as a Lisp Object and the bit
pattern used to address the record.
This is the default implementation of Lisp Object in 21.1 as well, but
I believe that 21.1 still has the option to use the traditional high
tag implementation. Here's the #define from 21.1'a config.h.in:
/* If defined, use a minimal number of tagbits. This allows usage of more
advanced versions of malloc (like the Doug Lea new GNU malloc) and larger
integers. */
/* --use-minimal-tagbits */
#undef USE_MINIMAL_TAGBITS
Wolfram> You had the M_MMAP_THRESHOLD set to 64k, a reasonable
Wolfram> choice IMHO; one Lisp vector allocation from the "temacs
Wolfram> -dump" run just exceeded that threshold.
Well, obviously we were doing something wrong, because your patch
fixed it. But it seems we understood the problems with traditional
high tags and Doug Lea malloc.
Wolfram> IMHO the autoconf test should check whether
Wolfram> mallopt(M_MMAP_MAX) is available, and deduce that it's a
Wolfram> variant of Doug Lea malloc.
FWIW, we actually use the following test, based on malloc_set_state
and __after_morecore_hook. I have never heard that it made a mistake,
either false positive or false negative.
if test "$with_dlmalloc" != "no"; then
doug_lea_malloc=yes
else
doug_lea_malloc=no
fi
after_morecore_hook_exists=yes
AC_CHECK_FUNC(malloc_set_state, ,doug_lea_malloc=no)
AC_MSG_CHECKING(whether __after_morecore_hook exists)
AC_TRY_LINK([extern void (* __after_morecore_hook)();],[__after_morecore_hook = 0],
[AC_MSG_RESULT(yes)],
[AC_MSG_RESULT(no)
after_morecore_hook_exists=no])
dnl Tests to get the value of GNU_MALLOC go here.
if test "$doug_lea_malloc" = "yes" -a "$GNU_MALLOC" =
"yes" ; then
GNU_MALLOC_reason="
- Using Doug Lea's new malloc from the GNU C Library."
AC_DEFINE(DOUG_LEA_MALLOC)
if test "$after_morecore_hook_exists" = "no" ; then
GNU_MALLOC_reason="
- Using Doug Lea's new malloc from the Linux C Library."
AC_DEFINE(_NO_MALLOC_WARNING_)
fi
fi
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
4:05 p.m.
Hi,
After upgrading from glibc 2.2.5 to 2.3.1 in Debian `sid', the
unexec
version of XEmacs builds successfully, but crashes immediately when
invoked. Are there reasons to believe that the new glibc might cause
problems with the unexec'd binary? It may be related to changes in
malloc, see below.
Certainly possible. However, I've personally checked xemacs-21.4.6
against the then-current glibc-2.2.90 in April, and it was working
fine. Since then, few malloc changes have gone in.
N.B. I don't consider this a glibc bug, I just want to try to
confirm
that it is related to the unexec process or malloc (we have an
alternative way to "dump" Lisp now, the so-called "portable dumper").
BTW: that "portable dumper" sounds like great news, please enable by
default soon :-).
I have downloaded the 21.4.10 CVS that you mentioned and will try it
tonight.
Until then, I'll try a shot in the dark and suggest:
2002-10-07 Wolfram Gloger <wg(a)malloc.de>
* malloc/malloc.c (sYSMALLOc): Only check for breakage due
to foreign sbrk()'s if arena is contiguous. Bug report from
Bruno Haible <bruno(a)clisp.org>.
--- malloc.c 2002/06/11 09:36:14 1.10
+++ malloc.c 2002/10/07 15:33:46
＠＠ -2911,7 +2911,7 ＠＠
if (brk == old_end && snd_brk == (char*)(MORECORE_FAILURE))
set_head(old_top, (size + old_size) | PREV_INUSE);
- else if (old_size && brk < old_end) {
+ else if (contiguous(av) && old_size && brk < old_end) {
/* Oops! Someone else killed our space.. Can't touch anything. */
assert(0);
}
Regards,
Wolfram.
7838
days inactive
7843
days old
7 comments
4 participants
participants (4)
-
Stephen J. Turnbull -
Ville Skyttä -
Wolfram Gloger -
Wolfram Gloger