Very nice! I am truly amazed at how quickly you went right to the source
of the problem, given the fact you said you haven't looked at the code for
a while. Looks like I was barking up the wrong tree so to speak. I saw
the symptom, but was nowhere near identifying the root cause.
Thanks for fixing this. Will look for the patch after the archive updates
tonight (I'm not on xemacs-patches list).
Thanks again, Troy
-----Original Message-----
From: Olivier Galibert [mailto:galibert@pobox.com]
Sent: Thursday, November 16, 2000 12:24 AM
To: Troy Noble
Cc: 'xemacs-beta(a)xemacs.org'
Subject: Re: Who is the portable dumper expert? 21.2.37 Portable dumper
s till broken.
On Wed, Nov 15, 2000 at 03:25:24PM -0700, Troy Noble wrote:
> Olivier, thanks for being so gracious.
>
> So here's the situation. I've tried this on both NT with MSVC 5.0 and
> Linux with gcc/gdb using 21.2b36 and 21.2b37.
>
> In src/indent.c there is the following:
>
> void
> vars_of_indent (void)
> {
> DEFVAR_BOOL ("indent-tabs-mode", &indent_tabs_mode /*
> *Indentation can insert tabs if this is non-nil.
> Setting this variable automatically makes it local to the current buffer.
> */ );
> indent_tabs_mode = 1;
> }
>
> When the portable dumper is enabled, setting or clearing this variable has
> no
> effect. It's almost like the portable dumper loads it into memory but
never
> wires up the appropriate forwarding information so that the lisp code
knows
> how to find the variable. I suspect this is true of other such opaque
> variables,
> but haven't proven that conclusively yet.
>
> I do know that if I set "watch indent_tabs_mode" the only time it ever
gets
> set throughout the lifetime of my debug session is at startup time. Any
> subsequent
> calls to (setq-default indent-tabs-mode nil) or (setq indent-tabs-mode
nil)
> have no effect. If I stop the debugger (CTRL-C) and do "p
indent_tabs_mode"
> it's
> always 1. Here's where it gets set initially:
>
> Old value = 0
> New value = 1
> 0x4034e4b3 in memcpy (dstpp=0x8331f7c, srcpp=0x40580f34, len=4)
> at ../sysdeps/generic/memcpy.c:61
> 61 ../sysdeps/generic/memcpy.c: No such file or directory.
> (gdb) bt
> #0 0x4034e4b3 in memcpy (dstpp=0x8331f7c, srcpp=0x40580f34, len=4)
> at ../sysdeps/generic/memcpy.c:61
> #1 0x80c8407 in pdump_load_finish () at dumper.c:1041
> #2 0x80c891d in pdump_load (
> argv0=0xbfffe517 "/usr2/tnoble/build/xemacs-21.2.36/src/./xemacs") at
> dumper.c:1358
> #3 0x80b15e2 in xemacs_21_2_b36_i686_pc_linux (argc=1, argv=0xbfffe284,
> envp=0xbfffe28c, restart=0) at emacs.c:1164
> #4 0x80b3bd1 in main (argc=1, argv=0xbfffe284, envp=0xbfffe28c) at
> emacs.c:2595
>
>
> So am wondering if there is an additional step that might need to be added
> after the opaques are loaded in pdump_load_finish to make them visible to
> lisp. The code in question is:
>
> dumper.c, Line 1041:
>
> /* Put back the opaques */
> for (i=0; i<((dump_header *)pdump_start)->nb_opaquedmp; i++)
> {
> struct pdump_dumpopaqueinfo di = PDUMP_READ (p, struct
> pdump_dumpopaqueinfo);
> > memcpy (di.data, p, di.size);
> p += di.size;
> }
>
>
> Interestingly, when I turn off the portable dumper... I always get a watch
> breakpoint
> at the following location whenever I (setq-default indent-tabs-mode
> <whatever>):
>
> symbols.c, Line 1255:
>
> case SYMVAL_BOOLEAN_FORWARD:
> if (magicfun)
> magicfun (sym, &newval, Qnil, 0);
> *((int *) symbol_value_forward_forward (fwd))
> = !NILP (newval);
> > return;
>
>
> So do these opaques need to be added to the pdump_reloc_table (presumably
> in pdump_dump_rtable and then again when the rtable gets read back in) so
> they get manipulated in pdump_reorganize_hash_table, or is there something
> else that needs to happen?
>
> dumper.c, line 1089 and following:
>
> /* Final cleanups */
> /* reorganize hash tables */
> p = pdump_rt_list;
> for (;;)
> {
> pdump_reloc_table rt = PDUMP_READ (p, pdump_reloc_table);
> if (!rt.desc)
> break;
> if (rt.desc == hash_table_description)
> {
> for (i=0; i < rt.count; i++)
> pdump_reorganize_hash_table (PDUMP_READ (p, Lisp_Object));
> break;
> } else
> p += sizeof (Lisp_Object) * rt.count;
> }
>
> I don't mind doing the legwork, just not sure what direction to head
> at the moment since I've only just started crawling into the code.
>
> Thanks in advance for any direction, Troy
Ah grasshopper, if it was that easy :-)
pdump_reorganize_hash_table is a red herring. It exists only because
most hashcodes are computed from the actual addresses of the objects,
and since said addresses change depending on where the dump file was
loaded they have to be reorganized according to the new hashcodes.
So the question is, is the opaque dumped? If you look at symeval.h you can
see:
#define DEFVAR_BOOL(lname, c_location) \
DEFVAR_SYMVAL_FWD_INT (lname, c_location, SYMVAL_BOOLEAN_FORWARD, 0)
#define DEFVAR_SYMVAL_FWD_INT(lname, c_location, forward_type, magicfun) do{
\
DEFVAR_SYMVAL_FWD (lname, c_location, forward_type, magicfun);
\
dumpopaque (c_location, sizeof(int));
\
} while (0)
If you look at what dumpopaque does, you can see that it does what's
expected. I can even tell you that it does it correctly (I checked,
and anyway all C-accessible variables would explode if is wasn't
working, so it can't be that bad. I mean, XEmacs would not start at
all).
So The Truth is Somewhere Else(c)(r)(tm). You found (thanks, that's
exactly the starting point needed) a discrepancy of behaviour in the C
code between pdump and normal. If you check a bit deeper you can see
that is the pdump case the code never goes near the case
SYMVAL_BOOLEAN_FORWARD: for the symbol indent-tabs-mode. So let's
fish out the symbol and compare between pdump and normal:
With --pdump:
In *scratch* "(setq indent-tabs-mode nil)^C"
Program received signal SIGINT, Interrupt.
0x403c815e in __select () from /lib/libc.so.6
(gdb) break oblookup
Breakpoint 1 at 0x82001ce: file
/home/galibert/xemacs/xemacs-21.2/src/symbols.c, line 348.
(gdb) cond 1 size==16
(gdb) cont
Continuing.
In *scratch* "^J" (aka, execute line)
Breakpoint 1, oblookup (obarray=1079809524, ptr=0x85dc1ac
"indent-tabs-mode", size=16) at
/home/galibert/xemacs/xemacs-21.2/src/symbols.c:348
(gdb) finish
Run till exit from #0 oblookup (obarray=1079809524, ptr=0x85dc1ac
"indent-tabs-mode", size=16)
at /home/galibert/xemacs/xemacs-21.2/src/symbols.c:348
0x81ffc90 in Fintern (string=139009716, obarray=1079809524) at
/home/galibert/xemacs/xemacs-21.2/src/symbols.c:216
216 object = oblookup (obarray, XSTRING_DATA (string), len);
Value returned is $1 = 1078992296
(gdb) pobj $1
$2 = (Lisp_Symbol *) 0x40501da8
$3 = {lheader = {type = 4, mark = 0, c_readonly = 0, lisp_readonly = 0},
next = 0x0, name = 0x404837a0, value = 1079230256,
function = 1078786304, plist = 1079592596}
Symbol name: indent-tabs-mode
(gdb) pobj $2->value
$4 = (struct symbol_value_buffer_local *) 0x4053bf30
$5 = {magic = {lcheader = {lheader = {type = 3, mark = 0, c_readonly = 0,
lisp_readonly = 0}, next = 0x857f7c0, uid = 7892, free = 0},
value = 0x2, type = SYMVAL_BUFFER_LOCAL}, default_value = 1079009624,
current_value = 1079010440, current_buffer = 1079010440,
current_alist_element = 1079010440}
(gdb) pobj $4->default_value
$6 = (Lisp_Symbol *) 0x40506158
$7 = {lheader = {type = 4, mark = 0, c_readonly = 0, lisp_readonly = 0},
next = 0x0, name = 0x40487410, value = 1079009624,
function = 1078786304, plist = 1079010440}
Symbol name: t
(gdb) pobj $4->current_value
$8 = (Lisp_Symbol *) 0x40506488
$9 = {lheader = {type = 4, mark = 0, c_readonly = 0, lisp_readonly = 0},
next = 0x0, name = 0x404876d0, value = 1079010440,
function = 1078786304, plist = 1079010440}
Symbol name: nil
Hum hum, so the symbol is in fact a buffer local one. And there is no
sign of SYMVAL_BOOLEAN_FORWARD anywhere. Ok, let's try without pdump:
(gdb) pobj $4->default_value
$6 = (Lisp_Symbol *) 0x832bc4c
$7 = {lheader = {type = 4, mark = 0, c_readonly = 0, lisp_readonly = 0},
next = 0x0, name = 0x832b44c, value = 137542732, function = 137166796,
plist = 137542708}
Symbol name: t
(gdb) pobj $4->current_value
$8 = (struct symbol_value_forward *) 0x82b2d58
$9 = {magic = {lcheader = {lheader = {type = 0, mark = 1, c_readonly = 1,
lisp_readonly = 1}, next = 0x0, uid = 0, free = 0}, value = 0x8306c1c,
type = SYMVAL_BOOLEAN_FORWARD}, magicfun = 0}
We have a winner. The symbol is hosed. Time to have a look at the
struct symbol_value_buffer_local declaration and its description.
Just go in symeval.h and read it. I obviously didn't have my brain
engaged when I read it a long time ago, given what I wrote as a
description:
static const struct lrecord_description
symbol_value_buffer_local_description[] = {
{ XD_LISP_OBJECT, offsetof (struct symbol_value_buffer_local,
default_value) },
{ XD_LO_RESET_NIL, offsetof (struct symbol_value_buffer_local,
current_value), 3 },
{ XD_END }
};
Putting current_value systematically to Qnil? Yeah right. Once the
bug found, it was a Simple Matter Of Programming ;-)
The bugfix is in xemacs-patches.
OG.
PS: Don't worry, it's just that the portable dumper has reached a
level of stability good enough that bugs are at least that perverted.