10:47 a.m.
>>>> "Nix" == nix
<nix(a)esperi.demon.co.uk> writes:
Nix> The real problem with GCPROs is that they are nasty to maintain; the
Nix> type-accuracy they provide is good. It occurs to me that I could make
Nix> GCPROs maintainable by making all the alloc_*() routines do an automatic
Nix> GCPRO (well, do the same thing to the gcpro roots that GCPRO now does),
The alloc routines know what they're allocating, but they don't know
about the references to the memory being allocated. Unless you do
something very radical, you have to gcpro the address of the
Lisp_Object on the stack that is about to receive the pointer to the
allocated memory. This means that Fcons can't gcpro anything.
Perhaps I'm missing something.
The standard tricky thing about implementing a preprocessor that adds
gcpros automatically is that sometimes the Lisp_Objects that have to
be protected aren't even declared - they're temporaries.
For example
return Fcons_user1 (Fcons_user2 (Fcons (Qnil, Qnil)));
This has to be expanded into
{
Lisp_Object x = Qnil, y = Qnil, z = Qnil;
struct gcpro gcpro1, gcpro2, gcpro3;
GCPRO3 (x, y, z);
x = Fcons (Qnil, Qnil);
y = Fcons_user2 (x);
z = Fcons_user3 (y);
UNGCPRO;
return z;
}
(this example could of course be optimized - but this is the obvious translation)
(this example also shows how incredibly ugly gcpros are)
The current source code does a lot fewer GCPROs than the above,
because of detailed knowledge of various functions (can they gc, or
could they return a fresh object). It is possible to determine this
kind of information from a global analysis of the source code, but it
won't be easy. In particular, the C preprocessor will tend to make
the source more resistant to automated understanding by your gcpro
preprocessor. E.g.
#ifdef MULE
#define FROB(x) Fcons_user1 (Fcons_user2 (Fcons (x, x)))
#else
#define FROB(x) Fcons_user3 (Fcons_user4 (Fcons (x, x)))
#endif
...
return FROB(Qnil);
Nix> and could write a fairly simple preprocessor that takes (one file of)
Nix> the XEmacs C code and spits out a (transient) source file with
Nix> UNGCPRO-alikes inserted at the end of each block where they are
Nix> allocated. That would let us dump visible GCPROs in the source yet keep
Nix> type-accuracy...
> I've taken a look at the Boehm gc in the past, considering
its
> possible use in XEmacs. I rejected using it because:
> - it's non-portable (reading its portability layer is frightening)
Nix> That's my biggest worry, too. I don't really mind which GC I go to,
Nix> really; my primary goals are zapping the GCPRO construct as it currently
Nix> exists (although keeping its type-accuracy would be nice) and getting
Nix> something with better VM behaviour. Both of these can be done without
Nix> actually replacing the GC; it just seemed simpler to do that.
> However, the gcc project has recently started using boehm gc.
Since
> gcc is very portable, they must be making any necessary portability
> improvements to boehm gc as they go.
Nix> Not yet. There are many architectures that GCC works on that libjava and
Nix> parts of libobjc (the primary users of that gc) do not work on, and the
Nix> boehm-gc in the GCC tree is a rather aged implementation.
Bad news.
Nix> When version 6 comes out Hans has said he will move to making the GCC
Nix> tree's copy of the collector the master copy; if I'm still using the
Nix> boehm-gc in XEmacs by then, I'll migrate it to that version.
Very good news.
> Hmmm. It seems to me mark() has to examine most of the heap
anyways.
> Unless you do a lot of work, boehm gc might end up examining even more
> memory than the existing mark. Most of the guts of existing
> Lisp_Objects are other Lisp_Objects, so there isn't a whole lot of
> scope for GC_malloc_atomic() to be a big win.
Nix> This isn't true of anything except for cons cells and other sequences,
Nix> and things like hash tables, is it? For `leaf' objects (string data &c),
Nix> a conservative collector like Boehm's would wind up tracing them for
Nix> pointers they couldn't possibly contain pointers to anything.
Nix> (It's true that the gains from this bit over the *existing* allocator
Nix> would be marginal; it reduces the negative effects of using a
Nix> conservative collector substantially, though. It's a tradeoff; GCPRO
Nix> versus the occasional malloc_atomic()... but the more I think about it
Nix> the more it seems to me that a preprocessor that automatically inserts
Nix> UNGCPROs is the right way to go; type-accuracy *and* maintainability,
Nix> wow ;) )
Sorry, I meant there's little scope for a Boehm mark() to be better
than the existing mark() because the existing one, via lisp object
type mark methods, already knows exactly which object components might
contain other Lisp_Objects. The mark bitmap won't help you much
during the mark phase, in fact it ought to make things worse.
The place where the Boehm gc shines performance wise is not during gc
at all, but in the fact that gcpros can be dispensed with entirely
while the mutator runs. Normally the advantage of mark&sweep over
reference counting is: not slowing down the mutator. But gcproing does
slow down the mutator. But I really don't know how much overhead we
have from GCPROing, and it's hard to measure. My random guess is that
10% of the runtime of lisp function calls that do no work (i.e. return
immediately) is taken up with gcproing.
> we can also put the mark-bits into a bitmap without complete
> boehmification. Understanding and extracting that code from boehm gc
> would be a much smaller and still very instructive project.
Nix> True. I think, for the first implementation, I'll go with
Nix> -- Richard's infrastructure (general modifications), forward-ported to
Nix> 21.4; they look praiseworthy, and the forward-porting job doesn't
Nix> look too enormous
Nix> -- A bitmapped version of the current GC
Nix> -- and, if I can get it to work --- and I damned well will ;) --- a
Nix> GCPROizing preprocessor, so we can dump *visible* evidence of GCPRO
Nix> (that being what does the harm; we can forget it exists completely
Nix> if it is automatically maintained!)
Harder than it looks, as I try to point out above. I'll be very
impressed if you can produce a reliable GCPROizing preprocessor.
I think you'll have to run the source through the C preprocessor
first, which means your gcproizer has to run every build. Using the
standard Unix utilities, this will be hard (i.e. you'll want to use
something like perl or python). Most free software projects don't
have such dependencies for non-maintainer-mode, but I think the time
might be right to introduce such a dependency for xemacs.
Nix> (I might even be able to make it so that only one GCPRO/UNGCPRO
Nix> operation needs to be done for an entire stack frame, because the
Nix> preprocessor knows exactly what Lisp_Objects have been allocated in
Nix> a given frame, so it can insert a single operation that does all the
Nix> work...)
I like your plan very much.
ObAdvice: Have you read the internals manual section that deals with gc?