Re: Forthcoming revisions to the garbage collector; crazy huge hack
>>>> "Nix" == nix
<nix(a)esperi.demon.co.uk> writes:
Nix> Oh, FWIW the preprocessor should be able to handle just about anything,
Nix> 'cos I'm cheating. I'm taking c-parse.y and cpplex.c from GCC, and
Nix> tearing them down into something that can unambiguously spot C blocks
Nix> (`stmts_and_decls'), assignments (`expr_no_commas'), Lisp_Object
Nix> variable declarations (`decl' nodes), and temporaries. Everything else
Nix> that I can throw away I will; we don't need a complete C parser, let
Nix> alone the Objective C parts (sorry, Ovidiu ;) ).
We already have a makefile rule for running the C preprocessor:
make foo.i
should work (the rule might not be portable to every C compiler).
You'll want to run your C parser on the output of that.
Nix> Fun. (But if this were C++ it would not be fun, it would be torment...)
Ahhh, but this _is_ C++.
The production xemacs I use is compiled by a C++ compiler. And the
road is open to someone to take xemacs in a purely C++ direction, for
example to create Qt-xemacs.
> Sorry, I meant there's little scope for a Boehm mark() to be
better
> than the existing mark() because the existing one, via lisp object
> type mark methods, already knows exactly which object components might
> contain other Lisp_Objects. The mark bitmap won't help you much
> during the mark phase, in fact it ought to make things worse.
Nix> Hmm. Why is that? The current garbage collector has appalling VM
Nix> behaviour; every Lisp_Object is sucked back into memory, even if they
Nix> cannot contain other Lisp_Objects. If a mark bitmap is used, a
Nix> Lisp_Object will only be accessed for pointer tracing; if it cannot
Nix> contain another Lisp_Object, it won't be touched.
Oh, you mean that Lisp object types not containing any Lisp_Objects as
members are accessed by the current gc to access the mark bit in the
lrecord_header? That's true, but currently, also the type and the
c_readonly members need to be accessed. If you move *only* the mark
bits to a separate bitmap, you still need to determine the type of
each object. And even if you don't have to read the lrecord_header of
lisp objects that don't reference other lisp objects, I think you
won't win very much, because MOST lisp objects do reference other lisp
objects. You can look at the output of (garbage-collect) to figure
out what the potential savings are.
Nix> I'll admit that I haven't instrumented the existing gc to find out what
Nix> percentage of objects are accessed only to set the mark bit; I'll do so
Nix> as soon as this cold has worn off. (I'll feel a right idiot if the
Nix> answer is something like 1%, too...)
Programmers are notoriously poor at estimating performance questions.
That also means you shouldn't trust my pessimism above.
> The place where the Boehm gc shines performance wise is not
during gc
> at all, but in the fact that gcpros can be dispensed with entirely
> while the mutator runs. Normally the advantage of mark&sweep over
> reference counting is: not slowing down the mutator. But gcproing does
> slow down the mutator. But I really don't know how much overhead we
> have from GCPROing, and it's hard to measure. My random guess is that
> 10% of the runtime of lisp function calls that do no work (i.e. return
> immediately) is taken up with gcproing.
Nix> However, a (not huge) general slowdown like that is hard for humans to
Nix> spot. What humans definitely *do* spot (well, I know I've spotted it,
Nix> and so have friends of mine, new XEmacs hands and old) is the total
Nix> stoppage we get whenever GCing runs and the world freezes except for the
Nix> hammering of the disk :( if XEmacs is totally in swap, it can be frozen
Nix> for a minute or more while GC runs.
I don't see this happen. Your machine has to have enough memory to
run xemacs - it's a memory hog.
Better ways to fix this (the "resident set size problem") are:
- load less lisp.
- load lisp more lazily. (or gasp, unload lisp after it's not been
used for a while... think of loading lisp as "paging in")
- generational gc.
Nix> Given Moore's Law and the fact that machines with small amounts of
Nix> memory are getting steadily harder to find and steadily harder to run
Nix> late versions of XEmacs on (and probably don't see many upgrades
Nix> anyway), I am happy to *increase* the number of GCPROs substantially, to
Nix> ensure that we can freely modify arbitrary functions without triggering
Nix> GC hell. After all, GCPROs are really quite efficiently implemented;
You're complaining about how slow xemacs is, and also "it's ok for me
to make it a little slower".
Nix> it's not as if they call malloc() or something like that. (As a
Nix> *structure*, the gcprolist is praiseworthy. It's just the manual way
Nix> it's kept updated that's disgusting and restrictive.)
Offhand, it bothers me that most struct gcpro protect just one
variable, so much of the time the initialization and reading of the
nvars member is wasted. But I don't know whether this can be easily
optimized away. Could one have a gcprolist with only `simple'
variables, and a multiple_gcprolist that can protect arrays of
variables?
Nix> (Then I read of GCPRO and felt quite ill...)
There are other health-sapping critters lurking in the xemacs source code...