Martin Buchholz <martin(a)xemacs.org&gt; writes: The *huge* disadvantage of using a custom preprocessor is that in that case your code is no longer C code, which means that it cannot be analyzed with the usual C tools such as lint or cflow, debugged with GDB, etc. If we're going through that route, then let's rather work on a full elisp-to-C translator and write as much stuff as possible in elisp.

On Wed, 23 May 2001 at 22:16:51 +0900, Martin Buchholz <martin(a)xemacs.org&gt; wrote: I've hacked gdb internals before. You could teach gdb about the enhanced language, but then all the developers would have to used a custom gdb. Yep. All good enough reasons to shoot this proposal down. I'll go sit back in my corner and stew some more on the module interface, then. Somewhat ironically, I actually considered this preprocessor a step towards an Elisp-to-C translator, using the source to be preprocessed as the target of that translator. I'll rethink that, too. -- Jerry James, who appreciates curmudgeons that prevent young hotheads from unnecessarily tilting with windmills

Raymond Toy writes: Several years ago there was a project over at one of the German universities to write a Common Lisp to C Compiler (CLiCC). The last version (v0.6.5) claimed that it conformed to the Common Lisp specification in CLtL1 plus CLOS. The web site for the compiler is still active: http://www.informatik.uni-kiel.de/~wg/clicc.html A couple of years ago I used it to compile several small stand alone apps. I think that if someone were to pick up CLiCC and finish it (bring it up to ANSI Common Lisp compliance), it would probably make a good mechanism for developing something like XEmacs almost completely in Common Lisp. I intend to do something like this eventually. I would have already started, but all the inline comments are in German and I don't have the time to learn German just so that I can read the comments. Craig

Craig> Raymond Toy writes: Jerry> Somewhat ironically, I actually considered this preprocessor a step Jerry> towards an Elisp-to-C translator, using the source to be preprocessed as Jerry> the target of that translator. I'll rethink that, too. Craig> http://www.informatik.uni-kiel.de/~wg/clicc.html Craig> A couple of years ago I used it to compile several small stand alone Craig> apps. I think that if someone were to pick up CLiCC and finish it (bring Craig> it up to ANSI Common Lisp compliance), it would probably make a good Craig> mechanism for developing something like XEmacs almost completely in Craig> Common Lisp. I intend to do something like this eventually. Craig> I would have already started, but all the inline comments are in Craig> German and I don't have the time to learn German just so that I can Craig> read the comments. Then why not use gcl then? It's still being actively worked on (probably in the context of maxima, though). And comments are in English. (This is what keeps me from hacking on clisp: German comments.) Making it fit with xemacs might not be so easy.... Ray

Craig> The last time I tried gcl it claimed CLtL1 compatibility which doesn't Craig> include CLOS. Has it moved closer to ANSI Common Lisp? I already I don't follow gcl but I don't think it's any closer. It's certainly still missing lots of ANSI stuff like defpackage, loop, etc. CLOS is still implemented with PCL (but then CMUCL uses PCL too). However as a backend for xemacs it's probably doesn't matter since xemacs code probably wouldn't use any of missing ANSI things. :-) Ray

Craig> Raymond Toy writes: >> >> Craig> The last time I tried gcl it claimed CLtL1 compatibility which doesn't Craig> include CLOS. Has it moved closer to ANSI Common Lisp? I already Craig> Then I would have a difficult time calling it "actively maintained". Well, Clisp claims CLtL2 and ANSI compatibility but I think there are some things that are specified by ANSI that will never be done because they think it's wrong. So is clisp not actively maintained? Even though there have been 2 releases in the last couple of months? Craig> If those things were available in the development environment then Craig> XEmacs might use them :-) Craig> Modes are much easier to implement with an object system. Craig> Zmacs implements each mode (both major and minor) as Craig> flavors (Zetalisp object system). It makes it easier to Craig> inherit things that are common to all modes. But if modes are required to use the object system who's going to port over all the old code? Not me. Craig> Packages would be helpful in segregating the different lisp packages. People already do this by prefixing stuff. Having packages won't change existing code. That's why CLtL1 isn't so bad. There's so much old stuff that no one really wants to port it all over, and no one wants to maintain a version for XEmacs and Emacs. I think that's hard enough already. Besides, Hvroje's message about Kyle's approach is the right solution to me. No user-visible changes to the lisp code, but potentially major changes in speed. (That's why I stopped using VM. Too slow to load up my 100 MB mail files. If VM becomes fast, I might switch back.) Ray

Hrvoje Niksic writes: CLiCC was designed to do something like this. The web site references a PostScript file that describes their method better than I am about to attempt. CLiCC actually compiles the Lisp to a set of specialized C macros which are then compiled by a normal C compiler. Perhaps CLiCC could be adapted to use the (X)Emacs bytecode C code. One major advantage of CLiCC is that it is designed to do global optimization for the entire application. This is something that C compilers rarely (if ever) do. I have found that simple is usually best. Craig

Hrvoje> My favorite strategy for elisp-to-C is the one (I think) originally Hrvoje> proposed by Kyle Jones some time ago: look at the opcodes that our Hrvoje> current compiler generates. Now look at the code in bytecode.c that Hrvoje> interprets it. A simple translator would simply take byte-compiled Hrvoje> code and translate it into C that looks like bytecode.c, but inlined Hrvoje> to directly *do* the stuff that P-code wants to do instead of Hrvoje> interpreting the P-code. Hrvoje> Although this looks simple-minded, I believe it would be a *huge* Hrvoje> speed win over the current virtual machine. First, the cost of Hrvoje> interpretation would be removed, which is by itself a big win -- Hrvoje> the huge `case' statement would finally be eliminated. Secondly, such Hrvoje> "unwound" C code would be processed by the C optimizer which would Hrvoje> remove the many many redundant checks and thus make the code run Hrvoje> really fast. Hrvoje> The advantage of this strategy is that it opts for huge speedup in a Hrvoje> *realistic* environment, i.e. it doesn't require rebuilding 100% of Hrvoje> XEmacs to work. Furthermore, it builds on the work already invested Hrvoje> in the byte-code compiler. I've done some thinking about this in the past. This is a very EASY translator to write compared to some of the other projects being considered. The hard part is integrating this into xemacs. Like figuring out the shared object loading mechanism across platforms. Should all the package lisp be compiled to .so at installation time, or should there be a JIT that optimizes hotspots at run-time (it would have to call the compiler transparently to do so, however). Don't expect it to be a huge win, however. All the optimized machine code will take up more memory than the bytecodes. And many elisp operations like (setq i (1+ i)) will not be compilable to C i++, but instead will still require expensive frobbing of the lisp symbol `i'. A function call like (foo) will still require the notoriously slow Ffuncall, even if foo is a subr, due to the dynamic nature of lisp. But this is a great hack.

Hrvoje> Raymond Toy <toy(a)rtp.ericsson.se&gt; writes: Hrvoje> Because it's highly questionable how useful Gcl is. Hrvoje> To be useful in short-term future, an elisp-to-C translator should not Hrvoje> try to reimplement XEmacs from scratch or to turn Elisp into Common Hrvoje> Lisp (or, for that matter, CLtL1). Hrvoje> My favorite strategy for elisp-to-C is the one (I think) originally Hrvoje> proposed by Kyle Jones some time ago: look at the opcodes that our Hrvoje> current compiler generates. Now look at the code in bytecode.c that Hrvoje> interprets it. A simple translator would simply take byte-compiled Hrvoje> code and translate it into C that looks like bytecode.c, but inlined Hrvoje> to directly *do* the stuff that P-code wants to do instead of Hrvoje> interpreting the P-code. Hrvoje> Although this looks simple-minded, I believe it would be a *huge* Hrvoje> speed win over the current virtual machine. This strategy has been used many times before. (In Gofer and MzScheme, to name two examples.) It has its benefits, but a significant speed gain is, perhaps surprisingly at first, usually not among them. (People tend to overestimate the time spent on syntax dispatch.) Of course, I may be underestimating the interpretive overhead in XEmacs, but I rather doubt it. -- Cheers =8-} Mike Friede, Völkerverständigung und überhaupt blabla

Hrvoje Niksic writes: You fail to mention the potentially very great difficulty that would be felt by other important C tools named Hrvoje, Martin, James, and others!!! Seriously, I had to support code written by a fellow that wrote his own preprocessor (for an assembly language, but that doesn't matter). Basically, I inherited like 10,000 lines of code that was essentially in its own language. There were great plusses of his preprocessor -- a single statement did all the work for a procedure call -- but it was his language, and so debugging it was impossible (no text books available!) A better way to change the world might be to write elisp skeleton macros (using skeleton.el) that makes it real easy to insert C procedure definitions frameworks with all the interface stuff for elisp calling there already. In my mode verilog-mode.el, I have a AUTO feature that does local analysis of verilog code and then updates declarations based on other data that is in the procedure definition. Perhaps something could be written in elisp that would look at a C procedure, that is written according to a defined structure (I.E., has comments inserted at key points that say /* AUTOARG */ ) and update the routine definition to make it conform to what is needed for the c - elisp interface. Also a good idea. -- //' Michael McNamara <mac(a)verisity.com&gt; _ // Sr VP Technology 650-934-6888 \ // Verisity Design 650-934-6801 FAX \// <http://www.verisity.com> 408-930-6875 Cell -------------------------------------------------------------- Get my verilog emacs mode from <http://www.verilog.com>

Perhaps not really the same thing, but there's a preprocessor in EcoLisp (Embeddable Common Lisp) that might be interesting to have a look at. Have you seen it? I've got EcoLisp sources tucked away if anyone wants them.