Case for clisp

I haven't included my own contributions, but they are not really significant. Reginald's claims about Scheme are not my own, and they might well be incorrect -- I've included them for context only. The most important arguments /pro/ clisp are by Bruno Haible. -- Hrvoje Niksic <hniksic(a)srce.hr&gt; | Student at FER Zagreb, Croatia --------------------------------+-------------------------------- Lost in Static 18 / And the storm is closing in now ------- Start of forwarded message ------- Date: Thu Jul 2 11:17:27 1998 From: Various Subject: Digested Articles Topics: Re: perlemacs technical discussion Re: perlemacs technical discussion clisp, xemacs Re: clisp, xemacs Re: clisp, xemacs ---------------------------------------------------------------------- Date: 06 Feb 1998 17:25:28 -0800 From: perry(a)zso.dec.com (Reginald S. Perry) To: Hrvoje Niksic <hniksic(a)srce.hr&gt; Subject: Re: perlemacs technical discussion Message-Id: <sxlsopw1a0n.fsf(a)yakko.zso.dec.com&gt; Content-Type: text/plain; charset=US-ASCII Hrvoje Niksic <hniksic(a)srce.hr&gt; writes: Question: Did anyone consider using something like Bruno Haible's Clisp? Or do people want native compilation? - Reggie - ------------------ Reginald S. Perry e-mail: perry(a)zso.dec.com Digital Equipment Corporation Performance Manager Group http://www.UNIX.digital.com/unix/sysman/perf_mgr/ The train to Success makes many stops in the state of Failure. ------------------------------ Date: Sun, 08 Feb 1998 02:28:57 -0800 From: "Reginald S. Perry" <perry(a)zso.dec.com&gt; To: Hrvoje Niksic <hniksic(a)srce.hr&gt; Cc: perry(a)rust.unknown (Reginald S. Perry), Steven Baur <steve(a)xemacs.org&gt; Subject: Re: perlemacs technical discussion Message-Id: <9802081029.AA10961(a)yakko.zso.dec.com&gt; Content-Type: text/plain; charset=US-ASCII Sure. First the Lisp FAQ also has info so I will aproach this answer purely from my point of view. Clisp is a GPLed implementation of Common Lisp. Its flavor is that it has a CLTL1(Commom Lisp, the Language Vol. 1) feel but they have implemented a large portion of the passed votes in CLTL2 so they are sort of ANSI. There are a couple of cool things about Clisp. First its executable size and memory footprint is suprisingly small for a full-blown lisp implementation. Here are some stats from my Alpha box running DU 4.0B yakko> ps l -p 24127 -m UID PID PPID CP PRI NI VSZ RSS WCHAN S TTY TIME COMMAND 9158 24127 11723 0 42 0 4.65M 888K tty I + ttyq 00:00.05 /usr/local/lib/lisp/lisp.run -M /u yakko> ls -l /usr/local/lib/lisp total 3112 - rwxr-xr-x 1 root system 1785856 Jan 6 12:06 lisp.run* - rw-r--r-- 1 root system 1399572 Jan 6 12:06 lispinit.mem In addition, He has gone through lots of trouble to tune the implementation. It has generational garbage collection and hand tuned assembly code for x86 and a couple of other implementations. It also has short-float and long-float in which the long float has a variable mantissa length, which is a multiple of 16 (or 32, depending on the word size of the processor). These combinations mean that although the system uses byte compilation, some operations end up being suprisingly fast. The system runs on a lot of different machines/OSes including NT, OS/2, and Amigia. It would seem like a nice base from which to start although I am by no means a lisp of XEmacs expert. You might want to talk to Bruno (Bruno Haible [haible(a)ilog.fr]) I bet he would have some good suggestions. So while I agree that native compilation is the best, Clisp may be a good portable solution for people who dont have access to Unix to run CMUCL, Linux to run Allegro CL, dont have the cash to get Commom Lisp on Windows or want to use a CL based XEmacs but is running on OS/2 or some other system that commercial lisp vendors wont port their wares to. Anyway, I will attach this message Bruno sent to the mailing list pointing to the new location for the sources and binaries. - Reggie Hello all, The home location of CLISP moves. This means: * The mailing list was <clisp-list(a)ma2s2.mathematik.uni-karlsruhe.de&gt; and is now <clisp-list(a)clisp.cons.org&gt;. The address for administrative requests (subscribe, unsubscribe) for clisp-list was <listserv(a)ma2s2.mathematik.uni-karlsruhe.de&gt; and is now <listproc(a)clisp.cons.org&gt;. * The FTP server was ftp://ma2s2.mathematik.uni-karlsruhe.de/pub/lisp/clisp/ and is now ftp://ftp2.cons.org/pub/lisp/clisp/. Old versions of CLISP and binaries older than June 1995 (for 386bsd, acorn, atari, coherent386, sco, sequent-ptx, sgi-irix4, sgi-irix5, sun3-sunos4, sun386, sun4-sunos51, sun4-sunos52, sun4-sunos53, unixware) are only available from http://clisp.cons.org/~haible/clisp-archive/. * There is now a home page at http://clisp.cons.org/~haible/clisp.html. Please update your URLs. All of you who were subscribed to the old clisp-list on December 9, 1997, are now subscribed to the new clisp-list. You'll find below the message which you would have gotten if you had subscribed to the new clisp-list manually. Please read it, and save it for the day when you want to unsubscribe. Many thanks to Martin Cracauer who owns clisp.cons.org and to Walnut Creek CDROM who owns ftp2.cons.org. Bruno Haible <haible(a)clisp.cons.org&gt; =========================== welcome message ============================= You have been added to list clisp-list(a)clisp.cons.org. The system has recorded your address as ................... and in order for your messages to get posted (if the list accepts postings), you will have to send them from this address. If a message is ever rejected, please contact the list's owner: haible(a)ilog.fr For information on this service and how to use it, send the following request in the body of a mail message to listproc(a)clisp.cons.org: HELP All requests should be addressed to listproc(a)clisp.cons.org. clisp-list(a)clisp.cons.org This mailing list is for users of the CLISP implementation of Common Lisp by Bruno Haible and Michael Stoll. It is the proper forum for questions about CLISP and also bug reports, compatibility issues, problems of CLISP on specific platforms, enhancement of CLISP, miscellaneous comments, etc. The clisp-list is managed internally by the Boston University "listproc" program. To be added or removed from the clisp-list or to retrieve past articles posted to the clisp-list please send mail to: listproc(a)clisp.cons.org If you include the word "help" as the message body, "listproc" will return a list of valid commands and options. To have messages forwarded to everyone else on the clisp mailing list send them to: clisp-list(a)clisp.cons.org To retrieve the past articles posted to the clisp list, send mail to listproc(a)clisp.cons.org including the three words "get clisp-list mailing-list-archive" as message body. To unsubscribe from the clisp mailing list, send mail to listproc(a)clisp.cons.org including the two words "unsubscribe clisp-list" as message body. Please save this message for the day you want to unsubscribe. About CLISP: CLISP is a Common Lisp implementation by Bruno Haible of Karlsruhe University and Michael Stoll of Munich University, both in Germany. It mostly supports the Lisp described in "Common LISP: The Language (2nd edition)" and the ANSI Common Lisp standard. It runs on microcomputers (DOS, OS/2, Windows NT, Windows 95, Amiga 500-4000, Acorn RISC PC) as well as on Unix workstations (Linux, SVR4, Sun4, DEC Alpha OSF, HP-UX, NeXTstep, SGI, AIX, Sun3 and others) and needs only 2 MB of RAM. It is free software and may be distributed under the terms of GNU GPL. The user interface comes in German, English, French and Spanish. CLISP includes an interpreter, a compiler, a large subset of CLOS, a foreign language interface and a socket interface. An X11 interface is available through CLX and Garnet. Available by anonymous ftp from ftp2.cons.org in the directory /pub/lisp/clisp. For more information, look at http://clisp.cons.org/~haible/clisp.html. ------------------------------ Date: Mon, 9 Feb 1998 15:14:04 +0100 (MET) From: Bruno Haible <haible(a)ilog.fr&gt; To: Hrvoje Niksic <hniksic(a)srce.hr&gt; Subject: clisp, xemacs Message-Id: <199802091414.PAA08897(a)halles.ilog.fr&gt; Hello, Reginald Perry told me that he had suggested using CLISP inside XEmacs, for replacing the existing bytecode engine, I assume. Some technical points: * CLISP is pretty close to the ANSI Common Lisp standard (except for some exotic features which would have a noticeable efficiency penalty). * CLISP comes with its own bytecode compiler and bytecode interpreter engine. As with Emacs Lisp, interpreted functions are debuggable, and bytecode is not (because all source has been thrown away). * CLISP is very portable. Ports include Amiga, OS/2, and Win32. No such unportable things like `unexec'. * CLISP is very robust. All functions perform strict argument type checking. It's very difficult to make it crash, except by extending it with buggy C code. * CLISP has been in use continuously by about 1000 people since 1993, 200 of them on the mailing list. This has shaked out even the smallest bugs. * CLISP has a very efficient memory management and garbage collector: - The GC is compacting. Nowadays, after an uptime of a week, XEmacs' memory size grows to 40 MB. This would go away. - The full GC of a 1 MB world takes 0.1 seconds on a Sparc 10, much less than XEmacs' GC. Moreover, most GCs collect only the youngest generation of objects are thus even faster. - The implementation of this uses all all kinds of mmap and mprotect trickery. * I have started implementing multithreading into CLISP (a real one, based on kernel threads in the case of Linux and Solaris). * A little drawback: Dealing with Lisp objects in C code is not easy because they must be stored on the Lisp object stack during function calls, to protect against GC. (See file doc/extend.txt in the CLISP distribution.) * Integration: CLISP has once been integrated with a computer algebra system for number theory. The sources are not yet well prepared for this task, though. Please don't hesitate to ask me questions. I would very much like CLISP to be useful in this way. Greetings, Bruno Haible <haible(a)clisp.cons.org&gt; http://clisp.cons.org/~haible/clisp.html ------------------------------ Date: Tue, 10 Feb 1998 20:35:47 +0100 (MET) From: Bruno Haible <haible(a)ilog.fr&gt; To: Hrvoje Niksic <hniksic(a)srce.hr&gt; Cc: Steven Baur <steve(a)xemacs.org&gt; Subject: Re: clisp, xemacs Message-Id: <199802101935.UAA27926(a)halles.ilog.fr&gt; References: <199802091414.PAA08897(a)halles.ilog.fr&gt; <kigafbz7mfk.fsf(a)jagor.srce.hr&gt; Hello Hrvoje, Things like CLOS, LOOP, conditions, generic streams, sockets and FFI can be easily removed from clisp, but I think these are really added values (especially CLOS and conditions) which distinguish Common Lisp from other, less object-oriented, Lisps. CLISP is not yet prepared for omitting other stuff, but you just need to tell me what you want not to have, and I can add configuration options for them. There is some documentation in German in the first 10% of compiler.lsp. Otherwise the list of instructions is in bytecode.d, and the actual bytecode interpreter is at the end of eval.d. Basically, it's a stack machine, with multiple values in the Top Of Stack (as required by CL). There are special bytecodes for special language features (unwind-protect and handler-bind, for example). The set of bytecodes has been optimized so that compiled programs take as few space as possible. The bytecode is platform-independent. I agree that it would be nice; I have thought about a JIT-compiler which would transform the bytecode into machine code, but it does not yet exist. I cannot imagine a portable dumper: The only portable interface for creating executables is the system's linker. CLISP puts its data into mmap'ed memory images (sort of self-made shared libraries). Generational GC by itself does not consider the issue of virtual memory, but in CLISP's generational GC I take special measures so that GC has not to touch memory pages (every page has a small "todo" cache associated with it, and this cache is emptied only when the page is known to be in-core). I think it'd be easy to implement the gc_protect() function currently used by the Emacs C sources with CLISP too: object xx; gc_protect(xx); ... lots of code... gc_unprotect(xx); return; It'd be nice if we could make up a "wish list" of things which have to be modified in CLISP in order to help integration into XEmacs. - gc_protect - main program not called main() - better documentation on what issues? - autoloading as a primitive? - which parts to remove? - adding efficient low-level data types for text - ... Greetings Bruno ------------------------------ Date: Tue, 10 Feb 1998 22:13:48 +0100 (MET) From: Bruno Haible <haible(a)ilog.fr&gt; To: Hrvoje Niksic <hniksic(a)srce.hr&gt; Cc: Steven Baur <steve(a)xemacs.org&gt; Subject: Re: clisp, xemacs Message-Id: <199802102113.WAA22576(a)halles.ilog.fr&gt; References: <199802091414.PAA08897(a)halles.ilog.fr&gt; <kigafbz7mfk.fsf(a)jagor.srce.hr&gt; <199802101935.UAA27926(a)halles.ilog.fr&gt; <kigd8gv2ore.fsf(a)jagor.srce.hr&gt; The difference in size comes definitely from compiling to bytecode, not machine code. Only 5 people on the world know how to compile CMUCL :-) That's fine with me. Funcalls (from bytecode to bytecode) costs 3 C function calls: interpret_bytecode_ -> funcall -> funcall_closure -> interpret_bytecode_ I've never had the impression that clisp's function calls were particularly slow. For example, the last time I did Gabriel's benchmarks, on a PC Pentium 133 Mhz, Linux 1.2.13, the results were: DIV2iter 0.33 DIV2recur 0.43 (mapcar #'some-predefined-function ...) has an overhead of 1 call to `mapcar'. (mapcar (lambda ...) ...) is optimized by inlining the lambda (and by special bytecode instructions which occur frequently in this context). CLISP's weakness vs. machine-code compiling Lisps are structure accesses. A function call to a built-in function (with 3 arguments) in clisp, but only a single machine instruction in machine code. That's the price of full runtime type safety. Great! Finally... (Note that this wouldn't be a solution for clisp, because normal users on machines without a linker [naked Solaris] may want to save memory images in clisp. For XEmacs this is acceptable, because the saving occurs only as part of the build process.) I'll add similar capabilities to clisp. They are not present yet. Greetings, Bruno ------------------------------ End of forwardcTBCoX Digest *************************** ------- End of forwarded message -------