7:04 p.m.
Jerry James writes:
The word "stackless" appears to be a misnomer. Any idea
where that
came from?
From a particular implementation of Python called "Stackless
Python".
It's been around for many years, but Guido consistently refuses to
put
it into CPython, I don't know why offhand. It has a number of
supporters besides its author, though, and is well-debugged.
1. You have to choose between two painful alternatives.
a. [...]
b. A scheduler that only runs inside of well-defined functions, meaning
that a given stack can hog the CPU for arbitrary amounts of time.
It's not obvious to me that this wouldn't be a big win in XEmacs.
This isn't a realtime OS where you need to discipline badly-behaved
threads; it's a single-user application where judicious use of
cooperative multithreading might permit big wins from a few tweaks.
2. If you choose 1a, then you wind up doing all the work you would
have
to do for a "real" thread library anyway.
Agreed.
3. There are several mature, well-debugged thread libraries out
there. Why
ignore them and start the development and debugging process all over
again?
Because threads are hard. I could see using threads for certain
subsystems (eg, redisplay, and assuming Marcus and Olivier are up for
it, GC), but having true threads in the Lisp interpreter will be hard
(unless we're willing to replace the whole thing).
5. The world is going multi-core. Intel is talking about having
80-core
processors available in 2 years.
Why wait for Intel? You probably already have a couple lying around
(cf this month's ACM Queue---wow! an interview with Tom Duff and he's
not even the focus of attention? Yikes!)
I am interested in having a discussion on what it would take to
switch from our Reactor-based architecture to a multithreaded one.
Such a switch would have serious implications for the Lisp engine
and the garbage collector, for example.
Well, I have to admit that I will have very little useful to
contribute, but I'd be happy to watch and cheer from the sidelines!
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
9:03 p.m.
New subject: "Stackless" XEmacs?
On Wed, Apr 30, 2008 at 12:04 PM, Stephen J. Turnbull
<stephen(a)xemacs.org> wrote:
Jerry James writes:
> The word "stackless" appears to be a misnomer. Any idea where that
> came from?
From a particular implementation of Python called "Stackless Python".
It's been around for many years, but Guido consistently refuses to put
it into CPython, I don't know why offhand. It has a number of
supporters besides its author, though, and is well-debugged.
Interesting. I'll look that up and see what they've done.
> 1. You have to choose between two painful alternatives.
> a. [...]
> b. A scheduler that only runs inside of well-defined functions, meaning
> that a given stack can hog the CPU for arbitrary amounts of time.
It's not obvious to me that this wouldn't be a big win in XEmacs.
This isn't a realtime OS where you need to discipline badly-behaved
threads; it's a single-user application where judicious use of
cooperative multithreading might permit big wins from a few tweaks.
Yes, but we already do that! That's what the select()/poll() driven
event loop is all about: find out what I/O is ready to go, then run
the handler to drive it. There are some places where we don't follow
discipline and allow the process to block rather than defer activity
to a later iteration of the event loop. Fixing those would give us
the same benefits as this "stackless" architecture.
In effect, we keep information on the heap that would be sitting on
separate stacks in the "stackless" approach. We're more stackless
than the stackless approach!
> 3. There are several mature, well-debugged thread libraries
out there. Why
> ignore them and start the development and debugging process all over
> again?
Because threads are hard. I could see using threads for certain
subsystems (eg, redisplay, and assuming Marcus and Olivier are up for
it, GC), but having true threads in the Lisp interpreter will be hard
(unless we're willing to replace the whole thing).
Yes, threads are certainly hard. I'm not necessarily suggesting we
dive into using them. I'm suggesting that the stackless approach is
(a) a lot more work than fixing our discipline with respect to our
current architecture for about the same benefit; and (b) a lot less
work but with much less benefit than going to a threaded architecture.
> 5. The world is going multi-core. Intel is talking about
having 80-core
> processors available in 2 years.
Why wait for Intel? You probably already have a couple lying around
(cf this month's ACM Queue---wow! an interview with Tom Duff and he's
not even the focus of attention? Yikes!)
I haven't remembered to read Queue for months, since I stopped getting
the print issue. I'll have to dive online and read this issue.
> I am interested in having a discussion on what it would take
to
> switch from our Reactor-based architecture to a multithreaded one.
> Such a switch would have serious implications for the Lisp engine
> and the garbage collector, for example.
Well, I have to admit that I will have very little useful to
contribute, but I'd be happy to watch and cheer from the sidelines!
I've got too many irons in the fire right now. Let's postpone this
discussion for a few weeks.
--
Jerry James
http://loganjerry.googlepages.com/
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
3:56 a.m.
New subject: "Stackless" XEmacs?
Jerry James writes:
> It's not obvious to me that this wouldn't be a big win
in XEmacs.
> This isn't a realtime OS where you need to discipline badly-behaved
> threads; it's a single-user application where judicious use of
> cooperative multithreading might permit big wins from a few tweaks.
Yes, but we already do that! That's what the select()/poll() driven
event loop is all about: find out what I/O is ready to go, then run
the handler to drive it. There are some places where we don't follow
discipline and allow the process to block rather than defer activity
to a later iteration of the event loop. Fixing those would give us
the same benefits as this "stackless" architecture.
Well, my understanding is that stackless Python actually exposes those
"threads" to Python. I don't care how it's implemented, but I really
would like for (a) certain long-blocking applications like Gnus and VM
to more cooperative, and (b) to be able to get information about their
state, maybe, from another thread.
Yes, threads are certainly hard. I'm not necessarily suggesting
we
dive into using them. I'm suggesting that the stackless approach is
[no panacea].
Oh, of course not. I'm not pushing the stackless approach per se, I'm
just throwing out ideas. This one rang my chimes because (a) I have
immense respect for the Python development team and (b) because a 3rd
party brought it up in emacs-devel.
I've got too many irons in the fire right now. Let's
postpone this
discussion for a few weeks.
I'll assume that's a promise.
These damn Steves (Baur and Yegge, specifically) are giving me an
itch....
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
12:51 a.m.
New subject: "Stackless" XEmacs?
Jerry James wrote:
> > 1. You have to choose between two painful alternatives.
> > a. [...]
>
> > b. A scheduler that only runs inside of well-defined functions, meaning
> > that a given stack can hog the CPU for arbitrary amounts of time.
>
> It's not obvious to me that this wouldn't be a big win in XEmacs.
> This isn't a realtime OS where you need to discipline badly-behaved
> threads; it's a single-user application where judicious use of
> cooperative multithreading might permit big wins from a few tweaks.
Yes, but we already do that! That's what the select()/poll() driven
event loop is all about: find out what I/O is ready to go, then run
the handler to drive it. There are some places where we don't follow
discipline and allow the process to block rather than defer activity
to a later iteration of the event loop.
If your code is in the middle of a complex loop or recursion with a
lot of local state, it's a lot easier to call a hypthothetical "yield"
function than to turn your code inside-out so that it works as a
process filter.
The situation is analogous to converting a recursive-descent parser to
the pushdown automaton form created by e.g. yacc. The latter is a much
better fit for event-driven I/O, but who would write one by hand?
> Because threads are hard. I could see using threads for
certain
> subsystems (eg, redisplay, and assuming Marcus and Olivier are up for
> it, GC), but having true threads in the Lisp interpreter will be hard
> (unless we're willing to replace the whole thing).
Yes, threads are certainly hard. I'm not necessarily suggesting we
dive into using them. I'm suggesting that the stackless approach is
(a) a lot more work than fixing our discipline with respect to our
current architecture for about the same benefit; and (b) a lot less
work but with much less benefit than going to a threaded architecture.
AFAICT, the main issue is making dynamic scope and threads coexist.
Having a separate stack for each thread doesn't help much when all of
your local variables are just pointers to global symbols.
--
Glynn Clements <glynn(a)gclements.plus.com>
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
8:43 p.m.
New subject: "Stackless" XEmacs?
> 3. There are several mature, well-debugged thread libraries out
> there.
> Why
> ignore them and start the development and debugging process all
> over
> again?
Because threads are hard. I could see using threads for certain
Threads are nice :D
http://revolf.free.fr/beos/shots/shot_xemacs_beos_threads.001.png
Note they aren't very useful though, they are just spawn by the system
mostly, but I have to handle window redrawing in the main thread.
subsystems (eg, redisplay, and assuming Marcus and Olivier are up
for
it, GC), but having true threads in the Lisp interpreter will be hard
(unless we're willing to replace the whole thing).
Indeed being able to call the lisp engine from an arbitrary thread
would simplify the design of the BeOS and Haiku port, but it works not
so bad this way anyway :)
François.
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
6050
days inactive
6050
days old
4 comments
4 participants
participants (4)
-
François Revol -
Glynn Clements -
Jerry James -
Stephen J. Turnbull