7:59 a.m.
Hallo,
What have you done to fontlocking?
I switched from XEmacs 21.1... to 21.4.5 and my fontlocking slowed down by a
foctor like 10. In the stable Version files just opened now I've got to wait
for about 10 to 15 seconds for some ordinary files. The fancy progress bar is
only cold comfort for this annoying behaviour.
Also lazy shot is no real alternative, because then all scrolling and
isearching becomes slow.
Please make it fast again.
Norbert
10:42 a.m.
>>>> "Norbert" == Norbert Schirmer
<norbert.schirmer(a)web.de> writes:
Norbert> Hallo,
Norbert> What have you done to fontlocking?
Norbert> I switched from XEmacs 21.1... to 21.4.5 and my
Norbert> fontlocking slowed down by a foctor like 10. In the
Norbert> stable Version files just opened now I've got to wait for
Norbert> about 10 to 15 seconds for some ordinary files. The fancy
Norbert> progress bar is only cold comfort for this annoying
Norbert> behaviour. Also lazy shot is no real alternative,
Norbert> because then all scrolling and isearching becomes slow.
Hi Norbert!
If necessary/possible, please install the latest net-utils package (>=
1.23) and then use
Help->Send Bug Report...
or
M-x report-emacs-bug
as suggested in
http://www.xemacs.org/Debug/index.html
This will tell us enough about your XEmacs installation so that we
can help you.
You don't even state the platform you're on.
Consider installing the latest versions of all installed XEmacs
packages first.
Perhaps you're missing some .elc files and use .el instead?
Good Luck,
Adrian
Norbert> Please make it fast again.
Norbert> Norbert
--
Adrian Aichner
mailto:adrianï¼ xemacs.org
http://www.xemacs.org/
9:05 a.m.
Hi,
I have to admit we're having the same problem here in the office, I used
to maintain a list of latest 21.1/2/4/5 installed, and we all noticed
some very annoying speed decrease at least in java (and jde when we
tried it) mode with font-lock for files over 5000 lines.
We have some files with around 10000 lines of java, and pressing tab to
get to the next line can take up to 5 seconds, and never less than 2.
So we all switched back to 21.1 as I did not have time to look into
that. I can try to build the latest of the latest (althought I'm not
sure anymore if it's 21.4 or .5 i need these days) and try it and
send the Installation file here if you want.
--
# Stef Epardaud, # There is no limit to the power of computing ...
# Java Defeater # ... except mankind maybe ?
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
9:56 a.m.
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> I can try to build the latest of the latest (althought I'm
Stef> not sure anymore if it's 21.4 or .5 i need these days) and
Stef> try it and send the Installation file here if you want.
Please do. I'd prefer you do it with 21.4.6 rather than the 21.5
branch. Ben is about to merge in about 70,000 lines of code to 21.5,
and there's just no way we'd find anything in that.
Please try profiling (1) the font-lock operation itself and (2) the
movement command that is causing problems. C-h f profile for more
information.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
10:54 a.m.
On Wed, Feb 06, 2002 at 06:56:57PM +0900, Stephen J. Turnbull wrote:
Please do. I'd prefer you do it with 21.4.6 rather than the
21.5
branch. Ben is about to merge in about 70,000 lines of code to 21.5,
and there's just no way we'd find anything in that.
alright, will do that asap.
Please try profiling (1) the font-lock operation itself and (2) the
movement command that is causing problems. C-h f profile for more
information.
that too.
btw, i seem to recall you're working on mule, right?
i'm having some problems with 'common lisp' with utf-8 encoding, somehow
the displaying of a utf-8 file causes me a major problem, and i've had a
hard time telling xemacs how to use a unicode font. is there realy
support for utf-8 yet or is it still balancing between some packages and
mule. is there a howto to get started or an faq entry ?
thanx a lot.
--
# Stef Epardaud, # There is no limit to the power of computing ...
# Java Defeater # ... except mankind maybe ?
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
1:08 p.m.
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> alright, will do that asap.
Muchas gracias.
Stef> btw, i seem to recall you're working on mule, right?
Yes.
Stef> i'm having some problems with 'common lisp' with utf-8
Stef> encoding, somehow the displaying of a utf-8 file causes me a
Stef> major problem,
Install Mule-UCS from packages, put
(require 'un-define)
(set-coding-priority-list '(utf-8))
(set-coding-category-system 'utf-8 utf-8)
and you should be fine _with standard national fonts_ for all
character sets you use. It's not yet ï¼ nicode inside, it's still Mule
codes that basically encode the XLFD registry (ie, the national
standard) in each character.
Stef> and i've had a hard time telling xemacs how to use a unicode
Stef> font.
You can't do it directly, at least not without extreme hackery. Mule
doesn't have a real Unicode charset internally, so there's nothing to
bind a Unicode registry to. It would not be straightforward to
create, either, because Unicode is not ISO 2022-compatible. So you'd
have to translate it to multiple 96x96 pages. You can run -nw with
(set-terminal-coding-system 'utf-8) if you really want the Unicode
font for some reason.
It should be possible to do this right pretty soon once Ben merges his
new stuff. So I'm going to have to say "stick with Mule-UCS for the
moment."
Stef> is there realy support for utf-8 yet or is it still
Stef> balancing between some packages and mule.
No, there is not really support for Unicode as a character set. There
is real support (Mule-UCS) for Unicode as an external encoding, but
it's not optimal.
Stef> is there a howto to get started or an faq entry ?
No, there should be. Er ... FAQ is done. In CVS 21.5. What
documentation there is is in the Mule-UCS Texinfo manual.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
1:46 p.m.
New subject: utf-8 questions [was: Re: fontlock-speed]
On Wed, Feb 06, 2002 at 10:08:34PM +0900, Stephen J. Turnbull wrote:
Muchas gracias.
from me, from me...
Stef> btw, i seem to recall you're working on mule,
right?
Yes.
my memory amazes me ;)
Install Mule-UCS from packages, put
(require 'un-define)
(set-coding-priority-list '(utf-8))
(set-coding-category-system 'utf-8 utf-8)
and you should be fine _with standard national fonts_ for all
character sets you use. It's not yet ï¼ nicode inside, it's still Mule
codes that basically encode the XLFD registry (ie, the national
standard) in each character.
done that, thanx, i could not figure it out from the docs in 21.4.
still i cannot display a utf-8 file i have which uses most of the
different characters available. some end up alrioght, and some others
are plain wrong. also some fonts are missing so i assume it will not be
able to display it even with that.
Stef> and i've had a hard time telling xemacs how to use a
unicode
Stef> font.
You can't do it directly, at least not without extreme hackery. Mule
doesn't have a real Unicode charset internally, so there's nothing to
bind a Unicode registry to. It would not be straightforward to
create, either, because Unicode is not ISO 2022-compatible. So you'd
have to translate it to multiple 96x96 pages. You can run -nw with
(set-terminal-coding-system 'utf-8) if you really want the Unicode
font for some reason.
i did that in a gnome terminal that just cat that file correctly, and
although it displays half of the characters correctly, the second half
is wrong.
It should be possible to do this right pretty soon once Ben merges
his
new stuff. So I'm going to have to say "stick with Mule-UCS for the
moment."
wow, that would be good, i'm sick of the second half of the office using
vi claiming how superior it is for utf-8 ;)
Stef> is there a howto to get started or an faq entry ?
No, there should be. Er ... FAQ is done. In CVS 21.5. What
documentation there is is in the Mule-UCS Texinfo manual.
alright, i admit i only looked at the faq for 21.4 and the C-h a ucs
list ;)
i will look further then.
thanx a lot for the info!
--
# Stef Epardaud, # There is no limit to the power of computing ...
# Java Defeater # ... except mankind maybe ?
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
4:13 p.m.
New subject: utf-8 questions [was: Re: fontlock-speed]
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> still i cannot display a utf-8 file i have which uses most
Stef> of the different characters available. some end up alrioght,
Stef> and some others are plain wrong.
Hm. I've never seen a problem but I only use Latin-1 and Japanese,
and the author is Japanese....
Can you say which ones are wrong, or better yet send me the file? You
say Gnome terminal does it correctly? So that gives me something to
check against.
Stef> also some fonts are missing so i assume it will not be able
Stef> to display it even with that.
Right.
Stef> is there a howto to get started or an faq entry ?
> No, there should be. Er ... FAQ is done. In CVS 21.5. What
> documentation there is is in the Mule-UCS Texinfo manual.
Stef> alright, i admit i only looked at the faq for 21.4 and the
Stef> C-h a ucs
I meant "I just took what I wrote to you and added it to the FAQ and
_now_ it's available in 21.5." Or soon, anyway. :-)
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
4:28 p.m.
New subject: utf-8 questions [was: Re: fontlock-speed]
On Thu, Feb 07, 2002 at 01:13:32AM +0900, Stephen J. Turnbull wrote:
Hm. I've never seen a problem but I only use Latin-1 and
Japanese,
and the author is Japanese....
Can you say which ones are wrong, or better yet send me the file? You
say Gnome terminal does it correctly? So that gives me something to
check against.
yup, attached is the file. gnome-terminal displays it correctly after
some magic: in the properties disable multibyte support, use a 20pt
iso10646 font like fixed (misc), in the terminal do 'echo -n %G' and
then you can either cat it or less -r it.
Stef> also some fonts are missing so i assume it will not be
able
Stef> to display it even with that.
Right.
yup, but even in the gnome-terminal with the correct fonts it could not
do it...
I meant "I just took what I wrote to you and added it to the FAQ
and
_now_ it's available in 21.5." Or soon, anyway. :-)
aha, well it's at least one thing less to get asked for ;)
--
# Stef Epardaud, # There is no limit to the power of computing ...
# Java Defeater # ... except mankind maybe ?
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
5:22 p.m.
New subject: utf-8 questions [was: Re: fontlock-speed]
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> yup, attached is the file.
Ah, Markus's torture test. I've got that around somewhere.
Stef> yup, but even in the gnome-terminal with the correct fonts
Stef> it could not do it...
Right. That's because there is no way to represent the missing
characters _in the buffer_ in default Mule code. It could be faked,
somebody who knows CCL could probably code it up in about 30 minutes.
But there are only three people who know CCL and I haven't time to
become #4 at the moment. :-þ
So when you read the Unicode into the buffer, all of the
unrepresentable characters turn into ? permanently as far as XEmacs is
concerned. I doubt you'd get a better result in GNU Emacs, they
operate under the same constraints with Mule that we do.
You could try UTF-2000 XEmacs, http://www.m17n.org/utf-2000/
(Japanese, but there should be links to CVS and FTP in plain text).
But it's a bloated pig (last I looked about 30MB binary) and not very
stable. And it's based on XEmacs 21.2.35. Old....
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
1:04 p.m.
On Wed, Feb 06, 2002 at 06:56:57PM +0900, Stephen J. Turnbull wrote:
Please do. I'd prefer you do it with 21.4.6 rather than the
21.5
branch. Ben is about to merge in about 70,000 lines of code to 21.5,
and there's just no way we'd find anything in that.
Please try profiling (1) the font-lock operation itself and (2) the
movement command that is causing problems. C-h f profile for more
information.
alright, here is me trying to indent the second argument declaration of
a member function of a class of 7000 lines of java:
(the last function)
public void bla(boolean first -->, pressed
(while profiling the , press)
Function Name Ticks %/Total Call Count
================== ===== ======= ==========
scan-lists 1183 91.351 998
scan-sexps 87 6.718 996
re-search-backward 23 1.776 3
byte-code 2 0.154 1019
---------------------------------------------------
Total 1295 100.000
One tick = 1 ms
for info, it takes about 2 seconds
public void bla(boolean first,
-->TAB pressed
(while profiling the TAB press)
Function Name Ticks %/Total Call Count
================== ===== ======= ==========
scan-lists 1174 95.138 500
scan-sexps 43 3.485 499
re-search-backward 16 1.297 2
byte-code 1 0.081 515
---------------------------------------------------
Total 1234 100.000
One tick = 1 ms
WARNING: Error checking is turned on in this XEmacs. This might make
the measurements very unreliable.
for info it takes about 25 seconds. (p3 1Ghz, 256MRAM, system inactive
otherwise)
i used profile-key-sequence in the two tests.
here is my Installation:
uname -a: Linux westmalle 2.4.9-ac10 #1 Fri Sep 21 14:40:17 CEST 2001
i686 unkno
wn
./configure '--with-dialogs=athena' '--with-widgets=athena'
'--with-mule' '--prefix=/opt/xemacs' '--with-xpm'
'--with-gif' '--with-jpeg' '--with-tiff' '--with-png'
'--with-x11'
XEmacs 21.5-b4 "bamboo" configured for `i686-pc-linux'.
Compilation / Installation:
Source code location: /opt/xemacs/src/xemacs-21.6
Installation prefix: /opt/xemacs
Operating system description file: `s/linux.h'
Machine description file: `m/intel386.h'
Compiler: gcc -g -O3 -Wall -Wno-switch -Winline -Wmis
sing-prototypes -Wsign-compare -Wshadow
Relocating allocator for buffers: no
GNU version of malloc: yes
- Using Doug Lea's new malloc from the GNU C Library.
Linking with `-z nocombreloc'.
- Consider configuring with --pdump.
Window System:
Compiling in support for the X window system:
- X Windows headers location: /usr/X11R6/include
- X Windows libraries location: /usr/X11R6/lib
- Handling WM_COMMAND properly.
Compiling in support for the Athena widget set:
- Athena headers location: X11/Xaw
- Athena library to link: Xaw
Using Lucid menubars.
Using Lucid scrollbars.
Using Athena dialog boxes.
Using Athena native widgets.
TTY:
Compiling in support for ncurses.
Images:
Compiling in support for GIF images (builtin).
Compiling in support for XPM images.
Compiling in support for PNG images.
Compiling in support for JPEG images.
Compiling in support for TIFF images.
Sound:
Compiling in support for sound (native).
Compiling in support for NAS (network audio system).
Compiling in support for ESD (Enlightened Sound Daemon).
Databases:
Internationalization:
Compiling in support for Mule (multi-lingual Emacs).
Compiling in support for XIM (X11R5+ I18N input method).
- Using raw Xlib to provide XIM support.
Mail:
Compiling in support for "dot-locking" mail spool file locking method.
Other Features:
Inhibiting IPv6 canonicalization at startup.
Compiling in support for dynamic shared object modules.
Compiling in support for extra debugging code.
WARNING: ---------------------------------------------------------
WARNING: Compiling in support for runtime error checking.
WARNING: XEmacs will run noticeably more slowly as a result.
WARNING: Error checking is on by default for XEmacs beta releases.
WARNING: ---------------------------------------------------------
--
# Stef Epardaud, # There is no limit to the power of computing ...
# Java Defeater # ... except mankind maybe ?
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
5:01 p.m.
At 10:05 AM 2/6/02 +0100, Stef Epardaud wrote:
I have to admit we're having the same problem here in the office,
I used
to maintain a list of latest 21.1/2/4/5 installed, and we all noticed
some very annoying speed decrease at least in java (and jde when we
tried it) mode with font-lock for files over 5000 lines.
We have some files with around 10000 lines of java, and pressing tab to
get to the next line can take up to 5 seconds, and never less than 2.
So we all switched back to 21.1 as I did not have time to look into
that. I can try to build the latest of the latest (althought I'm not
sure anymore if it's 21.4 or .5 i need these days) and try it and
send the Installation file here if you want.
Some datapoints would be great. There are a number of possible culprits:
1 font-lock itself
2 regexp or syntax table issues
3 the progress bar
4 sematic
I would like to at least determine what is causing the slowdown. I see the
tab and interactive slowdown for large java files which indicates that its
not (3) I always had put this down to JDE's use of semantic but if its
faster in 21.1 I would like to get to the bottom of it.
andy
5:07 p.m.
On Wed, Feb 06, 2002 at 09:01:02AM -0800, Andy Piper wrote:
Some datapoints would be great. There are a number of possible
culprits:
ask me anything but the source file itself ;) although i'm sure i can
pick one of the xerces files and get the same result.
1 font-lock itself
2 regexp or syntax table issues
3 the progress bar
4 sematic
I would like to at least determine what is causing the slowdown. I see
the tab and interactive slowdown for large java files which indicates
that its not (3) I always had put this down to JDE's use of semantic
but if its faster in 21.1 I would like to get to the bottom of it.
it's not JDE since i'm using plain stupid java mode. also note that yes,
loading that file takes quite some time also, but it's really editing
that bothers me.
--
# Stef Epardaud, # There is no limit to the power of computing ...
# Java Defeater # ... except mankind maybe ?
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
5:41 p.m.
>>>> "Andy" == Andy Piper <andyp(a)bea.com>
writes:
Andy> Some datapoints would be great. There are a number of
Andy> possible culprits:
He sent me some profiles, which point straight at at scan-lists. CC'd
xemacs-beta.
I took a look at syntax.c (scan_lists). It looks like it's related to
Matt's syntax stuff after all. What's happening is that the
UPDATE_SYNTAX_CACHE macros are getting called at least once per
movement of point. These in turn call get-text-properties when the
cache gets invalidated, which is every time point crosses an extent
boundary. get-text-properties is a pretty heavy function to be
calling in a buffer scan routine, I think, since it does a
map_extents_bytind.
I suspect this could be tested by defining all those .*SYNTAX_CACHE.*
macros in syntax.h to the empty string.
Still, one scan shouldn't be _that_ bad. The next question is why
does one invocation of `c-electric-semi&comma' call `scan-lists' 998
times?
I think there's a pretty good chance there's something broken in
cc-mode or font-lock here.
Time for bed; may I expect a solution in the morning? :) :)
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
8:39 p.m.
-- Andy Piper <andyp(a)bea.com> spake thusly:
At 02:41 AM 2/7/02 +0900, Stephen J. Turnbull wrote:
> I think there's a pretty good chance there's something broken in
> cc-mode or font-lock here.
Unlikely since the behavior doesn't show up in 21.1 with the same
versions of cc-mode.
Not necessarily. If scan-lists got slower due to the use of
get-text-properties, it's possible that it's now exposing a bug in
cc-mode or font-lock. I don't know that that's the case, since I
haven't really looked at what's going on in-depth.
If it's not the case, however, a reasonable fix might be a buffer-local
'use-syntax-table-properties' variable which defaults to nil, thus only
enabling the extents lookups for modes which use them.
Or perhaps an even better fix would be to improve the performance of
get-text-properties.
3:09 p.m.
Matt Tucker <tuck(a)whistlingfish.net> writes:
If it's not the case, however, a reasonable fix might be a
buffer-local 'use-syntax-table-properties' variable which defaults
to nil, thus only enabling the extents lookups for modes which use
them.
Yes, but what about the modes that *do* or will use the local syntax
tables? Not to mention that some uses of local syntax tables could
span several modes; for example, someone might want to write a
multi-mode utility that used them.
Unacceptable performance of local syntax tables would remove much of
their utility.
Or perhaps an even better fix would be to improve the performance of
get-text-properties.
That would be a neat thing, but it might not be easy. I remember that
Ben had some hints how to do that. He might be able to help.
Failing that, could the syntax code be modified to better interact
with how XEmacs extents work? Like, is scan-lists really calling
get-text-properties that often now? Could it be modified to call
lower-level functions, and/or to cache the results of those calls?
The redisplay code already contains similar optimizations, which
include the concept of "extent runs" and "stacks of extents". The
needs of the syntax table code are probably much simpler than those of
the redisplay, but perhaps the same kind of optimizing logic is
applicable.
3:44 p.m.
>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
Hrvoje> Failing that, could the syntax code be modified to better
Hrvoje> interact with how XEmacs extents work? Like, is
Hrvoje> scan-lists really calling get-text-properties that often
Hrvoje> now? Could it be modified to call lower-level functions,
I think g-t-p already basically does that. If you need to actually
get a property that might be on any of the extents, g-t-p probably
can't be much improved on.
Hrvoje> and/or to cache the results of those calls?
It already does that, however the cache is getting invalidated a lot
more often than it needs to. (It gets invalidated on every extent
endpoint, whether that extent has a syntax table or not.) In
particular, it should be possible to keep a buffer local variable
which does not _default_ whether local syntax tables are used or not,
but rather is updated when you set a local syntax table property on an
extent in the buffer. Since only cperl mode uses them now, that would
be enough for 21.4.
Then, rather than just a flag, you generalize to keeping a list of the
extents that have syntax table properties on them, and consult that
rather than doing g-t-p.
Hrvoje> The redisplay code already contains similar optimizations,
Hrvoje> which include the concept of "extent runs" and "stacks of
Hrvoje> extents".
This is another good idea, since font-locking closely resembles
redisplay in a lot of ways.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
8:12 a.m.
stephen is correct about what needs to be done to optimize this. other than
caching of the sort he mentions, there isn't much that can obviously be done to
speed up get-text-properties. if you look at the code you'll see it basically
boils down to a map-extents call -- and map-extents is already optimized to hell
and back. it already uses a stack of extents to optimize sequential accesses,
and extents themselves are already optimized to favor lookup over change.
"Stephen J. Turnbull" wrote:
>>>>> "Hrvoje" == Hrvoje Niksic <hniksic(a)arsdigita.com>
writes:
Hrvoje> Failing that, could the syntax code be modified to better
Hrvoje> interact with how XEmacs extents work? Like, is
Hrvoje> scan-lists really calling get-text-properties that often
Hrvoje> now? Could it be modified to call lower-level functions,
I think g-t-p already basically does that. If you need to actually
get a property that might be on any of the extents, g-t-p probably
can't be much improved on.
Hrvoje> and/or to cache the results of those calls?
It already does that, however the cache is getting invalidated a lot
more often than it needs to. (It gets invalidated on every extent
endpoint, whether that extent has a syntax table or not.) In
particular, it should be possible to keep a buffer local variable
which does not _default_ whether local syntax tables are used or not,
but rather is updated when you set a local syntax table property on an
extent in the buffer. Since only cperl mode uses them now, that would
be enough for 21.4.
Then, rather than just a flag, you generalize to keeping a list of the
extents that have syntax table properties on them, and consult that
rather than doing g-t-p.
Hrvoje> The redisplay code already contains similar optimizations,
Hrvoje> which include the concept of "extent runs" and "stacks of
Hrvoje> extents".
This is another good idea, since font-locking closely resembles
redisplay in a lot of ways.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
10:35 a.m.
Ben Wing <ben(a)666.com> writes:
stephen is correct about what needs to be done to optimize this.
other than caching of the sort he mentions, there isn't much that
can obviously be done to speed up get-text-properties. if you look
at the code you'll see it basically boils down to a map-extents call
Are you sure about this? get-text-property ends up calling extent_at
in a loop, and each invocation of extent_at calls map_extents_bytind.
I might be misreading the code, but to me this indicates a possibility
that get-text-property requires time proportional to n^2, where n is
the number of extents in the buffer.
Please correct me if I'm wrong.
4:53 a.m.
>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
Hrvoje> I might be misreading the code, but to me this indicates a
Hrvoje> possibility that get-text-property requires time
Hrvoje> proportional to n^2, where n is the number of extents in
Hrvoje> the buffer.
I haven't done a careful analysis of extent_at_byteind, but note the
"prior" argument in the call to extent_at_byteind:
while (1)
{
extent = extent_at_bytind (position, object, Qtext_prop, prior,
fl, 0);
if (NILP (extent))
return Qnil;
if (EQ (prop, Fextent_property (extent, Qtext_prop, Qnil)))
break;
prior = XEXTENT (extent);
}
Ie, each extent gets visited once.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
5:34 a.m.
"Stephen J. Turnbull" wrote:
>>>>> "Hrvoje" == Hrvoje Niksic <hniksic(a)arsdigita.com>
writes:
Hrvoje> I might be misreading the code, but to me this indicates a
Hrvoje> possibility that get-text-property requires time
Hrvoje> proportional to n^2, where n is the number of extents in
Hrvoje> the buffer.
I haven't done a careful analysis of extent_at_byteind, but note the
"prior" argument in the call to extent_at_byteind:
while (1)
{
extent = extent_at_bytind (position, object, Qtext_prop, prior,
fl, 0);
if (NILP (extent))
return Qnil;
if (EQ (prop, Fextent_property (extent, Qtext_prop, Qnil)))
break;
prior = XEXTENT (extent);
}
Ie, each extent gets visited once.
and we don't need to search from the beginning of the extent list each time to
find the next one, because the extent list is ordered. g-t-p is at worst
log(N)*# of extents overlapping a position.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
9:22 a.m.
"Stephen J. Turnbull" <stephen(a)xemacs.org> writes:
>>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
Hrvoje> I might be misreading the code, but to me this indicates a
Hrvoje> possibility that get-text-property requires time
Hrvoje> proportional to n^2, where n is the number of extents in
Hrvoje> the buffer.
I haven't done a careful analysis of extent_at_byteind, but note the
"prior" argument in the call to extent_at_byteind:
I'm aware of the PRIOR argument; I just wasn't sure that it was
helping performance. extent_at_byteind calls the mapper function
every time, and then sorts the results, etc. Or is that not the case?
For example, wouldn't get-text-property be faster if it were
implemented with one call to map_extents_bytind which did all the
necessary magic? Am I wrong in assuming that map_extents_bytind
traverses the extents in display order?
5:03 p.m.
>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
Hrvoje> For example, wouldn't get-text-property be faster if it
Hrvoje> were implemented with one call to map_extents_bytind which
Hrvoje> did all the necessary magic?
Up to this point I've been assuming that Ben (or whoever) knew what he
was doing in map_extents_bytind; it is now time for me to admit that,
and bow out of the thread, as I do not understand that function.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
1:09 a.m.
Hrvoje Niksic wrote:
"Stephen J. Turnbull" <stephen(a)xemacs.org> writes:
>>>>>> "Hrvoje" == Hrvoje Niksic
<hniksic(a)arsdigita.com> writes:
>
> Hrvoje> I might be misreading the code, but to me this indicates a
> Hrvoje> possibility that get-text-property requires time
> Hrvoje> proportional to n^2, where n is the number of extents in
> Hrvoje> the buffer.
>
> I haven't done a careful analysis of extent_at_byteind, but note the
> "prior" argument in the call to extent_at_byteind:
I'm aware of the PRIOR argument; I just wasn't sure that it was
helping performance. extent_at_byteind calls the mapper function
every time, and then sorts the results, etc. Or is that not the case?
For example, wouldn't get-text-property be faster if it were
implemented with one call to map_extents_bytind which did all the
necessary magic? Am I wrong in assuming that map_extents_bytind
traverses the extents in display order?
hmm ... i see. yes, it would be faster to do only one map-extents call, but it
may not be significant. the logic of the current text-properties code is
largely left over from the implementation that jamie did before i rewrote the
extents code. basically, the current approach is (time to move SOE) + O(M^2)
where M == number of extents overlapping the position in question. your
approach would be (time to move SOE) + O(M). usually M is very small. i.e.
we're talking about differences in constant factors, not anything related to the
number of extents in the buffer. you still aren't changing the fact that you're
calling g-t-p at each position in the buffer, so the biggest speedup would come
from avoiding this by clever caching. but if the call to g-t-p is dominating
the font-lock time, you could get it noticeably faster -- maybe 1.5 to 2 times
if you're lucky.
9:37 a.m.
Ben Wing <ben(a)666.com> writes:
basically, the current approach is (time to move SOE) + O(M^2) where
M == number of extents overlapping the position in question. your
approach would be (time to move SOE) + O(M). usually M is very
small.
I think you're right. Rewriting g-t-p would speed things up, but not
noticably in the common case.
i.e. we're talking about differences in constant factors, not
anything related to the number of extents in the buffer. you still
aren't changing the fact that you're calling g-t-p at each position
in the buffer, so the biggest speedup would come from avoiding this
by clever caching. but if the call to g-t-p is dominating the
font-lock time, you could get it noticeably faster -- maybe 1.5 to 2
times if you're lucky.
Again, you're right. It would probably make sense to do both: speed
up g-t-p by calling map_extents directly, and not call it on every
character.
I'll try to tackle some of this when I have time.
10:09 a.m.
On Wed, Feb 06, 2002 at 09:55:40AM -0800, Andy Piper wrote:
At 02:41 AM 2/7/02 +0900, Stephen J. Turnbull wrote:
>I think there's a pretty good chance there's something broken in
>cc-mode or font-lock here.
Unlikely since the behavior doesn't show up in 21.1 with the same versions
of cc-mode.
alright, you made me doubt, so i checked out the latest version of 21.1
with cvs, compiled it and ran it and got a crash before showing the
frame.
don't know exactly if it's my fault or not since i compiled it on
another machine and installed it in a second one, but all the librairies
are there and yesterday it worked for 21.6.
here is Installation:
uname -a: Linux westmalle 2.4.9-ac10 #1 Fri Sep 21 14:40:17 CEST 2001
i686 unknown
./configure '--with-dialogs=athena' '--with-mule'
'--prefix=/opt/xemacs' '--with-xpm' '--with-gif'
'--with-jpeg' '--with-tiff' '--with-png' '--with-x11'
XEmacs 21.1.14 "Cuyahoga Valley" configured for `i686-pc-linux'.
Where should the build process find the source code?
/opt/xemacs/src/xemacs-21.1
What installation prefix should install use? /opt/xemacs
What operating system and machine description files should XEmacs use?
`s/linux.h' and `m/intel386.h'
What compiler should XEmacs be built with? gcc -g -O3 -Wall -Wno-switch
Should XEmacs use the GNU version of malloc? yes
(Using Doug Lea's new malloc from the GNU C Library.)
Should XEmacs use the relocating allocator for buffers? yes
What window system should XEmacs use? x11
Where do we find X Windows header files? /usr/X11R6/include
Where do we find X Windows libraries? /usr/X11R6/lib
Compiling in support for XAUTH.
Compiling in support for XPM images.
Compiling in support for PNG image handling.
Compiling in support for (builtin) GIF image handling.
Compiling in support for JPEG image handling.
Compiling in support for TIFF image handling.
Compiling in native sound support.
Compiling in support for ncurses.
Compiling in Mule (multi-lingual) support.
Compiling in XIM (X11R5+ I18N input method) support.
Using raw Xlib to provide XIM support.
Compiling in support for proper session-management.
Using Lucid menubars.
Using Lucid scrollbars.
Using Athena dialog boxes.
Compiling in DLL support.
movemail will use "dot-locking" for locking mail spool files.
Using Lisp_Objects with minimal tagbits.
here is the message when it brweaks:
Lisp backtrace follows:
# bind (frame-being-created)
make-frame(nil #<x-device on ":0.0" 0x221e>)
frame-initialize()
# bind (debugger debug-on-error command-line-args-left)
command-line()
# (unwind-protect ...)
normal-top-level()
# (condition-case ... . error)
# (catch top-level ...)
Segmentation fault (core dumped)
and here is the gdb session:
GNU gdb 5.1
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "i386-linux"...
Core was generated by `./xemacs-21.1.14'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libgdkxft.so...done.
Loaded symbols for /usr/lib/libgdkxft.so
Reading symbols from /usr/X11R6/lib/libXaw.so.7...done.
Loaded symbols for /usr/X11R6/lib/libXaw.so.7
Reading symbols from /usr/lib/libtiff.so.3...done.
Loaded symbols for /usr/lib/libtiff.so.3
Reading symbols from /usr/lib/libpng.so.3...done.
Loaded symbols for /usr/lib/libpng.so.3
Reading symbols from /usr/lib/libjpeg.so.62...done.
Loaded symbols for /usr/lib/libjpeg.so.62
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /usr/X11R6/lib/libXpm.so.4...done.
Loaded symbols for /usr/X11R6/lib/libXpm.so.4
Reading symbols from /usr/X11R6/lib/libXmu.so.6...done.
Loaded symbols for /usr/X11R6/lib/libXmu.so.6
Reading symbols from /usr/X11R6/lib/libXt.so.6...done.
Loaded symbols for /usr/X11R6/lib/libXt.so.6
Reading symbols from /usr/X11R6/lib/libXext.so.6...done.
Loaded symbols for /usr/X11R6/lib/libXext.so.6
Reading symbols from /usr/X11R6/lib/libX11.so.6...done.
Loaded symbols for /usr/X11R6/lib/libX11.so.6
Reading symbols from /usr/X11R6/lib/libSM.so.6...done.
Loaded symbols for /usr/X11R6/lib/libSM.so.6
Reading symbols from /usr/X11R6/lib/libICE.so.6...done.
Loaded symbols for /usr/X11R6/lib/libICE.so.6
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libncurses.so.5...done.
Loaded symbols for /lib/libncurses.so.5
Reading symbols from /lib/libm.so.6...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /usr/lib/libgtk-1.2.so.0...done.
Loaded symbols for /usr/lib/libgtk-1.2.so.0
Reading symbols from /usr/lib/libgdk-1.2.so.0...done.
Loaded symbols for /usr/lib/libgdk-1.2.so.0
Reading symbols from /usr/lib/libgmodule-1.2.so.0...done.
Loaded symbols for /usr/lib/libgmodule-1.2.so.0
Reading symbols from /usr/lib/libglib-1.2.so.0...done.
Loaded symbols for /usr/lib/libglib-1.2.so.0
Reading symbols from /usr/X11R6/lib/libXi.so.6...done.
Loaded symbols for /usr/X11R6/lib/libXi.so.6
Reading symbols from /usr/X11R6/lib/libXft.so.1...done.
Loaded symbols for /usr/X11R6/lib/libXft.so.1
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/X11R6/lib/libXrender.so.1...done.
Loaded symbols for /usr/X11R6/lib/libXrender.so.1
Reading symbols from /usr/lib/libfreetype.so.6...done.
Loaded symbols for /usr/lib/libfreetype.so.6
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_compat.so.2...done.
Loaded symbols for /lib/libnss_compat.so.2
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/gconv/ISO8859-1.so...done.
Loaded symbols for /usr/lib/gconv/ISO8859-1.so
#0 0x4031b911 in kill () from /lib/libc.so.6
(gdb) where
#0 0x4031b911 in kill () from /lib/libc.so.6
#1 0x080a6b2b in fatal_error_signal (sig=11) at emacs.c:264
#2 0x4031b848 in sigaction () from /lib/libc.so.6
#3 0x4014a58b in XtInitializeWidgetClass () from /usr/X11R6/lib/libXt.so.6
#4 0x4014a746 in XtInitializeWidgetClass () from /usr/X11R6/lib/libXt.so.6
#5 0x4014b024 in _XtCreatePopupShell () from /usr/X11R6/lib/libXt.so.6
#6 0x4014b0b5 in XtCreatePopupShell () from /usr/X11R6/lib/libXt.so.6
#7 0x08167120 in x_create_widgets (f=0x8446908, lisp_window_id=137130708,
parent=137130708) at frame-x.c:1952
#8 0x081676a7 in x_init_frame_1 (f=0x8446908, props=137130708) at frame-x.c:2183
#9 0x080fdcc7 in Fmake_frame (props=137130708, device=138674456) at frame.c:383
#10 0x080aa0cb in funcall_recording_as (recorded_as=137227276, nargs=2, args=0xbfffeb44)
at eval.c:3197
#11 0x080ae373 in Ffuncall (nargs=3, args=0xbfffeb44) at eval.c:3238
#12 0x08088489 in Fbyte_code (bytestr=136307416, vector=136307508, maxdepth=11) at
bytecode.c:416
#13 0x080aad8b in funcall_lambda (fun=136307596, nargs=0, arg_vector=0xbfffee48) at
eval.c:3607
#14 0x080aa1c1 in funcall_recording_as (recorded_as=137851476, nargs=0, args=0xbfffee44)
at eval.c:3214
#15 0x080ae373 in Ffuncall (nargs=1, args=0xbfffee44) at eval.c:3238
#16 0x08088489 in Fbyte_code (bytestr=136452716, vector=136452884, maxdepth=9) at
bytecode.c:416
#17 0x080aad8b in funcall_lambda (fun=136453104, nargs=0, arg_vector=0xbffff248) at
eval.c:3607
#18 0x080aa1c1 in funcall_recording_as (recorded_as=137966860, nargs=0, args=0xbffff244)
at eval.c:3214
#19 0x080ae373 in Ffuncall (nargs=1, args=0xbffff244) at eval.c:3238
#20 0x08088489 in Fbyte_code (bytestr=136450340, vector=136450600, maxdepth=13) at
bytecode.c:416
#21 0x080aad8b in funcall_lambda (fun=136451148, nargs=0, arg_vector=0xbffff324) at
eval.c:3607
#22 0x080aa93d in apply_lambda (fun=136451148, numargs=0, unevalled_args=137130708) at
eval.c:3511
#23 0x080a9bbe in Feval (form=137826536) at eval.c:3085
#24 0x080ae022 in condition_case_1 (handlers=137130820, bfun=0x80a929c <Feval>,
barg=137826536, hfun=0x8090f84 <cmd_error>, harg=137130708) at eval.c:1653
#25 0x0809100f in top_level_1 (dummy=137130708) at cmdloop.c:206
#26 0x080adef4 in internal_catch (tag=137205388, func=0x8090fe0<top_level_1>,
arg=137130708, threw=0x0) at eval.c:1328
#27 0x0809091a in initial_command_loop (load_me=137130708) at cmdloop.c:285
#28 0x080a54ae in xemacs_21_1_14_i686_pc_linux () at emacs.c:1725
#29 0x080a6d50 in main (argc=1, argv=0xbffff7c4, envp=0xbffff7cc) at emacs.c:2142
#30 0x4030b65f in __libc_start_main () from /lib/libc.so.6
hope it helps. once again, it might be related to compiling on a
different machine.
--
# Stef Epardaud, # Teachers have potentially more power than military,
# Java Defeater # the former can teach us how to not need the latter.
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
11:18 a.m.
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> don't know exactly if it's my fault or not since i compiled
Stef> it on another machine and installed it in a second one, but
Stef> all the librairies are there and yesterday it worked for
Stef> 21.6.
No, it's the GNU binutils breakage.
Use "LDFLAGS='-z nocombreloc' ./configure ..."
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
12:25 p.m.
On Thu, Feb 07, 2002 at 08:18:47PM +0900, Stephen J. Turnbull wrote:
No, it's the GNU binutils breakage.
Use "LDFLAGS='-z nocombreloc' ./configure ..."
alright, that did it, thanx again.
so, with 21.1, the ',' insertion is instantaneous, and the indenting of
the second param decl takes exactly 3 seconds. indenting lines in the
function itself takes about half a second, which is also better than
21.6
so there really is a difference. want me to profile that one too ?
do you want me to try with another long java file that i can give you?
--
# Stef Epardaud, # Teachers have potentially more power than military,
# Java Defeater # the former can teach us how to not need the latter.
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
1:48 p.m.
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> so there really is a difference. want me to profile that one
Stef> too ?
Sure, if you've got the minute or so to do it. :-)
Stef> do you want me to try with another long java file that i can
Stef> give you?
Yes. Andy probably has test cases of his own, but the biggest files I
have to test are like src/syntax.c and the delay is not noticable on
my machines.
just send it to me personally, I don't think you need to post a 7000
line file on xemacs-beta ...
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
1:34 a.m.
Stef Epardaud <stef(a)lunatech.com> writes:
On Thu, Feb 07, 2002 at 08:18:47PM +0900, Stephen J. Turnbull wrote:
> No, it's the GNU binutils breakage.
>
> Use "LDFLAGS='-z nocombreloc' ./configure ..."
alright, that did it, thanx again.
so, with 21.1, the ',' insertion is instantaneous, and the indenting of
the second param decl takes exactly 3 seconds. indenting lines in the
function itself takes about half a second, which is also better than
21.6
so there really is a difference. want me to profile that one too ?
do you want me to try with another long java file that i can give
you?
It's very important that you provide the file, preferrably along with
a "how-to-repeat" procedure that begins with `xemacs -vanilla'.
The reason for this is that I've encountered the font-lock/indentation
slowness myself, and complained about it. The maintainer of the
syntax code asked me to provide an example, which is when I found that
I couldn't repeat the extreme slowness using `xemacs -vanilla', or
after firing up any new XEmacs. If you have a repeatable case for
slowness, please provide it.
7:38 a.m.
On Fri, 08 Feb 2002, Hrvoje Niksic stipulated:
The reason for this is that I've encountered the
font-lock/indentation
slowness myself, and complained about it. The maintainer of the
syntax code asked me to provide an example, which is when I found that
I couldn't repeat the extreme slowness using `xemacs -vanilla', or
after firing up any new XEmacs. If you have a repeatable case for
slowness, please provide it.
I ran into a case of gutter-related hyperslowness a while back, the
problem being that *everything* triggered a gutter redraw, and that
window_changed was being set by things like every single mouse movement.
It went away when I stopped being stupid and activating lazy-lock and
lazy-shot at the same time (thick typo, I was trying to use lazy-shot
alone). Even so, on the off-chance that this might be useful, here are
some half-digested notes on the symptoms (and other random things I
noticed at the same time):
lazy-lock ocnditionalizes on `frame-modified-tick' --- but repainting
the gutters increments this variable; so whenever the idle-hook fires,
lazy-lock kicks up, changes the window (even if it's done nothing
visible, especially in minibuffers), and the gutters promptly get
repainted --- which increments frame-modified-hook again, so the next
time it kicks up it'll hit that again! (Why does a window change trigger
a gutter repaint anyway? windows != gutters...)
Ancillary wonders: pgup triggers a gutter redisplay, but scrolling up
doesn't, even if scroll-in-place is not in use.
Popping up the minibuffer triggers a gutter_changed (by what, pray?
I have no toolbars active or anything.)
Disclaimer: none of this may apply with a sane configuration. This
hasn't been checked much. This may make no sense. This is half-digested
notes to myself. :)
--
`It is to be proven that Linux Kernel is the most stable than MS Windows
that it uses less stable.' --- Bryan Parkoff being very comprehensible
8:19 a.m.
Nix <nix(a)esperi.demon.co.uk> writes:
I ran into a case of gutter-related hyperslowness a while back, the
problem being that *everything* triggered a gutter redraw
I've seen that too, but it's a different issue because I'm not even
*using* gutters. (I have both fancy progress meter and buffer tabs
turned off.)
For me a good proof of the gutter slowness remains the unusable
Edit-Find dialog box.
On another note, I'm not using any of the lazy-* things because I
never needed them. font-lock has always been fast enough for me, even
on machines much slower than the one I'm using now. We should be
careful not to ruin it -- all the lazy-* things are optimizations that
come with a price.
5:28 p.m.
Yeah, there was a long discussion on this a while back. I concluded that
lazy-lock was brain-dead and advised people not to use it :) Of course
there may be some dissention about this observation....
The window change triggers a gutter repaint because gutters are displayed
using the same (or similar code). I you don't do this you get into some
weird non-redisplay problems when window proprties and things change.
andy
At 07:38 AM 2/8/2002 +0000, Nix wrote:
On Fri, 08 Feb 2002, Hrvoje Niksic stipulated:
> The reason for this is that I've encountered the font-lock/indentation
> slowness myself, and complained about it. The maintainer of the
> syntax code asked me to provide an example, which is when I found that
> I couldn't repeat the extreme slowness using `xemacs -vanilla', or
> after firing up any new XEmacs. If you have a repeatable case for
> slowness, please provide it.
I ran into a case of gutter-related hyperslowness a while back, the
problem being that *everything* triggered a gutter redraw, and that
window_changed was being set by things like every single mouse movement.
It went away when I stopped being stupid and activating lazy-lock and
lazy-shot at the same time (thick typo, I was trying to use lazy-shot
alone). Even so, on the off-chance that this might be useful, here are
some half-digested notes on the symptoms (and other random things I
noticed at the same time):
lazy-lock ocnditionalizes on `frame-modified-tick' --- but repainting
the gutters increments this variable; so whenever the idle-hook fires,
lazy-lock kicks up, changes the window (even if it's done nothing
visible, especially in minibuffers), and the gutters promptly get
repainted --- which increments frame-modified-hook again, so the next
time it kicks up it'll hit that again! (Why does a window change trigger
a gutter repaint anyway? windows != gutters...)
Ancillary wonders: pgup triggers a gutter redisplay, but scrolling up
doesn't, even if scroll-in-place is not in use.
Popping up the minibuffer triggers a gutter_changed (by what, pray?
I have no toolbars active or anything.)
Disclaimer: none of this may apply with a sane configuration. This
hasn't been checked much. This may make no sense. This is half-digested
notes to myself. :)
--
`It is to be proven that Linux Kernel is the most stable than MS Windows
that it uses less stable.' --- Bryan Parkoff being very comprehensible
8:23 a.m.
-- Hrvoje Niksic <hniksic(a)arsdigita.com> spake thusly:
It's very important that you provide the file, preferrably along
with
a "how-to-repeat" procedure that begins with `xemacs -vanilla'.
The reason for this is that I've encountered the font-lock/indentation
slowness myself, and complained about it. The maintainer of the
syntax code asked me to provide an example, which is when I found that
I couldn't repeat the extreme slowness using `xemacs -vanilla', or
after firing up any new XEmacs. If you have a repeatable case for
slowness, please provide it.
I (being the maintainer in question) will heartily second that. I'm
happy to fix the problem, but it would help me a great deal if I had
some definitely reproducible case to work with.
9:41 a.m.
On Fri, Feb 08, 2002 at 02:34:11AM +0100, Hrvoje Niksic wrote:
It's very important that you provide the file, preferrably along
with
a "how-to-repeat" procedure that begins with `xemacs -vanilla'.
The reason for this is that I've encountered the font-lock/indentation
slowness myself, and complained about it. The maintainer of the
syntax code asked me to provide an example, which is when I found that
I couldn't repeat the extreme slowness using `xemacs -vanilla', or
after firing up any new XEmacs. If you have a repeatable case for
slowness, please provide it.
right, after trying to send the attachment to Stephen for a day while my
mails keep bouncing for stephane(a)xemacs.org with a connection refused,
here is the file:
http://www1.lunatech.com/~stephane/TraverseSchema.java
try to
indent the second argument declaration by putting it on a separate line
to the method defined at line 8939, reportSchemaError()
indenting that line with TAB takes me over 30secs on a GHz machine...
have fun! ;) (if you need anything more i'll be pleased to help)
--
# Stef Epardaud, # Teachers have potentially more power than military,
# Java Defeater # the former can teach us how to not need the latter.
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
11:30 a.m.
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> right, after trying to send the attachment to Stephen for a
Stef> day while my mails keep bouncing for stephane(a)xemacs.org
Use stephen(a)xemacs.org if you haven't done so already.
Stef> with a connection refused, here is the file:
Stef> http://www1.lunatech.com/~stephane/TraverseSchema.java
--
Adrian Aichner
mailto:adrianï¼ xemacs.org
http://www.xemacs.org/
11:34 a.m.
On Fri, Feb 08, 2002 at 12:30:49PM +0100, Adrian Aichner wrote:
>>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> right, after trying to send the attachment to Stephen for a
Stef> day while my mails keep bouncing for stephane(a)xemacs.org
Use stephen(a)xemacs.org if you haven't done so already.
haha, sorry, i meant stephen, it's stephen that bounced ;)
years of typing my own name with this spelling, typing it with a
different spelling is tricky :)
--
# Stef Epardaud, # Teachers have potentially more power than military,
# Java Defeater # the former can teach us how to not need the latter.
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
12:35 p.m.
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> On Fri, Feb 08, 2002 at 02:34:11AM +0100, Hrvoje Niksic
Stef> wrote:
> It's very important that you provide the file, preferrably
> along with a "how-to-repeat" procedure that begins with `xemacs
> -vanilla'.
Yep, I get problems indenting, and -vanilla doesn't make things any
faster. Turning off font-lock speeds things up dramatically (about
70%), but it still takes a lo-o-ong time.
Looks like we have something to work with:
xemacs -vanilla
M-: (setq auto-mode-alist nil) RET
C-x C-f TraverseSchema.java
M-x java-mode RET
M-g 8941 RET
C-s m a j o r , RET
C-o C-f
and profiling the TAB keystroke gives:
Function Name Ticks %/Total Call Count
==================== ===== ======= ==========
scan-lists 1589 94.247 307
scan-sexps 49 2.906 305
re-search-backward 45 2.669 2
c-indent-command 1 0.059 1
c-parse-state 1 0.059 1
c-guess-basic-syntax 1 0.059 1
-----------------------------------------------------
Total 1686 100.000
about 7 secs subjective time. scan-lists and scan-sexps are both
implemented by the same C function, scan_lists. I wonder why the big
difference in ticks?
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
4:44 p.m.
>>>> "Andy" == Andy Piper <andyp(a)bea.com>
writes:
Andy> At 02:41 AM 2/7/02 +0900, Stephen J. Turnbull wrote:
> I think there's a pretty good chance there's something
broken
> in cc-mode or font-lock here.
Andy> Unlikely since the behavior doesn't show up in 21.1 with the
Andy> same versions of cc-mode.
It's quite possible that without Matt's code you simply don't notice
that 997 of those 998 calls to scan_list are redundant, in any Emacs.
Remember all we're doing here is to _indent one line_. In the case of
the TAB (which causes twice as many scans!!), font-lock should not
change anything---if it does there's a bug! Arguably font-lock (less
likely, cc-mode) is broken. In languages where indentation is not
syntactically significant, there is no good reason for changes in
indentation to cause XEmacs to go charging up and down the buffer
scanning sexps.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
9:20 p.m.
On 08 Feb 2002 at 01:44:37 +0900, "Stephen J. Turnbull"
<stephen(a)xemacs.org> wrote:
Remember all we're doing here is to _indent one line_. In the
case of
the TAB (which causes twice as many scans!!), font-lock should not
change anything---if it does there's a bug! Arguably font-lock (less
likely, cc-mode) is broken. In languages where indentation is not
syntactically significant, there is no good reason for changes in
indentation to cause XEmacs to go charging up and down the buffer
scanning sexps.
Here's one more datapoint for you. I'm using PVS
(<URL:http://pvs.csl.sri.com/>). Loading its prelude.pvs file, font
locked, used to be nearly instantaneous under XEmacs 21.1. With 21.4,
it takes about 8 seconds on my dual 500 MHz Pentium III machine. The
file is 3721 lines, 105,407 characters. This is just loading the file;
I'm not trying to edit it. I haven't yet been bugged enough to try
tracking down the problem. If there is anything you want me to try, let
me know.
Incidentally, this is the value of font-lock-defaults under PVS:
(pvs-font-lock-keywords nil t)
So the case-fold argument is t. This is the value of pvs-font-lock-keywords:
("\\b[aA][nN][dD]\\b\\|\\b[aA][nN][dD][tT][hH][eE][nN]\\b\\|\\b[aA][rR][rR][aA][yY]\\b\\|\\b[aA][sS][sS][uU][mM][iI][nN][gG]\\b\\|\\b[aA][sS][sS][uU][mM][pP][tT][iI][oO][nN]\\b\\|\\b[aA][uU][tT][oO][__][rR][eE][wW][rR][iI][tT][eE]\\b\\|\\b[aA][uU][tT][oO][__][rR][eE][wW][rR][iI][tT][eE][++]\\b\\|\\b[aA][uU][tT][oO][__][rR][eE][wW][rR][iI][tT][eE][--]\\b\\|\\b[aA][xX][iI][oO][mM]\\b\\|\\b[bB][eE][gG][iI][nN]\\b\\|\\b[bB][uU][tT]\\b\\|\\b[bB][yY]\\b\\|\\b[cC][aA][sS][eE][sS]\\b\\|\\b[cC][hH][aA][lL][lL][eE][nN][gG][eE]\\b\\|\\b[cC][lL][aA][iI][mM]\\b\\|\\b[cC][lL][oO][sS][uU][rR][eE]\\b\\|\\b[cC][oO][nN][dD]\\b\\|\\b[cC][oO][nN][jJ][eE][cC][tT][uU][rR][eE]\\b\\|\\b[cC][oO][nN][tT][aA][iI][nN][iI][nN][gG]\\b\\|\\b[cC][oO][nN][vV][eE][rR][sS][iI][oO][nN]\\b\\|\\b[cC][oO][nN][vV][eE][rR][sS][iI][oO][nN][++]\\b\\|\\b[cC][oO][nN][vV][eE][rR][sS][iI][oO][nN][--]\\b\\|\\b[cC][oO][rR][oO][lL][lL][aA][rR][yY]\\b\\|\\b[dD][aA][tT][aA][tT][yY][pP][eE]\\b\\|\\b[eE][lL][sS][eE]\\b\\|\\b[eE!
][lL][sS][iI][fF]\\b\\|\\b[eE][nN][dD]\\b\\|\\b[eE][nN][dD][aA][sS][sS][uU][mM][iI][nN][gG]\\b\\|\\b[eE][nN][dD][cC][aA][sS][eE][sS]\\b\\|\\b[eE][nN][dD][cC][oO][nN][dD]\\b\\|\\b[eE][nN][dD][iI][fF]\\b\\|\\b[eE][nN][dD][tT][aA][bB][lL][eE]\\b\\|\\b[eE][xX][iI][sS][tT][sS]\\b\\|\\b[eE][xX][pP][oO][rR][tT][iI][nN][gG]\\b\\|\\b[fF][aA][cC][tT]\\b\\|\\b[fF][aA][lL][sS][eE]\\b\\|\\b[fF][oO][rR][aA][lL][lL]\\b\\|\\b[fF][oO][rR][mM][uU][lL][aA]\\b\\|\\b[fF][rR][oO][mM]\\b\\|\\b[fF][uU][nN][cC][tT][iI][oO][nN]\\b\\|\\b[hH][aA][sS][__][tT][yY][pP][eE]\\b\\|\\b[iI][fF]\\b\\|\\b[iI][fF][fF]\\b\\|\\b[iI][mM][pP][lL][iI][eE][sS]\\b\\|\\b[iI][mM][pP][oO][rR][tT][iI][nN][gG]\\b\\|\\b[iI][nN]\\b\\|\\b[iI][nN][dD][uU][cC][tT][iI][vV][eE]\\b\\|\\b[jJ][uU][dD][gG][eE][mM][eE][nN][tT]\\b\\|\\b[lL][aA][mM][bB][dD][aA]\\b\\|\\b[lL][aA][wW]\\b\\|\\b[lL][eE][mM][mM][aA]\\b\\|\\b[lL][eE][tT]\\b\\|\\b[lL][iI][bB][rR][aA][rR][yY]\\b\\|\\b[mM][aA][cC][rR][oO]\\b\\|\\b[mM][eE][aA][sS][uU][rR][eE]\\b\\|\!
\b[nN][oO][nN][eE][mM][pP][tT][yY][__][tT][yY][pP][eE]\\b\\|\\b[nN][oO][tT]\\b\\|\\b[oO]\\b\\|\\b[oO][bB][lL][iI][gG][aA][tT][iI][oO][nN]\\b\\|\\b[oO][fF]\\b\\|\\b[oO][rR]\\b\\|\\b[oO][rR][eE][lL][sS][eE]\\b\\|\\b[pP][oO][sS][tT][uU][lL][aA][tT][eE]\\b\\|\\b[pP][rR][oO][pP][oO][sS][iI][tT][iI][oO][nN]\\b\\|\\b[rR][eE][cC][uU][rR][sS][iI][vV][eE]\\b\\|\\b[sS][uU][bB][lL][eE][mM][mM][aA]\\b\\|\\b[sS][uU][bB][tT][yY][pP][eE][sS]\\b\\|\\b[sS][uU][bB][tT][yY][pP][eE][__][oO][fF]\\b\\|\\b[tT][aA][bB][lL][eE]\\b\\|\\b[tT][hH][eE][nN]\\b\\|\\b[tT][hH][eE][oO][rR][eE][mM]\\b\\|\\b[tT][hH][eE][oO][rR][yY]\\b\\|\\b[tT][rR][uU][eE]\\b\\|\\b[tT][yY][pP][eE]\\b\\|\\b[tT][yY][pP][eE][++]\\b\\|\\b[vV][aA][rR]\\b\\|\\b[wW][hH][eE][nN]\\b\\|\\b[wW][hH][eE][rR][eE]\\b\\|\\b[wW][iI][tT][hH]\\b\\|\\b[xX][oO][rR]\\b"
("\\(<|\\||-\\||->\\||=\\||>\\|\\[\\]\\|/\\\\\\)" 1
font-lock-function-name-face) ("\\((#\\|#)\\|\\[#\\|#\\]\\)" 0
font-lock-pvs-record-parens-face) ("\\((:\\|:)\\|(|\\||)\\|(\\|)\\)!
" 1 font-lock-pvs-parens-face) ("\\(\\[|\\||\\]\\||\\[\\|\\]|\\|||\\)" 1
font-lock-pvs-table-face) ("\\({\\||\\|}\\)" 1 font-lock-pvs-set-brace-face)
("\\(\\[\\|->\\|\\]\\)" 1 font-lock-pvs-function-type-face)
("\\(!\\|!!\\|#\\|##\\|\\$\\|\\$\\$\\|&\\|&&\\|\\*\\|\\*\\*\\|\\+\\|\\+\\+\\|-\\|/\\|//\\|/=\\|/\\\\\\|::\\|:=\\|<\\|<-\\|<<\\|<<=\\|<=\\|<=>\\|<>\\|<|\\|=\\|==\\|==>\\|=>\\|>\\|>=\\|>>\\|>>=\\|ï¼ \\|ï¼ ï¼ \\|\\[\\]\\|\\\\/\\|\\^\\|\\^\\^\\||-\\||->\\||=\\||>\\|~\\)"
1 font-lock-function-name-face))
I figured that since case-fold was t, I could change all of those [lower
upper] pairs to just the lower-case letter. Doing so seems to have
reduced the font locking time by about 1 second, but I have not done
this scientifically (i.e., with repeated experiments). I was going to
try using regexp-opt to see if that sped things up any more, but I can't
use '\b' in the input to that function, and I don't see an easy way to
glom it on afterwards. How have others handled the need to match only
entire words?
Also incidentally, has anyone noticed the comment about XEmacs in the
documentation for the variable jde-java-font-lock-max-names-by-regexp?
--
Jerry James
5:56 a.m.
>>>> "Jerry" == Jerry James
<james(a)eecs.ku.edu> writes:
Jerry> I haven't yet been bugged enough to try tracking down the
Jerry> problem. If there is anything you want me to try, let me
Jerry> know.
I'd be interested in relative execution profiles (21.1 v. 21.4,
loading and locking a big slow file v. just starting) for the
following C functions (all from extent.c, I believe):
Fget_char_property
Fnext_extent_change
Fprevious_extent_change
These are the potentially heavyweight function calls added by Matt's
syntax table stuff in syntax.c, scan_lists, which is where the Lisp
profiler says time is going. They get called from the function
update_syntax_cache as needed. update_syntax_cache gets called once
per character in the region scanned AFAICT, so the question is "how
often is `as needed'?" Number of additional calls is the main thing,
some estimate of time per call would be interesting too, but it's not
primary.
Sorry, I know nothing about profiling C yet, I guess it's time to
learn.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
11:34 p.m.
"Stephen J. Turnbull" <stephen(a)xemacs.org>, on 08 Feb 2002 at 14:56:48
+0900 you wrote:
Sorry, I know nothing about profiling C yet, I guess it's time
to
learn.
Well, if you take the tack I just tried, you'll soon find out that
XEmacs is NOT set up to work with gprof. This is what I've done so
far.
1) Configure with --cflags="-g -pg" and --ldflags="-pg".
2) Edit src/Makefile.in so that it does not explicitly specify the crt
files to link with. I wasn't sure what would cause problems and what
wouldn't, so I:
a) Removed "-lgcc -lc -lgcc /usr/lib/crtn.o" from the end of ld_libs_all
b) Removed the -nostdlib flag from LD
c) Emptied start_files and lib_cc
(Is there some platform on which all this is necessary? Letting the
compiler sort all of this out certainly works fine on Linux and
Solaris, as far as I can tell.)
3) Watch temacs dump perfectly good profiling data ... and then watch
xemacs run, exit normally, and fail to write any profiling data. I
thought at first that maybe XEmacs was calling _exit() instead of
exit(). There are a number of places in the source where _exit() is
used instead of exit():
a) callproc.c: child_setup, relocate_fd
b) ecrt0.c: various startup routines
c) process-unix.c: unix_create_process
d) sysdep.c: sys_subshell, mkdir, rmdir
But, no, XEmacs is exiting via Fkill_emacs, which definitely calls
exit(). It took me awhile to figure this one out, but I finally got
it. I've been bitten by the dumping process. The glibc code for
__gmon_start__ looks like this:
------------------------------------------------------------------------
void
__gmon_start__ (void)
{
#ifdef HAVE_INITFINI
/* Protect from being called more than once. Since crti.o is linked
into every shared library, each of their init functions will call us. */
static int called;
if (called)
return;
called = 1;
#endif
/* Start keeping profiling records. */
__monstartup ((u_long) &_start, (u_long) &etext);
/* Call _mcleanup before exiting; it will write out gmon.out from the
collected data. */
atexit (&_mcleanup);
}
------------------------------------------------------------------------
So temacs, before dumping, has called = 0, and sets it to 1. The
value of 1 is then dumped, so xemacs enters __gmon_start__, says
"I've been here before!", and leaves.
Argh! This is going to make it kind of hard to profile XEmacs 21.1,
where --pdump is not available. Anyway, I used hexl mode to edit the
binary and changed the dumped value back to 0. Now I am getting profile
data out of xemacs. But this took so much time that I don't have time
to do the same to XEmacs 21.4 now. I'll get back to you later on that.
--
Jerry James
12:01 a.m.
Generally for profiling you should run temacs *not* a dumped xemacs. I
think make run-temacs should do this for you.
andy
At 05:34 PM 2/8/2002 -0600, Jerry James wrote:
"Stephen J. Turnbull" <stephen(a)xemacs.org>, on 08 Feb
2002 at 14:56:48
+0900 you wrote:
> Sorry, I know nothing about profiling C yet, I guess it's time to
> learn.
Well, if you take the tack I just tried, you'll soon find out that
XEmacs is NOT set up to work with gprof. This is what I've done so
far.
1) Configure with --cflags="-g -pg" and --ldflags="-pg".
2) Edit src/Makefile.in so that it does not explicitly specify the crt
files to link with. I wasn't sure what would cause problems and what
wouldn't, so I:
a) Removed "-lgcc -lc -lgcc /usr/lib/crtn.o" from the end of ld_libs_all
b) Removed the -nostdlib flag from LD
c) Emptied start_files and lib_cc
(Is there some platform on which all this is necessary? Letting the
compiler sort all of this out certainly works fine on Linux and
Solaris, as far as I can tell.)
3) Watch temacs dump perfectly good profiling data ... and then watch
xemacs run, exit normally, and fail to write any profiling data. I
thought at first that maybe XEmacs was calling _exit() instead of
exit(). There are a number of places in the source where _exit() is
used instead of exit():
a) callproc.c: child_setup, relocate_fd
b) ecrt0.c: various startup routines
c) process-unix.c: unix_create_process
d) sysdep.c: sys_subshell, mkdir, rmdir
But, no, XEmacs is exiting via Fkill_emacs, which definitely calls
exit(). It took me awhile to figure this one out, but I finally got
it. I've been bitten by the dumping process. The glibc code for
__gmon_start__ looks like this:
------------------------------------------------------------------------
void
__gmon_start__ (void)
{
#ifdef HAVE_INITFINI
/* Protect from being called more than once. Since crti.o is linked
into every shared library, each of their init functions will call
us. */
static int called;
if (called)
return;
called = 1;
#endif
/* Start keeping profiling records. */
__monstartup ((u_long) &_start, (u_long) &etext);
/* Call _mcleanup before exiting; it will write out gmon.out from the
collected data. */
atexit (&_mcleanup);
}
------------------------------------------------------------------------
So temacs, before dumping, has called = 0, and sets it to 1. The
value of 1 is then dumped, so xemacs enters __gmon_start__, says
"I've been here before!", and leaves.
Argh! This is going to make it kind of hard to profile XEmacs 21.1,
where --pdump is not available. Anyway, I used hexl mode to edit the
binary and changed the dumped value back to 0. Now I am getting profile
data out of xemacs. But this took so much time that I don't have time
to do the same to XEmacs 21.4 now. I'll get back to you later on that.
--
Jerry James
5:37 p.m.
On Fri, 08 Feb 2002 at 16:01:58 -0800, Andy Piper <andyp(a)bea.com> wrote:
Generally for profiling you should run temacs *not* a dumped xemacs.
I think
make run-temacs should do this for you.
Why is that? Because of the problem I described with the static
variable? Or are you worried that a dumped xemacs will start up with
profiling data from temacs loading and dumping Lisp files? If the
latter, that does not seem to be the case. I just did two runs, one
with "make run-temacs" and an immediate shutdown:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
30.22 1.62 1.62 14326733 0.00 0.00 mark_object
10.07 2.16 0.54 173 3.12 3.12 compact_string_chars
9.33 2.66 0.50 260887 0.00 0.00 re_match_2_internal
3.73 2.86 0.20 173 1.16 1.16 sweep_conses
...
and one with the dumped xemacs and an immediate shutdown:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
24.40 0.41 0.41 253616 0.00 0.00 re_match_2_internal
7.74 0.54 0.13 1079760 0.00 0.00 mark_object
5.95 0.64 0.10 1617028 0.00 0.00 readchar
4.76 0.72 0.08 45265 0.00 0.00 read_atom_0
...
Clearly, the dumped xemacs did not include the times from the run of
temacs in its profiling output. In fact, this seems to argue that it
would be better to profile the dumped xemacs and not temacs, since the
latter will be skewed due to loading up lisp. Or is there some other
lurking danger of which I am unaware?
--
Jerry James
5:57 p.m.
Its more because most profiling has been done using quantify and that
doesn't like the dumped state of the executable. But if its works for you
then great!
I have a feeling that there are hooks in the code to turn on profiling once
XEmacs has come up, so that the startup delay does not matter.
andy
At 11:37 AM 2/12/02 -0600, Jerry James wrote:
On Fri, 08 Feb 2002 at 16:01:58 -0800, Andy Piper
<andyp(a)bea.com> wrote:
> Generally for profiling you should run temacs *not* a dumped xemacs. I
think
> make run-temacs should do this for you.
Why is that? Because of the problem I described with the static
variable? Or are you worried that a dumped xemacs will start up with
profiling data from temacs loading and dumping Lisp files? If the
latter, that does not seem to be the case. I just did two runs, one
with "make run-temacs" and an immediate shutdown:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
30.22 1.62 1.62 14326733 0.00 0.00 mark_object
10.07 2.16 0.54 173 3.12 3.12 compact_string_chars
9.33 2.66 0.50 260887 0.00 0.00 re_match_2_internal
3.73 2.86 0.20 173 1.16 1.16 sweep_conses
...
and one with the dumped xemacs and an immediate shutdown:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
24.40 0.41 0.41 253616 0.00 0.00 re_match_2_internal
7.74 0.54 0.13 1079760 0.00 0.00 mark_object
5.95 0.64 0.10 1617028 0.00 0.00 readchar
4.76 0.72 0.08 45265 0.00 0.00 read_atom_0
...
Clearly, the dumped xemacs did not include the times from the run of
temacs in its profiling output. In fact, this seems to argue that it
would be better to profile the dumped xemacs and not temacs, since the
latter will be skewed due to loading up lisp. Or is there some other
lurking danger of which I am unaware?
--
Jerry James
2:56 p.m.
On 08 Feb 2002, Jerry James stated:
But, no, XEmacs is exiting via Fkill_emacs, which definitely
calls
exit(). It took me awhile to figure this one out, but I finally got
it. I've been bitten by the dumping process.
[snip clear explanation]
Thank you! I've been looking for the cause of this problem (and failing
to find it) for some time.
--
`It is to be proven that Linux Kernel is the most stable than MS Windows
that it uses less stable.' --- Bryan Parkoff being very comprehensible
11:10 p.m.
On 08 Feb 2002 at 14:56:48 +0900, "Stephen J. Turnbull"
<stephen(a)xemacs.org> wrote:
I'd be interested in relative execution profiles (21.1 v. 21.4,
loading and locking a big slow file v. just starting) for the
following C functions (all from extent.c, I believe):
Fget_char_property
Fnext_extent_change
Fprevious_extent_change
The two binaries used in the following tests were compiled with the same
compiler and the same compiler options (the RedHat 7.1 version of gcc,
which calls itself "2.96 20000731"; curses on RedHat for doing this!).
They are using the same Lisp packages. I configured them with nearly
identical options (the difference being --gung-ho for 21.1.14). The
tests were done with "-q" to eliminate effects from my customizations.
Gprof seems to think that none of those 3 functions is being called at
all in 21.1.14, although they are in 21.4.6. Here are the top 20
time-consuming functions from a run of 21.1.14 in which I simply started
PVS and then exited:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls us/call us/call name
16.54 0.22 0.22 1136031 0.19 0.22 mark_object
7.52 0.32 0.10 1623314 0.06 0.06 readchar
5.26 0.39 0.07 14610 4.79 7.75 Fbyte_code
4.51 0.45 0.06 80664 0.74 3.39 read1
3.01 0.49 0.04 300305 0.13 0.13 mark_cons
3.01 0.53 0.04 156433 0.26 0.33 find_symbol_value_1
2.26 0.56 0.03 64778 0.46 0.47 read_list_conser
2.26 0.59 0.03 41737 0.72 2.04 read_atom_0
2.26 0.62 0.03 39839 0.75 0.95 oblookup
2.26 0.65 0.03 16173 1.85 2.84 sequence_reader
1.50 0.67 0.02 1139360 0.02 0.02 get_PURESIZE
1.50 0.69 0.02 535294 0.04 0.04 Fcdr
1.50 0.71 0.02 177294 0.11 0.20 reader_nextchar
1.50 0.73 0.02 145610 0.14 0.14 Fset
1.50 0.75 0.02 103408 0.19 0.29 Lstream_flush_out
1.50 0.77 0.02 81498 0.25 0.25 mark_symbol
1.50 0.79 0.02 62370 0.32 0.32 symbol_value_buffer_local_info
1.50 0.81 0.02 36262 0.55 0.83 internal_equal
1.50 0.83 0.02 14097 1.42 3.08 funcall_lambda
1.50 0.85 0.02 5489 3.64 3.91 concat
Now here is a run of 21.1.14 in which I started PVS, loaded and
font-locked prelude.pvs, and then exited:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
26.62 1.23 1.23 73445 0.02 0.02 re_match_2_internal
6.71 1.54 0.31 400462 0.00 0.00 extent_list_locate
6.06 1.82 0.28 15324 0.02 0.03 Fbyte_code
4.55 2.03 0.21 1900525 0.00 0.00 mark_object
3.46 2.19 0.16 174054 0.00 0.00 map_extents_bytind
3.03 2.33 0.14 99071 0.00 0.00 funcall_recording_as
2.38 2.44 0.11 35414 0.00 0.00 Fstring_equal
1.73 2.52 0.08 42611 0.00 0.00 read_atom_0
1.52 2.59 0.07 1643689 0.00 0.00 readchar
1.52 2.66 0.07 242820 0.00 0.00 advance_plist_pointers
1.52 2.73 0.07 19756 0.00 0.07 sys_re_search_2
1.30 2.79 0.06 994806 0.00 0.00 buffer_or_string_extent_info
1.30 2.85 0.06 558049 0.00 0.00 mark_cons
1.30 2.91 0.06 179703 0.00 0.00 Fextent_property
1.08 2.96 0.05 288412 0.00 0.00 do_symval_forwarding
1.08 3.01 0.05 221980 0.00 0.00 specbind
1.08 3.06 0.05 11 4.55 4.55 sweep_conses
0.87 3.10 0.04 493243 0.00 0.00 Fset
0.87 3.14 0.04 230898 0.00 0.00 find_symbol_value
0.87 3.18 0.04 172407 0.00 0.00 Fcons
Next is a run of 21.4.6 in which I just start PVS and then exit:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
11.40 0.13 0.13 1085405 0.00 0.00 mark_object
6.14 0.20 0.07 1645826 0.00 0.00 readchar
5.26 0.26 0.06 15297 0.00 0.01 execute_optimized_program
5.26 0.32 0.06 7 8.57 8.57 compact_string_chars
4.39 0.37 0.05 183586 0.00 0.00 do_symval_forwarding
4.39 0.42 0.05 81429 0.00 0.00 read1
3.51 0.46 0.04 42220 0.00 0.00 read_atom_0
2.63 0.49 0.03 365211 0.00 0.00 mark_cons
2.63 0.52 0.03 76523 0.00 0.00 mark_symbol
2.63 0.55 0.03 7 4.29 4.29 sweep_conses
1.75 0.57 0.02 83613 0.00 0.00 store_symval_forwarding
1.75 0.59 0.02 82376 0.00 0.00 Fcar
1.75 0.61 0.02 58528 0.00 0.00 make_uninit_string
1.75 0.63 0.02 42220 0.00 0.00 read_atom
1.75 0.65 0.02 8683 0.00 0.00 prepare_to_modify_buffer
1.75 0.67 0.02 4134 0.00 0.00 Fmemq
1.75 0.69 0.02 1153 0.02 0.07 set_buffer_internal
1.75 0.71 0.02 7 2.86 2.86 sweep_symbols
0.88 0.72 0.01 178960 0.00 0.00 reader_nextchar
0.88 0.73 0.01 121717 0.00 0.00 find_symbol_value_1
And finally, a run of 21.4.6 in which I load and font-lock prelude.pvs,
and then exit:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
26.39 4.57 4.57 10784976 0.00 0.00 extent_list_locate
21.25 8.25 3.68 122395 0.03 0.03 re_match_2_internal
3.75 8.90 0.65 1011217 0.00 0.00 map_extents_bytind
3.41 9.49 0.59 14611763 0.00 0.00 update_syntax_cache
2.77 9.97 0.48 2579833 0.00 0.00 soe_move
2.66 10.43 0.46 8559888 0.00 0.00 extent_list_at
2.02 10.78 0.35 6358638 0.00 0.00 extent_list_locate_from_pos
1.91 11.11 0.33 1806873 0.00 0.00 extent_in_region_p
1.85 11.43 0.32 744589 0.00 0.00 Fextent_property
1.62 11.71 0.28 20110 0.01 0.02 execute_optimized_program
1.56 11.98 0.27 12375343 0.00 0.00 buffer_or_string_extent_info
1.21 12.19 0.21 2060087 0.00 0.00 mark_object
1.15 12.39 0.20 2227380 0.00 0.00 gap_array_insert_els
1.10 12.58 0.19 4996504 0.00 0.00 gap_array_adjust_markers
1.10 12.77 0.19 2521559 0.00 0.00 get_buffer_pos_char
1.04 12.95 0.18 6094395 0.00 0.00
buffer_or_string_stack_of_extents_force
1.04 13.13 0.18 929921 0.00 0.00 extent_at_bytind
1.04 13.31 0.18 819566 0.00 0.00 extent_find_beginning_of_run
0.98 13.48 0.17 1113690 0.00 0.00 extent_list_insert
0.98 13.65 0.17 1089475 0.00 0.00 advance_plist_pointers
The full gprof results are temporarily available here, in the order
presented above:
<URL:http://www.ittc.ku.edu/~james/xemacs/21.1.14.prof>
<URL:http://www.ittc.ku.edu/~james/xemacs/21.1.14.prelude.prof>
<URL:http://www.ittc.ku.edu/~james/xemacs/21.4.6.prof>
<URL:http://www.ittc.ku.edu/~james/xemacs/21.4.6.prelude.prof>
Each file is a bit over 1 MB in size. Let me know if you want me to try
anything else.
--
Jerry James
7:45 a.m.
On 08 Feb 2002, Stephen J. Turnbull spake:
Remember all we're doing here is to _indent one line_. In the
case of
the TAB (which causes twice as many scans!!), font-lock should not
change anything---if it does there's a bug! Arguably font-lock (less
likely, cc-mode) is broken. In languages where indentation is not
syntactically significant, there is no good reason for changes in
indentation to cause XEmacs to go charging up and down the buffer
scanning sexps.
Um, since C is a (somewhat) context-sensitive language, and since
cc-mode doesn't actually know the grammar of C properly, arbitrary
amounts of chasing over the buffer may be required to work out what the
right indentation should be (it has to go to the top of the defun then
count bracelike constructs all the way down to point, at least). In some
(admittedly pathological) cases involving K&R declarations and/or
hanging-brace K&R brace style, it may have to chase all the way up to
the top of the buffer and back down to work out what your indentation
level is.
And there are scan-sexps calls in `c-forward-token-1',
`c-backward-token-1', and `c-parse-state', which are the workhorses of
the cc-mode indentation engine; the first two of these are called
basically all the time :)
I don't think this will be fixable without truly massive changes to
cc-mode (which would probably cripple its indentation engine again).
--
`It is to be proven that Linux Kernel is the most stable than MS Windows
that it uses less stable.' --- Bryan Parkoff being very comprehensible
8:49 a.m.
>>>> "nix" == nix
<nix(a)esperi.demon.co.uk> writes:
nix> On 08 Feb 2002, Stephen J. Turnbull spake:
> Remember all we're doing here is to _indent one line_.
nix> Um, since C is a (somewhat) context-sensitive language, and
I doubt the minimal degree of context sensitivity in C/C++/Java is
what's relevant here. I'm sure it's dominated by the fact that the
indentation itself is context-sensitive. But I thought context was
relative, that is invariably local to the current list and its prefix
(eg, there are all kinds of switch () {} conventions, and they
generally are quite different from things like function definitions
that look sort of like that).
I gather you're trying to tell me that's false?
nix> In some (admittedly pathological) cases involving K&R
nix> declarations and/or hanging-brace K&R brace style, it may
nix> have to chase all the way up to the top of the buffer and
nix> back down to work out what your indentation level is.
Yeah, but it doesn't seem to be an issue of pathology. It simply
seems to be a matter of buffer size. It seems that something is
_always_ chasing indentation or syntax over the whole buffer.
Did you look at the profile? One indentation causes 998 scan-lists in
a 7000 line file! What with comment lines and the like, it sounds to
me like it's doing a scan-list for a very large proportion of the
lists in the file. This could very easily mean that get-text-property
happens on the order of N^2 due to nesting of lists. Even
microseconds add up quickly in units of 100^2.
nix> I don't think this will be fixable without truly massive
nix> changes to cc-mode (which would probably cripple its
nix> indentation engine again).
That would be a bummer. But it would also be a bummer to find
ourselves forced to give up local syntax tables just because one
important application is wedded to an N^2 algorithm or something like
that.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
9:10 a.m.
>>>> "Stephen" == Stephen J Turnbull
<stephen(a)xemacs.org> writes:
>>>> "nix" == nix
<nix(a)esperi.demon.co.uk> writes:
nix> On 08 Feb 2002, Stephen J.
Turnbull spake:
>> Remember all we're doing here is to _indent one line_.
nix> Um, since C is a (somewhat) context-sensitive language, and
Stephen> I doubt the minimal degree of context sensitivity in
Stephen> C/C++/Java is what's relevant here. I'm sure it's
Stephen> dominated by the fact that the indentation itself is
Stephen> context-sensitive.
Oops, brain-o, ignore the above. Failure to mentally change gears
between "context-sensitive language" and anticipating the everyday use
of "context" in the main point below.
Stephen> But I thought context was relative, that is invariably
Stephen> local to the current list and its prefix [...]
Stephen> I gather you're trying to tell me that's false?
Note that this in the context (yet another use of that word!) of
fontlock and indentation, where an approximation ought to be good
enough.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
2:17 p.m.
On 08 Feb 2002, Stephen J. Turnbull muttered drunkenly:
nix> In some (admittedly pathological) cases involving
K&R
nix> declarations and/or hanging-brace K&R brace style, it may
nix> have to chase all the way up to the top of the buffer and
nix> back down to work out what your indentation level is.
Yeah, but it doesn't seem to be an issue of pathology. It simply
seems to be a matter of buffer size. It seems that something is
_always_ chasing indentation or syntax over the whole buffer.
OK.
Did you look at the profile? One indentation causes 998 scan-lists
in
a 7000 line file! What with comment lines and the like, it sounds to
me like it's doing a scan-list for a very large proportion of the
lists in the file.
If I read the code correctly, it would need to do this if
c-recognize-knr-p was non-nil and your function headers were indented
(and maybe in other circumstances too; once, it needed to do it whenever
the K&R-style 1TBS was used, but I'm not sure if this is still the
case.)
nix> I don't think this will be fixable without truly
massive
nix> changes to cc-mode (which would probably cripple its
nix> indentation engine again).
That would be a bummer. But it would also be a bummer to find
ourselves forced to give up local syntax tables just because one
important application is wedded to an N^2 algorithm or something like
that.
Silly suggestion, but why not add an extra optional parameter to
scan-lists to turn off the use of local syntax tables for that call?
That way, pathological code like that in cc-mode that is found to be too
slow can have this new parameter set to t whenever it calls scan-lists
(in the hot spots where it does so pathologically often), and it can
speed up again without losing local syntax tables in more than one
place.
(Disclaimer: This may well not actually be possible: I haven't read the
code. If what I'm saying is garbage, please disregard it :) )
--
`It is to be proven that Linux Kernel is the most stable than MS Windows
that it uses less stable.' --- Bryan Parkoff being very comprehensible
10:25 a.m.
Nix <nix(a)esperi.demon.co.uk> writes:
Silly suggestion, but why not add an extra optional parameter to
scan-lists to turn off the use of local syntax tables for that call?
That sounds like a kludge to me. Which is not to say we shouldn't
resort to optimization-motivated kludges if all else fails, but let's
first make sure that we have tried all else.
The other thing is, on some level it's simply wrong to even provide a
parameter that just ignores the local syntax tables. Whatever set up
the code local syntax tables has probably done it for good reasons,
which the cc-mode might not even be aware of. Turning them off with a
flip of a lever will undermine the (correct) code that tries to make
use of the feature.
8:14 a.m.
New subject: fontlock-speed, any news ?
Hi,
I was just wondering if anyone was actually working on what you all
talked about to speed things up for font-lock. I ended up disabling
font-lock these days because it was simply not doable anymore and i
wanted more than 21.1. Not that i'm holding my breath, but just out of
curiosity, has this matter been put aside or there is someone actually
implementing all the great improvements you talked about (i have to
admit you lost me after one of two technical discussions about internals
;)
thanx.
--
# Stef Epardaud, # Teachers have potentially more power than military,
# Java Defeater # the former can teach us how to not need the latter.
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
8:42 a.m.
New subject: fontlock-speed, any news ?
>>>> "Stef" == Stef Epardaud
<stef(a)lunatech.com> writes:
Stef> I was just wondering if anyone was actually working on what
Stef> you all talked about to speed things up for font-lock.
Not that I know of. Personally I need to get a 21.4.7 and 21.5.5 out
the door, and then I'll try and formalize a few of these important
projects. This (fontlock) is _the most important_ one, but I really
should get the small improvements from the last couple of months out
the door, and I really would like a release of 21.5.5 before Ben
completes his merge.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Don't ask how you can "do" free software business;
ask what your business can "do for" free software.
8:54 a.m.
New subject: fontlock-speed, any news ?
On Thu, Feb 28, 2002 at 05:42:39PM +0900, Stephen J. Turnbull wrote:
Not that I know of. Personally I need to get a 21.4.7 and 21.5.5
out
the door, and then I'll try and formalize a few of these important
projects. This (fontlock) is _the most important_ one, but I really
should get the small improvements from the last couple of months out
the door, and I really would like a release of 21.5.5 before Ben
completes his merge.
alright, that sounds good that it's rated important already :)
I'm not sure what this Ben merge is but it seems important as i've heard
about it a lot these days (obvious display of my ignorance).
anyways, i'm very happy that somebody plans to work on font-lock, if you
get some improvements or need testing, or even if you have a faintest
idea of an estimate date for this, i'll be glad to hear/help.
thanx again.
--
# Stef Epardaud, # Teachers have potentially more power than military,
# Java Defeater # the former can teach us how to not need the latter.
# Earth # Lunatech Research,
# Solar System # soon we'll quit researching and start finding...
8091
days inactive
8114
days old
56 comments
10 participants
participants (10)
-
Adrian Aichner -
Andy Piper -
Ben Wing -
Hrvoje Niksic -
Jerry James -
Matt Tucker -
Nix -
Norbert Schirmer -
Stef Epardaud -
Stephen J. Turnbull