10:56 a.m.
I ran the Cywgin `xemacs'; I notice it's quite fast once it starts up but
dog-slow doing so.
I imagine much of the time is spent doing multiple stats down through the
package tree.
I'd like to suggest the following:
The basic idea is that upon first startup XEmacs remembers the package roots
and the contents and modification dates of all the directories it has to
traverse. Ideally this remembrance should be done in some system-wide place
but this may not be possible unless xemacs is running as `root' --
therefore, either we do this automatically when xemacs is run this way, or
we just do it using a flag, which can function as a way of a sysadmin
updating the cache. In other circumstances, use a file in ~/.xemacs.
Then, next time, use this cache instead of actually reading the directories.
However, go through each resulting directory in load-path and stat it to see
what the modification time is; if so, we need to discard the file listing
for that directory and reread it (thereby potentially adding new directories
to the cache that need to be recursed into, or removing directory trees from
the cache).
As far as I know, this algorithm/procedure has many advantages:
[a] simple
[b] 100% accurate
[c] relatively fast -- it requires that we stat only once per directory
instead of once per directory entry, plus three times extra per directory
(for default, default.el, default.elc), plus probably other overhead.
[d] allows but does not require the administrator to run a command to
generate the cache
6:22 p.m.
>>>> "Ben" == Ben Wing <ben(a)666.com>
writes:
Ben> I ran the Cywgin `xemacs'; I notice it's quite fast once it starts up but
Ben> dog-slow doing so.
Ben> I imagine much of the time is spent doing multiple stats down through the
Ben> package tree.
Unless something's changed that I don't know about, it does only one
pass. But you're right that they take up some time.
Ben> Then, next time, use this cache instead of actually reading the directories.
Ben> However, go through each resulting directory in load-path and stat it to see
Ben> what the modification time is; if so, we need to discard the file listing
Ben> for that directory and reread it (thereby potentially adding new directories
Ben> to the cache that need to be recursed into, or removing directory trees from
Ben> the cache).
It seems to me this would incur the same number of stats as we have.
We don't actually stat the individual files---just the directories.
Or am I misunderstanding something you said?
--
Cheers =8-} Mike
Friede, Völkerverständigung und überhaupt blabla
2:56 a.m.
Ben> I ran the Cywgin `xemacs'; I notice it's quite fast
once
it starts
Ben> up but dog-slow doing so.
Ben> I imagine much of the time is spent doing multiple stats down
Ben> through the package tree.
Unless something's changed that I don't know about, it does
only one pass. But you're right that they take up some time.
I see.
Ben> Then, next time, use this cache instead of actually reading
the
Ben> directories. However, go through each resulting directory in
Ben> load-path and stat it to see what the modification time
is; if so,
Ben> we need to discard the file listing for that directory
and reread
Ben> it (thereby potentially adding new directories to the cache that
Ben> need to be recursed into, or removing directory trees from the
Ben> cache).
It seems to me this would incur the same number of stats as
we have. We don't actually stat the individual files---just
the directories. Or am I misunderstanding something you said?
Well, in that case my strategy would require only a single stat, not a stat
of each directory. I was assuming you had to stat each file of each
directory to see if it was a subdirectory.
Btw I just did an strace under Cygwin and found some interesting things.
Unfortunately the results appear skewed; could some people under Unix run
strace (trace, truss, etc.) and send me the results?
Some things I noticed:
-- each .xpm file in the toolbar is getting loaded 3 times -- once when the
fallback, once when the X value, once when the mswin value. Under Cygwin,
for some reason, these loads are taking absolutely forever; maybe it's
because we're using XpmReadDataFromFile. Simple caching could avoid the 3x
load, but I wonder if we shouldn't just dump the XPM info directly into the
exe?
-- every time you load an autoload, you get the following sequence:
1. stat foo.elc. This occurs in locate_file_in_directory_mapper,
`locate-file' from `load'.
2. access foo.elc. From locate_file_open_or_access_file, same..
3. stat foo.elc. From `insert-file-contents' from `find-magic-cookie...'
from `load'.
4. open foo.elc. From same.
5. lseek(#, 1, SEEK_SET); lseek(#, 0, SEEK_CUR); fstat; readv(808 bytes);
readv(0 bytes); lseek(#, 1, SEEK_SET); readv(808 bytes); readv(0 bytes);
readv(0 bytes); close. From same.
6. stat foo.elc. This occurs in locate_file_in_directory_mapper,
`locate-file' from `load-internal'.
7. open foo.elc. From locate_file_open_or_access_file, same.
8. fstat. From load-internal, checking for out-of-date elc.
9. stat foo.el. From same.
10. lseek(#, 0, SEEK_CUR); fstat; lseek(#, 0, SEEK_SET); readv(809 bytes);
readv(0 bytes); readv(0 bytes); close
A lot of statting here for one file.
#5 appears to be
lseek(#, 1, SEEK_SET) from insert-file-contents-internal
lseek(#, 0, SEEK_CUR) from make_filedesc_stream_1
fstat from filedesc_seekable_p, undecided_init_coding_stream
read(3000, returns 808 bytes) from i-f-c-i -> Lstream_read -> coding_reader
-> Lstream_read -> filedesc_reader
read(0, returns 0 bytes) from same
lseek(#, 1, SEEK_SET) ??? Can't find this
read = readv(808 bytes) ??? Can't find this
read = readv(0 bytes) ??? Can't find this
read(0, returns 0 bytes) from same
close
#10 appears to be
lseek(#, 0, SEEK_CUR) from make_filedesc_stream_1
fstat ??? Can't find this
lseek(#, 0, SEEK_SET) ??? Can't find this
read(8192, returns 809 bytes)
read(8192, returns 0 bytes)
read(8192, returns 0 bytes)
close
"Can't find this" is presumably because something changed (for the better)
between 21.4 and 21.5; but it might be a Windows-Cygwin difference, since
the strace is 21.4 Cygwin and the debugging is 21.5 Windows.
BTW the 1 in lseek is due to the "son of a bitch" bug I just corrected.
2:34 p.m.
>>>> "Ben" == Ben Wing <ben(a)666.com>
writes:
Ben> Well, in that case my strategy would require only a single stat, not a stat
Ben> of each directory. I was assuming you had to stat each file of each
Ben> directory to see if it was a subdirectory.
No. We used to do that very early in the game, but no longer. The
relevant definitions that control this aspect of path searching are
these:
(defvar packages-hierarchy-depth 1
"Depth of package hierarchies.")
(defvar packages-load-path-depth 1
"Depth of load-path search in package hierarchies.")
(defvar packages-data-path-depth 1
"Depth of data-path search in package hierarchies.")
Ben> Btw I just did an strace under Cygwin and found some interesting things.
Ben> Unfortunately the results appear skewed; could some people under Unix run
Ben> strace (trace, truss, etc.) and send me the results? [...]
Yes---these are all stats where we would benefit from getting rid of
them. But they're all below the level of the path searching.
--
Cheers =8-} Mike
Friede, Völkerverständigung und überhaupt blabla
7075
days inactive
7076
days old
3 comments
2 participants
participants (2)
-
Ben Wing -
Michael Sperber