Syntax cache is the holdup [was: releasing 21.4 as stable]

Description of status as I understand it below. But first, a Word Ben, Jan: If either of you could do something about completing Jan's patch referenced below, that would be very helpful. It's beyond me. Also, Jan's patch introduces a "full-buffer" extent to allow the syntax cache to work the same in regions without a syntax-table property as with it. Ben has a comment in his patch that full-buffer extents slow things down. What's the story here? Ben: don't worry about porting your patch to 21.4. It looks mostly mechanical to deal with the renaming of the *pos types; I can do that and will ask for help when I need it. Anybody else who thinks they understand the syntax stuff is welcome to jump in, too, but please coordinate with Ben and Jan. Andy> So I think that a sync is necessary and not too Andy> difficult. Stephen and I have reached some sort of Andy> resolution on this I think. Agreed. There is a difference in philosophies. But Windows has never been on the critical path. Right. The slowdown may be as much as two orders of magnitude, I haven't measured against 21.1 or GNU Emacs. "Canonical" test case: put the file in http://turnbull.sk.tsukuba.ac.jp/Tools/XEmacs/TraverseSchema.java somewhere, adjust the following expr to point to it, (progn (switch-to-buffer "*tmp*") (insert-file (expand-file-name "~/TraverseSchema.java")) (goto-line 8942) (search-forward ", ") (open-line 1) (forward-char) (java-mode) (let ((result (profile (c-indent-command)))) (switch-to-buffer "*scratch*") (insert (format "%s" result)))) execute, wait 15-60 secs depending on your processor and state of swapping. Jan demonstrates a possibility of a 10x at least speedup in http://list-archive.xemacs.org/xemacs-beta/200204/msg00318.html However, this patch doesn't actually manage the syntax cache (it basically will never invalidate it), and worse, it (accidentally, AFAICT) removes the option for turning it off. (AFAIK only cperl mode currently uses the cache, because it is only necessary if you use the 'syntax-table text property.) I may include this patch in a beta based on 21.4.9, but it is NOT a candidate for release. Ben suggests using next-single-property-change in a followup; this change by itself does not help. AFAICT it does the same thing the syntax cache code currently does, except at a lower level. To balance any possible speedup, it introduces a bunch of extra Lisp/C conversions. Ben's suggestion combined that with smarter code to invalidate the cache less often; I don't understand that because that's exactly what this is suppose to do. Ben has a thorough rewrite of the syntax cache stuff in http://list-archive.xemacs.org/xemacs-patches/200204/msg00217.html Unfortunately it includes the usual quota of spurious type renamings, making it impossible to see how big a change it really is without actually porting it. I plan to tackle that next, as I don't know how to complete Jan's patch. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN My nostalgia for Icon makes me forget about any of the bad things. I don't have much nostalgia for Perl, so its faults I remember. Scott Gilbert c.l.py