David> Ok, so it would appear that the next release announcement David> should essentially have to contain something like: David> Since at the current point of time no XEmacs developer David> can be interested enough in preview-latex to even answer David> questions, the Why, how can you possibly conclude that? I estimated that what with the lack of any XEmacs-specific information on a rarely manifesting bug dealt with by a non-XEmacs developer, you would realize that it would take at least 10 days to do the necessary research, and have a bit of patience. Since you are in a hurry, my answer is "I'm sorry, but I have no idea what you are talking about; I simply don't recall any such report. If you would give us test cases and information about versions and configurations of XEmacs so that we can replicate the bug with the version in question, then we can probably get back to you about current status, including exploring various configurations that might be relevant and whether it has been solved or can easily be solved in XEmacs---in about a week. If we have to do all that research, it'll be several weeks, if it happens at all." You're certainly welcome to promote GNU Emacs at the expense of XEmacs in your announcement; I've encouraged you to do that in the past, as I see no constituency (aside from maybe poor Uwe Brauer, who works his butt off) for preview-latex among XEmacs users. Alternatively, you can tell your users to do their part by making a fuss about the bugs that bite them on XEmacs channels. But please leave the XEmacs developers' attitude out of it; we would be quite happy to help your XEmacs users if there were evidence that they want it. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

David> Because my request that included the specific date and time David> and comment from the XEmacs developer that could be dug out David> from the CVS logs _Your_ CVS logs, and AFAICT Nix is nothing like an XEmacs developer who happens to use preview-latex. I would absolutely _love_ to be proven wrong about that, too! "C'mon, Nix, who loves ya, baby!" David> It would appear obvious that _if_ somebody was intending to David> do the necessary research but considered it to take some David> time, he would indicate this, at the very least to save David> others doing unnecessary work. Why, David, I had no intention of doing the necessary work if Nix, or Uwe, or Steve Youngs, or Richard Stallman would do it for us. Some of these people have lives and/or spend much of their time "at the end of a long time lag...". So I was going to give them a week, which coincidentally coincides with a good time for me. I don't like the situation much better than you do, but there's only so much I can do single-handedly. See also http://list-archive.xemacs.org/xemacs-beta/200501/msg00297.html. In other words, no, I don't think anything's obvious about that. David> The bug was pretty severe: complete corruption of all image David> loads by :file specifier after dired has been loaded. Sounds spectacular! Nope, no spectacular bugs here, sorry. David> A thorough research on the mailing list also turns up David> <URL:http://sourceforge.net/mailarchive/message.php?msg_id=2367671> David> where Nick describes the problem origin after hunting it David> down. Thank you. Now we have a place to start---this is actually about 20% of the way to a bug report at this point! David> I don't see why I should try to make a secret of that. David> That is not bringing anybody's "attitude" into play, as I David> don't indulge in speculation about the reasons. No, your users will do that for you, because they know and respect you. For good reason, too, and that's why I ask you to lighten up a bit. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

On Tue, 01 Feb 2005, Stephen J. Turnbull whispered secretively: I'm afraid I've suffered multiple hardware failures and near-crippling RSI over the last year plus, which has pretty much killed my ability to develop *anything* :( I've got a new keyboard (yay Maltron!) and my RSI is fading accordingly, and I'm slowly easing myself back into things; I expect that in a month or so I'll be able to actually *do* something about some of the millions of bugs I triggered that everyone I know seemingly keeps, er, bugging me about. Apologies! My guilt glands are in full flow :( I'll see what I can do to compensate by describing this bug a little more clearly. (I've learnt a lot about that sort of stuff since 2002: I've been doing a lot more reviewing other people's code in my day job, so I had to get better...) Oh dear, I remember this little monster now. Despite my poor wording in that post it is *not* a dired bug; it's a problem with the interaction between insert-file-contents-internal and the file-name-handler-alist. ... and no, the underlying bug is *not* fixed, David: sorry. This is my fault: I reprehensibly vanished off the face of the Earth before fixing it. So here's the bug report I was too dilatory to provide (or maybe I did provide it: I can't remember.) A:file-based extent read of a PNG file (which is what we were trying to do) trips png_normalize(), which trips off simple_image_type_normalize(), which calls make_string_from_file() to read the file data in. That's how basically all images get read in XEmacs right now, if I follow the code correctly. *That* calls Finsert_file_contents_internal like so: Finsert_file_contents_internal (file, Qnil, Qnil, Qnil, Qnil, Qnil, Qnil); Because the last two arguments are nil, no charset conversion should happen, which is obviously the right thing in this case. Finsert_file_contents_internal says, near the top, /* If the file name has special constructs in it, call the corresponding file handler. */ handler = Ffind_file_name_handler (filename, Qinsert_file_contents); if (!NILP (handler)) { val = call6 (handler, Qinsert_file_contents, filename, visit, start, end, replace); goto handled; } i.e., if there's a matching handler, it gets called, *and that handler does not get told what coding systems we're using for this read*. Now dired has a function `dired-check-file-name-handler-alist', which inserts, as I said, '("." . dired-handler-fn) into the file-name-handler-alist whenever there is a dired buffer open. That function's sole purpose is to track updates to directories that dired buffers are open on, so they `magically update'. So it inhibits itself and thunks down to the underlying call for absolutely every operation: (apply op args) So the call chain runs like this: call applicable coding system for file ops png_normalize coding-system-for-read simple_image_type_normalize coding-system-for-read make_string_from_file coding-system-for-read Finsert_file_contents_internal nil dired-handler-fn (or any other coding-system-for-read <----- handler fn) Finsert_file_contents coding-system-for-read Finsert_file_contents_insernal coding-system-for-read The problem happens to be tripped internally by the glyph-instantation code but is *not* specific to it: you can reproduce it by opening a dired buffer and trying to read something into any buffer at all using `insert-file-contents-internal' with CODESYS set to anything other than what `coding-system-for-read' is set to. Watch the CODESYS get ignored. (I saw this on a non-MULE XEmacs: file-coding being applied at the wrong time is enough to corrupt PNG files quite badly!) The only solution I can think of is to locally bind the codesys argument to `coding-system-for-read' inside Finsert_file_contents_insernal: would that be acceptable? If it is, I can spin a patch in the next day or so. [snip more bickering caused by my dilatoriness, argh! Sorry!] -- `Blish is clearly in love with language. Unfortunately, language dislikes him intensely.' --- Russ Allbery

On Fri, 04 Feb 2005, Stephen J. Turnbull wrote: I have. It's excellent... but the single largest RSI-saver (voice recognition) doesn't work well when you enunciate as poorly and variably as I do. So a Maltron keyboard it is, and now, after only a few weeks, I can't imagine why these keyboards aren't in use *everywhere*. They whip the flat pasty white rear of QWERTY keyboards. :) (My RSI wasn't as bad as Ben's, which sounds (reads?) truly horrific. It was partially Ben's cautionary example which led me to ease off as hard as I did, and may well have saved my hands. Thanks, Ben!) Maybe. I'm behind a superfascist corporate firewall right now which blocks xemacs.org, so I can't check the archives. Please! The last thing I want to do is cause a repeat of the XEmacs-21.4.8 fiasco. Will do... This is a completely tentative patch, bceause I'm still not quite clear on the interactions of bindings, GCPRO, unwind-protect and the like in the C layer. I *think* it's pointless to specbind() DEFVAR_LISPed variables, because it seems to me that this would break the magic link between the C variable and the Lisp symbol --- the magic forwarding symbol would wind up on the specbinding stack --- and the C code uses the variable... so I've tried to record_unwind_protect it instead. I'm not quite sure if I've done it right, or if I need to GCPRO something (I don't think so: stuff on the specbinding stack and stuff in DEFVAR_LISPed variables are both found by the garbage collector anyway, right?) record_unwind_protect()?) It compiles, I don't know if it runs, I don't know if it's safe, I don't know if there's a better way to do it: 2005-02-04 Nix <nix(a)esperi.org.uk&gt; * fileio.c (restore_read_coding_system_unwind): New function. (Finsert_file_contents_internal): Locally bind the read coding system around file-name-handler-alist calls. Index: src/fileio.c =================================================================== RCS file: /pack/xemacscvs/XEmacs/xemacs/src/fileio.c,v retrieving revision 1.66.2.6 diff -u -r1.66.2.6 fileio.c --- src/fileio.c 2004/06/23 01:54:46 1.66.2.6 +++ src/fileio.c 2005/02/04 10:58:06 ＠＠ -298,6 +298,17 ＠＠ return Fset_marker (point_marker, Qnil, Qnil); } +#ifdef FILE_CODING +/* Restore coding-system-for-read. */ + +static Lisp_Object +restore_read_coding_system_unwind (Lisp_Object coding_system) +{ + Vcoding_system_for_read = coding_system; + return Qnil; +} +#endif + /* Versions of read() and write() that allow quitting out of the actual I/O. We don't use immediate_quit (i.e. direct longjmp() out of the signal handler) because that's way too losing. ＠＠ -2790,6 +2801,11 ＠＠ an unwind_protect */ filename = Fexpand_file_name (filename, Qnil); + +#ifdef FILE_CODING + record_unwind_protect (restore_read_coding_system_unwind, Vcoding_system_for_read); + Vcoding_system_for_read = codesys; +#endif /* If the file name has special constructs in it, call the corresponding file handler. */ OK, now *that* goes in my sig where Russ can see it. :) -- `Blish is clearly in love with language. Unfortunately, language dislikes him intensely.' --- Russ Allbery

On Fri, 04 Feb 2005, David Kastrup said: Agreed: but what pain I had simply dissolved shortly after I started using the new keyboard, and hasn't been back. (The RSI was light enough that stopping typing for a few days made most symptoms fade completely in any case; I'd not gone anywhere near as far as Ben. The only problem was that starting typing again made things hurt within a quarter hour... that doesn't happen anymore. Downside: they're bloody expensive keyboards; not things to drop your drink in.) s/character/paranoia/:) [snip] I think I overspoke. It felt nasty, but I think it was just moderate. Compared to Ben, it's probably `light'. Only because I was being completely paranoid and saying `any pain, stop right now' combined with very strict type-break.el timings (still in force; fifteen minutes on, five minutes off, maximum of one-half day spent doing any typing at all: work is annoyed, but there's nothing they can do other than pay for expensive keyboards in the hope that things improve). I've seen what *bad* RSI does and I don't want it coming anywhere near me. This crippled my *productivity*, but it didn't cripple *me*, if you see what I mean. i.e., my disappearances have been preventative maintenance. True. I hope not! I've tried them (well, a couple of years back, when a friend got one). They have a ~40% error rate with me. I'm trying to avoid that (hence the intermittent online presence). Most of the package I posted today was written over the preceding week (I had the germ of the idea last weekend). I'd not try writing that much in one day at the moment. I think the coding-system stuff is different. -- `Blish is clearly in love with language. Unfortunately, language dislikes him intensely.' --- Russ Allbery

What is a maltron keyboard? Btw voice recognition sucks, unless something has recently changed dramatically, which I find hard to believe since I'm studying this field right now :) Worse than you can imagine. But something else obviously went wrong besides just plain rsi, although clearly triggered by it. Exactly what, nobody knows. If you want to hear about something extremely scary, go look up "Reflex Sympathetic Dystrophy". This isn't what happened to me, but the out-of-proportion reaction and the progressiveness are similar.

This is a singularly bad name :( In Spanish, "maltron" reminds one of "mala electronica" = "bad electronics".

"Ben Wing" <ben(a)666.com&gt; writes: Whatever. I think it is rather likely that the fix is not in the yet released 21.4.17, and even if it were, I don't want to deprecate 21.4.15 for preview-latex use if I can avoid it. I take it that this bug has not been encountered except by people using special software, and this even though XEmacs comes with tool bars and things. Am I right to conclude that the xbm and xpm image loading modes of XEmacs (which are the default formats used for icons unless I am mistaken) basically are not affected by this bug since they are principally ASCII formats and seemingly coded in a manner that don't mind if the end of line conversion is set to some weird value via the dired.el operation? If this could be asserted, it would make loading preview-latex somewhat faster under XEmacs and would remove the necessity for at least some workaround code. We would still need workaround code for the actual in-text images generated by TeX (which are usually PNG), but for nothing else. Correct? -- David Kastrup, Kriemhildstr. 15, 44793 Bochum

Nix <nix(a)esperi.org.uk&gt; writes: You just watch your wrists, nobody's upset. Well, I was talking nonsense here. The load time of preview-latex is not really affected, at least now, because we still search for the icon images anyway. Strictly speaking, this does not make sense since the package installation process should be aware of where it placed the images instead of groping all over the disk for them. But that's a preview-latex internal problem and does not concern XEmacs. The previous problem where the icons needed to be present at compilation time is gone. I changed the respective code in preview-latex. This is the stuff that is actually costing performance since a preview-latex run can easily include hundreds to thousands of images, most of which will probably not ever need loading before they get replaced by the next batch, because the part of the file where they are located might not be actually displayed. This is an overdesigned excuse... Anyway, once this gets fixed and the fix is operative, I'd be glad to hear a related _condition_ with which we can reliably test for the presence of the fix (emacs-version<= does not cut it since both Emacs 21.4 and 21.5 series are involved), so that we can then stop preloading potentially unneeded image files in preview-latex and leave that job to XEmacs, once XEmacs is capable of dealing with it. Thanks, -- David Kastrup, Kriemhildstr. 15, 44793 Bochum

"Stephen J. Turnbull" <stephen(a)xemacs.org&gt; writes: Well, the way it looks we are going to use the following autoconf macro (EMACS_LISP is internal to our autoconf scripts as well): # AC_LISPIFY_DIR # First argument is a variable name where a lisp expression is to be # substituted with AC_SUBST. # Second argument is the original path in shell-quoted syntax, usually # something like [["${whatever}"]]. # If the expression is not an absolute path, it is evaluated relative # to the current file name. AC_DEFUN(AC_LISPIFY_DIR,[ EMACS_LISP([$1],[[(prin1-to-string (if (file-name-absolute-p path) (expand-file-name (file-name-as-directory path)) (backquote (expand-file-name (, (file-name-as-directory path)) (file-name-directory load-file-name)))))]],,-no-site-file,path,[$2]) AC_SUBST([$1])] ) You place something like AC_LISPIFY_DIR(lisppackageimagedir,[[${packageimagedir}]]) into configure.ac, and then in any *.in file a string (defvar preview-image-dir ＠lisppackageimagedir＠ "The directory where preview finds its images.") gets replaced by "c:\\stupid\\dirname\\with\\quotes and spaces\\" if $packageimagedir is an absolute directory path, and with (expand-file-name "../images/" (file-name-directory load-file-name)) if $packageimagedir is a relative path (which it will be when we are creating an XEmacs package). The latter is the sort of thing that will get used in XEmacs packages for AUCTeX as well as preview-latex: locating the images relative to the package containing (among other things) the autoload definitions for the package. There is no reason this purely relative scheme could not well be used in XEmacs packages. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum

[Cc: list drastically trimmed] On Fri, 4 Feb 2005, Skip Montanaro mused: Interesting: I get it most in *cold* weather... Snap.:) Well, Noah Friedman had written it (type-break.el) already; I just had to hack it a little bit (to make it stricter and remove the code that let me skip typing breaks). That really does look nifty. I think I might try to tie it into type-break, if I could think of a benefit. I may well have a look at it: it looks more developed than what I'm presently using. Thanks for the pointer! -- `Blish is clearly in love with language. Unfortunately, language dislikes him intensely.' --- Russ Allbery

* Uwe Brauer (2005-02-02) writes: What!? I was being censored? You [beeep]! That you haven't encountered problems doesn't mean they don't exist. Here is one of my all time favorites regarding font locking which is working fine in Emacs but fails in XEmacs, i.e. fontification is spilling out of the verbatim environment: \documentclass{article} \begin{document} \begin{verbatim} echo $LANG \end{verbatim} \end{document} XEmacs' font-lock.el sucks noodles when it comes to syntactic keywords. BTW, I had to write the above example in Emacs as XEmacs died when I wanted to test it. Here is the message I got when not running in GDB: --8<---------------cut here---------------start------------->8--- Fatal error (11). Your files have been auto-saved. Use `M-x recover-session' to recover them. Your version of XEmacs was distributed with a PROBLEMS file that may describe your crash, and with luck a workaround. Please check it first, but do report the crash anyway. Please report this bug by invoking M-x report-emacs-bug, or by selecting `Send Bug Report' from the Help menu. If necessary, send ordinary email to `crashes(a)xemacs.org&#39;. *MAKE SURE* to include the XEmacs configuration from M-x describe-installation, or equivalently the file Installation in the top of the build tree. *Please* try *hard* to obtain a C stack backtrace; without it, we are unlikely to be able to analyze the problem. Locate the core file produced as a result of this crash (often called `core' or `core.<process-id>', and located in the directory in which you started XEmacs or your home directory), and type gdb /usr/bin/xemacs core then type `where' at the debugger prompt. No GDB on your system? You may have DBX, or XDB, or SDB. (Ask your system administrator if you need help.) If no core file was produced, enable them (often with `ulimit -c unlimited' in case of future recurrance of the crash. Lisp backtrace follows: re-search-backward("\\(\\\\\\(new\\|provide\\)command\\*?{?\\\\\\([a-zA-Z]+\\)]*\\)\\]\\)\\|\\(\\\\\\(new\\|provide\\)command\\*?{?\\\\\\([a-zA-Z]+\\)}?\\[\\([0-9]+\\)\\]\\)\\|\\(\\\\\\(new\\|provide\\)command\\*?{?\\\\\\([a-zA-Z]+\\)}?\\)\\|\\(\\\\newenvironment\\*?{?\\([a-zA-Z]+\\)}?\\[\\([0-9]+\\)\\]\\[\\)\\|\\(\\\\newenvironment\\*?{?\\([a-zA-Z]+\\)}?\\[\\([0-9]+\\)\\]\\)\\|\\(\\\\newenvironment\\*?{?\\([a-zA-Z]+\\)}?\\)\\|\\(\\\\newtheorem{\\([a-zA-Z]+\\)}\\)\\|\\(\\\\]+?\\)}\\)\\|\\(\\\\DeclareRobustCommand\\*?{?\\\\\\([A-Za-z]+\\)}?\\[\\([0-9]+\]*?\\)\\]\\)\\|\\(\\\\DeclareRobustCommand\\*?{?\\\\\\([A-Za-z]+\\)}?\\[\\([0-9]+\\)\\]\\)\\|\\(\\\\DeclareRobustCommand\\*?{?\\\\\\([A-Za-z]+\\)}?\\)\\|\\(\\\\DeclareMath\\(?:Symbol\\|Delimiter\\|Accent\\|Radical\\){?\\\\\\([A-Za-z]+\\)}?\\)\\|\\(\\\\\\(Declare\\|Provide\\)Text\\(?:Command\\|Symbol\\|Accent\\|Composite\\){?\\\\\\([A-Za-z]+\\)}?\\)\\|\\(\\\\Declare\\(?:Text\\|Old\\)FontCommand{?\\%\\{}]+\\)}\\)\\|\\(\\\\\\(in! dex\\|glossary\\){\\([^}{]*\\({[^}{]*\\({[^}{]*\\({[^}{]*}[^}{]*\\)*}[^}{]*\\)*}[^}{]*\\)*\\)}\\)\\|\\(\\\\document\\(style\\|class]+?\\)}\\)\\|\\(\\\\use\\(package\\)\\(\\[\\([^]\\\\]*\\)\\]\\)?{\\(\\([^#}\\\\\]\\)+?\\)}\\)\\|\\(\\\\def\\\\\\([a-zA-Z]+\\)[^a-zA-Z＠]\\)\\|\\(\\\\let\\\\\\([a-zA-Z]+\\)[^a-zA-Z＠]\\)\\|\\(\\\\font\\\\\\([a-zA-Z]+\\)[^a-zA-Z＠]\\)\\|\\(\\\\chardef\\\\\\([a-zA-Z]+\\)[^a-zA-Z＠]\\)\\|\\(\\\\new\\(count\\|dimen\\|muskip\\|skip\\)\\\\\\([a-z]+\\)[^a-zA-Z＠]\\)\\|\\(\\\\newfont{?\\\\\\([a-zA-Z]+\\)}?\\)\\]+\\)?\\)\\|\\(\\\\mathchardef\\\\\\([a-zA-Z]+\\)[^a-zA-Z＠]\\)" nil t)\\\\\\.\n # bind (regexp end beg regexp-list) TeX-auto-parse-region(TeX-auto-full-regexp-list nil 999999) # (unwind-protect ...) # bind (case-fold-search) TeX-auto-parse() TeX-auto-apply() # bind (force) TeX-update-style() # bind (ignore optional) TeX-arg-document(nil) # bind (arg optional) TeX-parse-argument(nil TeX-arg-document) # bind (TeX-arg-opening-brace TeX-arg-closing-brace last-optional-rejected skip-opt args) TeX-parse-arguments((TeX-arg-document)) # bind (exit-mark position args symbol) TeX-parse-macro("documentclass" (TeX-arg-document)) # bind (symbol) TeX-insert-macro("documentclass") # bind (ignore) LaTeX-env-document("document") apply(LaTeX-env-document "document" nil) # bind (entry environment) LaTeX-environment-menu("document") # bind (entry environment arg) LaTeX-environment(nil) # bind (command-debug-status) call-interactively(LaTeX-environment) # (condition-case ... . error) # (catch top-level ...) --8<---------------cut here---------------end--------------->8--- Here is a backtrace: --8<---------------cut here---------------start------------->8--- Program received signal SIGSEGV, Segmentation fault. 0x08162035 in bytind_to_bufpos_func () (gdb) bt #0 0x08162035 in bytind_to_bufpos_func () #1 0x081a6aff in bi_find_next_emchar_in_string () #2 0x081abfdb in complex_vars_of_search () #3 0x081aa4d5 in Fre_search_backward () #4 0x080ba952 in Ffuncall () #5 0x0808f953 in funcall_compiled_function () #6 0x0808f568 in funcall_compiled_function () #7 0x080ba816 in Ffuncall () #8 0x0808f953 in funcall_compiled_function () #9 0x0808f568 in funcall_compiled_function () #10 0x080ba816 in Ffuncall () #11 0x0808f953 in funcall_compiled_function () #12 0x0808f568 in funcall_compiled_function () #13 0x080ba816 in Ffuncall () #14 0x0808f953 in funcall_compiled_function () #15 0x0808f568 in funcall_compiled_function () #16 0x080ba816 in Ffuncall () #17 0x0808f953 in funcall_compiled_function () #18 0x0808f568 in funcall_compiled_function () #19 0x080ba816 in Ffuncall () #20 0x0808f953 in funcall_compiled_function () #21 0x0808f568 in funcall_compiled_function () #22 0x080ba816 in Ffuncall () ---Type <return> to continue, or q <return> to quit--- #23 0x0808f953 in funcall_compiled_function () #24 0x0808f568 in funcall_compiled_function () #25 0x080ba816 in Ffuncall () #26 0x0808f953 in funcall_compiled_function () #27 0x0808f568 in funcall_compiled_function () #28 0x080ba816 in Ffuncall () #29 0x0808f953 in funcall_compiled_function () #30 0x0808f568 in funcall_compiled_function () #31 0x080ba816 in Ffuncall () #32 0x0808f953 in funcall_compiled_function () #33 0x0808f568 in funcall_compiled_function () #34 0x080ba816 in Ffuncall () #35 0x080bacf8 in Fapply () #36 0x080ba871 in Ffuncall () #37 0x0808f953 in funcall_compiled_function () #38 0x0808f568 in funcall_compiled_function () #39 0x080ba816 in Ffuncall () #40 0x0808f953 in funcall_compiled_function () #41 0x0808f568 in funcall_compiled_function () #42 0x080ba816 in Ffuncall () #43 0x08092fab in Fcall_interactively () #44 0x080b96d0 in Fcommand_execute () #45 0x08101c75 in complex_vars_of_event_stream () ---Type <return> to continue, or q <return> to quit--- #46 0x080fdafd in Fdispatch_event () #47 0x0809ba4d in Fcommand_loop_1 () #48 0x080b8673 in condition_case_1 () #49 0x0809bfde in syms_of_cmdloop () #50 0x080b852a in internal_catch () #51 0x0809b798 in initial_command_loop () #52 0x080b51fb in xemacs_21_4_16_i386_debian_linux () #53 0x080b632d in main () --8<---------------cut here---------------end--------------->8--- This happened with a stock XEmacs 21.4.16 provided by Debian/sid and a CVS AUCTeX which is a few days old. I can reproduce it by starting XEmacs, opening a LaTeX file via `C-x C-f' and adding a document skeleton via `C-c C-e'. -- Ralf

* Uwe Brauer (2005-02-02) writes: More or less. Fontification involves several passes. A syntactic keyword pass, a syntactic pass and a keyword pass. In Emacs the syntactic keyword pass will apply a verbatim face to the contents of the verbatim environment. In the keyword pass happening after that, fontification will only be done if there is no verbatim face present. But in XEmacs the syntactic keyword pass is not working (at least with AUCTeX) and fontification in the verbatim environment cannot be inhibited as the test for the verbatim face fails. The backtrace I posted is not related to font locking. It seems to be a problem with shy groups in the regular expressions. -- Ralf

[Copying to the AUCTeX mailing list as well.] * Stephen J. Turnbull (2005-02-02) writes: As we are about to release AUCTeX 11.55, we are in need of a workaround for 21.4.16. David posted the following on the AUCTeX list (string-match "\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(.\ \\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)" "x") but didn't consider it a real and bullet-proof solution. As a last resort we could prevent the regexps triggering the segfault from being applied in XEmacs. -- Ralf

Ralf> As we are about to release AUCTeX 11.55, we are in need of a Ralf> workaround for 21.4.16. (when (and (featurep 'xemacs) (emacs-version>= 21 4 16) (not (emacs-version>= 21 4 17))) (error 'invalid-state "XEmacs 21.4.16 (and only that release) has a fatal bug in the regexp code that AUCTeX triggers in several places. Upgrade to 21.4.17.")) in some appropriate place in the auctex load sequence is what I would suggest; even if AUCTeX works around it, other applications could trigger it. Ralf> David posted the following on the AUCTeX list Ralf> (string-match "\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(\\(.\ Ralf> \\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)\\)" "x") Doesn't work right, I think. You need the matches of _all_ groups to _fail_, otherwise you are going to get stale data pointing who knows where. (string-match "x" "x") should do what's needed IIRC (since num_nonshy_groups == num_groups, all registers > 0 should get set to nil, no "holes" you can fall into). Ralf> As a last resort we could prevent the regexps triggering the Ralf> segfault from being applied in XEmacs. Any regexp using shy groups can trigger the regexp, unless you can prove that all matches that succeeded before it had fewer capturing groups than the regexp using shy groups. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Ralf Angeli <angeli(a)iwi.uni-sb.de&gt; writes: Actually, shy regexps are by now used rather widely in Lisp packages (for example, preview-latex). XEmacs' version of regexp-opt _also_ produces shy regexps, and it is used all across the XEmacs code base and external packages, even those that don't explicitly use shy regexps themselves. As far as I can see, the main symptom would be random core dumps. I don't think that people will assign the blame to AUCTeX for core dumps. So basically this is major XEmacs problem affecting its operation all across the board, and it does not make sense for AUCTeX to deal in an isolated manner with it. This is a matter of XEmacs release and quality management. The release should be pulled and people prominently given notice. If package authors in general are asked to include code like the above, then we will probably follow suit. But you would be doing a disservice to your users by pretending this matter is even close to being dealt with responsibly if AUCTeX refuses to load. So my take on this matter is that we will _not_ change our code in the above way _unless_ there is an _official_ call of the XEmacs development team to _all_ package developers to do this. Such a call would probably only make sense in connection with a _general_ warning about that XEmacs release (which is supposed to be in the stable series). Unless the XEmacs developers declare this step necessary for _all_ package developers in order to control the spread of this fundamentally buggy release, I don't think we should muddy our own code base, but instead just mention this problem in the release notes, and maybe as package conflicts. It is also a possibility that our installation procedure check for working shy regexps (after all, this is not the first time they have created havoc within XEmacs, as can be witnessed from the preview-latex change log entries from Nix). But the proper place for such checks would really be in regression tests for XEmacs, not in the installation procedures of AUCTeX. People who are using 21.4.16 will be hosed also without using AUCTeX. Shy regexps are an integral part of XEmacs via regexp-opt. It does not make sense to make AUCTeX the poster child for telling this to people. My opinion is to mention this prominently in the release notes as long as it is to be feared that this XEmacs version is around. And how long this is to be feared very much depends on how the XEmacs developers will choose to deal with the problem. If it is their opinion that an occasional core dump in normal operation is no reason to warn people off a release in a prominent manner, then it is _us_ that is going to get blamed when some stock XEmacs/AUCTeX combination gets made available and refuses to work. This XEmacs version must not get put on _any_ software distribution intended to be stable since XEmacs' operation as a whole is affected, and it is beyond the power of the AUCTeX developers to tell distributors "don't use 21.4.16", except possibly by package conflicts. And if a distributor finds AUCTeX conflicting with XEmacs, he will in case of doubt throw out AUCTeX, obviously. We can't reasonably substitute for XEmacs quality management in this instance. Seconded. I did. I consider this of high relevance to the XEmacs community, and I would appreciate it if the auc-tex list were kept informed in case the XEmacs developers decide to take any measures for deprecating 21.4.16 outside of this particular thread. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum

On Thu, Feb 03 2005, Vin Shelton wrote: [...] There have been a couple of reports about Gnus not working correctly (infinite loops) on XEmacs 21.4.16 too: http://thread.gmane.org/gmane.emacs.xemacs.beta/17592 http://thread.gmane.org/gmane.emacs.gnus.general/59452 http://thread.gmane.org/gmane.emacs.xemacs.beta/17534 http://thread.gmane.org/gmane.emacs.xemacs.beta/17437 Hopefully these problems are fixed in the upcoming XEmacs 21.4.17. I agree with David's position. We should not clutter AUCTeX with workarounds for this particular XEmacs release. We should probably release AUCTeX 11.55 (a couple of days?) after XEmacs 21.4.17 is out. Bye, Reiner. -- ,,, (o o) ---ooO-(_)-Ooo--- | PGP key available | http://rsteib.home.pages.de/

Vin Shelton <acs(a)xemacs.org&gt; writes: If I judge the extent of XEmacs' use of shy regexps correctly, this appears like the responsible thing to do, even though I am lacking the data to compare the severity to the "normal" level of regressions to be expected. "most of the reports" seems to imply that the effects were not just from AUCTeX users. In case it was not apparent from the search pattern, I was looking for _either_ regexp-opt _or_ the characteristic \\(?: sequence starting a shy group. Since it is an initialization problem, the longer XEmacs runs, the less likely you are to see any bad effects. The consequences will be most dire for batch runs and "test cases" which immediately start the work without loading other packages. Once the session is running, the probability for triggering the problem becomes low _except_ for asynchronous running stuff like process filters and background font lock, as they get fresh match data structures to work with. Definitely. If I understood this correctly, the problem in AUCTeX occured in the context of font lock patterns, and when preparing a minimal test case. It does not appear to me that AUCTeX would be particularly worse affected than most other packages. I have no clue about the timelines for distributors who package XEmacs, so the only suggestion I could add would be that it might conveivably be a good idea to split the bad news about XEmacs 21.4.16 and the good news about XEmacs 21.4.17 into two separate announcements, so that people will be prepared to wait for 21.4.17. Whether such a reaction would seem exaggerated (when compared with your normal procedures), you are in a much better position to judge than I am. But I can't think of anything more to ask than what you already have offered. Oh, I am quite sure we all have had our own shares of "Oh God, did we release _that_?" experiences. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum

David> As far as I can see, the main symptom would be random core David> dumps. I don't think that people will assign the blame to David> AUCTeX for core dumps. If they only occur (in their experience) while using AUCTeX they will. That's been my experience, anyway. On c.e.x I occasionally have to explain to users that "no, a crash is _never_ the fault of the Lisp program, it's always the fault of XEmacs itself." And you point out that issue yourself later in your post. David> But you would be doing a disservice to your users by David> pretending this matter is even close to being dealt with David> responsibly if AUCTeX refuses to load. Nobody is pretending anything; I'm simply saying what I think AUCTeX should do if it wants to help protect AUCTeX+XEmacs users from a crash that AUCTeX is not responsible for, and protect AUCTeX from "guilt by association". David> So my take on this matter is that we will _not_ change our David> code in the above way _unless_ there is an _official_ call David> of the XEmacs development team to _all_ package developers David> to do this. I think that should normally be a sufficient condition for you to do it, but why are you making it necessary? AUCTeX is simply the highest-risk application (I am not seeing crash reports about 21.4.16, except from you), therefore AUCTeX _might_ want to go further than other applications in warning about that XEmacs version. David> Such a call would probably only make sense in connection David> with a _general_ warning about that XEmacs release (which David> is supposed to be in the stable series). Please be more careful in your phrasing. The series _is_ stable, and has been since well before it was officially promoted from "gamma". It may not be as stable as GNU Emacs release, but based on following emacs-devel for many months I assure you that over the period that you have been publically (eg on comp.emacs.xemacs) recommending use of GNU Emacs CVS with AUCTeX and preview-latex, XEmacs 21.4 has been far more stable than GNU Emacs CVS. We did what we could. It is appropriate for Vin to take responsibility as he did, but Ben (who made the port of his design as difficult as possible without resorting to assembler) and I (who did the port incorrectly) introduced the bug. However, the buggy code did pass over 300 regression tests (specifically for regexps; altogether XEmacs 21.4.16 passed nearly 12,000 tests, the entire suite repeated on a variety of platforms in various configurations before release), and was used for several weeks by several testers, including those who reported the regexp bug triggered by Gnus that made putting the code into production high priority. Ie, the code got into 21.4.16 because several beta testers used it intensively for a couple weeks, it fixed one Gnus problem (though it then introduced another), and no regressions were observed. Obviously, they are not heavy AUCTeX users, since AUCTeX (at least CVS version) is insta-crash with that release. But we stuck the fork in, and it was done. What would you have done, David? Yes, it does make sense to deprecate the release (== announce it contains a fatal bug that manifests in a very important application), but two months after release really it's too late to withdraw it in the sense of preventing distribution: it's not feasible to erase it from CVS. All we can do is encourage people, including distros, to upgrade promptly. David> But the proper place for such checks would really be in David> regression tests for XEmacs, not in the installation David> procedures of AUCTeX. No, it is not. Of course, those regression tests are _already_ in CVS and will be released with 21.4.17, but they can't possibly be in distributed code---we don't distribute code that fails _any_ regression tests. Unfortunately we don't have Guido's time machine. :-( I'm working on separating the regression tests from the release, as well, but this is a hard problem. David> People who are using 21.4.16 will be hosed also without David> using AUCTeX. Shy regexps are an integral part of XEmacs David> via regexp-opt. No, they are not being hosed without using AUCTeX. The release has been out for two months, and Gnus users are somewhat happy because one bug was fixed (although fixing it unmasked an infloop) while no crashes are being reported except from the AUCTeX developers. Basically, the only likely candidates for a crash are those apps that loop over a set of regexps, and in the loop try to access match data that for many of the regexps doesn't exist. (There are also broken apps that don't test for success of a match before accessing the match data, but I doubt AUCTeX does that.) I think font-lock satisfies the "loop over regexps" part of that description, but it doesn't crash or infloop in general. Gnus apparently both loops and accesses non-existent slots in the match-data, but rather than crash it infloops, which is usually much less serious. So AUCTeX is doing something unusual, and is at unusually high risk. David> It does not make sense to make AUCTeX the poster child for David> telling this to people. That is of course your decision, but my mileage varies. David> This XEmacs version must not get put on _any_ software David> distribution intended to be stable since XEmacs' operation David> as a whole is affected, That is neither your decision nor ours, but we are warning the distributors we are in contact with. David> and it is beyond the power of the AUCTeX developers to tell David> distributors "don't use 21.4.16", If "tell" means "command", it's beyond our power, too: XEmacs is free software, distributed "as is", NO WARRANTY, etc, etc. If "tell" means "inform", then it is within your power, too, and I would hope that you have close contact with downstream vendors who provide AUCTeX in .deb or RPM format, and warn them. After all, if the analysis above is correct, it's your users, more than anyone else, who will suffer if you don't. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

David Kastrup <dak(a)gnu.org&gt; writes: Just to set the record straight, what I meant was, that infloops in gnus were the #1 symptom of this bug. If you're only counting crashes, I have no idea what the leading innocent bystander was. - Vin

"Stephen J. Turnbull" <stephen(a)xemacs.org&gt; writes: AUCTeX 11.55's release announcement pretty strongly warned about using XEmacs-21.4.16. However, according to Uwe Brauer, the XEmacs release candidate in CVS (as of yesterday) was still experiencing problems in connection with AUCTeX and shy groups. He did not give any details, and I strongly urged him to do a report to you if he found anything still amiss, as time is definitely running out according to your release schedule. I certainly hope he has done so if his findings were accurate. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum