Re: Infinite recursion of function-inlining in byte-compiler

I've tracked this bug down the the following change between revisions 1.9 and 1.10 of byte-optimize.el: *************** *** 351,356 **** --- 357,363 ---- (byte-compile-warn "attempt to open-code %s with too many arguments" name)) form) + (setq body (mapcar 'byte-optimize-form body)) (let ((newform (if bindings (cons 'let (cons (nreverse bindings) body)) This additional call into the byte-optimizer (added by Ben in a partial sync to FSF 20.7 for XEmacs 21.4.13) results in an infinite recursion when trying to compile some recursive inlined procedures. Here's a partial lisp backtrace of one such failed optimization: mapcar(byte-optimize-form ((foo))) byte-compile-unfold-lambda(((lambda nil (foo)))) byte-optimize-form-code-walker(((lambda nil (foo))) nil) byte-optimize-form(((lambda nil (foo))) nil) byte-optimize-form((foo)) mapcar(byte-optimize-form ((foo))) byte-compile-unfold-lambda(((lambda nil (foo)))) byte-optimize-form-code-walker(((lambda nil (foo))) nil) byte-optimize-form(((lambda nil (foo))) nil) byte-optimize-form((foo)) mapcar(byte-optimize-form ((foo))) byte-compile-unfold-lambda(((lambda nil (foo)))) byte-optimize-form-code-walker(((lambda nil (foo))) nil) byte-optimize-form(((lambda nil (foo))) nil) byte-optimize-form((foo) nil) byte-optimize-form-code-walker((progn (foo)) nil) byte-optimize-form((progn (foo)) nil) byte-compile-top-level((progn (foo)) nil lambda) byte-compile-lambda((lambda (arg) (foo))) byte-compile-file-form-defmumble((defun bar (arg) (foo)) nil) byte-compile-file-form-defun((defun bar (arg) (foo))) byte-compile-file-form((defun bar (arg) (foo))) #<compiled-function nil "...(43)" [byte-compile-unresolved-functions byte-compile-inbuffer 1 " \n" nil looking-at ";" byte-compile-file-form read byte-compile-flush-pending byte-compile-warn-about-unresolved-functions] 3>() call-with-condition-handler(#<compiled-function (error-info) "...(4)" [error-info byte-compile-report-error] 2> #<compiled-function nil "...(43)" [byte-compile-unresolved-functions byte-compile-inbuffer 1 " \n" nil looking-at ";" byte-compile-file-form read byte-compile-flush-pending byte-compile-warn-about-unresolved-functions] 3>) byte-compile-from-buffer(#<buffer "compile-me.el"> "/Users/toomim/harmonia/src/langs/c/compile-me.el" t) byte-compile-buffer("compile-me.el") call-interactively(byte-compile-buffer) command-execute(byte-compile-buffer t) execute-extended-command(nil) call-interactively(execute-extended-command) In this case, the optimizer needs to stop trying to inline the function foo() into itself. I can think of three ways to do this: 1) Remove the new (setq body (mapcar 'byte-optimize-form body)) statement. This would make the optimizer more conservative (as it was in 21.4.12) as a tradeoff for being able to compile recursive defsubst's. 2) Change all defsubst's into defun's in programs that don't compile with the new xemacs. 3) Make the optimizer catch and prevent recursive inlining: when optimizing forms created by byte-compile-inline-expand(foo), don't expand any inline called "foo". I think #3 is the best option, but I'm not sure where the new code should go. It seems like it might be tricky to catch all the places in the optimizer in which you'd want to remember/forget that you're inlining a function called "foo". Any ideas? If not, what do you think about #1? Thanks, Michael Michael Toomim wrote: