jwz> I reported this bug to Red Hat on the assumption that this was a bug in jwz> tcsh or readline, but I suppose it's equally likely that it's a bug in jwz> how xemacs is dealing with ptys (since the bug does not occur when jwz> running tcsh in gnome-terminal.) I also assume they will completely jwz> ignore my report without even looking at it, since emacs is involved. jwz> So I'll report it to you guys as well: jwz> http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=82610 jwz> Since upgrading from RH 7.2 to RH 8.0, I can no longer compose long jwz> command lines in tcsh when running in an XEmacs *shell* buffer. I don't jwz> know whether this is a bug in tcsh, or in readline, or what, but it does jwz> not happen with bash. jwz> It's not merely truncating the command line -- it's also apparently jwz> leaving the un-read characters on the input buffer, and then jwz> interpreting them as commands afterward! This is potentially jwz> disasterous, and could easily lead to loss of files, if there were jwz> redirections or something on the command line. Historically, all Unix systems had a limit on the number of characters you can input on the command line - generally 255. This is one of the most stupid system limits imaginable. /usr/include/sys/param.h:33:#define CANBSIZ MAX_CANON /usr/include/bits/posix1_lim.h:50:#define _POSIX_MAX_CANON 255 /usr/include/bits/confname.h:29: _PC_MAX_CANON, /usr/include/bits/confname.h:30:#define _PC_MAX_CANON _PC_MAX_CANON /usr/include/linux/limits.h:11:#define MAX_CANON 255 /* size of the canonical input queue */ You can still see this today on a Sun machine. Start /bin/sh in an xterm, and type some long command line. I get something like this: $ echo xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: not found It depends on your shell, and your system. On my Linux system, the limit seems to have been raised to about 40k. To reproduce this in an xterm, you have to get your shell into canonical mode, e.g. `bash --noediting' or `unset edit' in tcsh. Try building up a huge `echo' command - you probably won't be able to input more than 40k. My guess is that the Linux folks have raised the MAX_CANON limit, but without changing the system header files. Perhaps because Linus and Uli don't really talk much. xemacs tries to circumvent the system limit. If the command line is longer than 200 or so, it sends the command line in chunks of 200, with a '^D' character in between (!). This insanity has mostly worked for the past decade. "Usually" the '^D's are discarded. csh has always been a little special, because '^D' also does file completion. I remember that ten years ago, unset filec seemed to fix things when some user had a problem just like yours. Things you could play with, to see whether it perturbs the problem space: set/unset filec stty eof '^D' stty eof '^P' (some other random char) set/unset autolist set/unset edit set term=dumb/vt100 Look at the tcsh binding for '^D' Perhaps the system header files now return a value for MAX_CANON that is too large?? On my system, fpathconf (0, _PC_MAX_CANON) still returns 255. If you just want to ameliorate the problem for yourself, you can hack XEmacs' get_pty_max_bytes() : int get_pty_max_bytes (int fd) { return 40000; // or maybe 9999999999, or maybe 200 } Let me know if any of these ideas work for you. ---------------------------------------------------------------- In canonical input mode, the operating system provides input editing facilities: some characters are interpreted specially to perform editing operations within the current line of text, such as ERASE and KILL. *Note Editing Characters::. The constants `_POSIX_MAX_CANON' and `MAX_CANON' parameterize the maximum number of bytes which may appear in a single line of canonical input. *Note Limits for Files::. You are guaranteed a maximum line length of at least `MAX_CANON' bytes, but the maximum might be larger, and might even dynamically change size. - Macro: int VEOF This is the subscript for the EOF character in the special control character array. `TERMIOS.c_cc[VEOF]' holds the character itself. The EOF character is recognized only in canonical input mode. It acts as a line terminator in the same way as a newline character, but if the EOF character is typed at the beginning of a line it causes `read' to return a byte count of zero, indicating end-of-file. The EOF character itself is discarded. Usually, the EOF character is `C-d'. ---------------------------------------------------------------- How can XEmacs work better? Here's an idea: When the input line is bigger than 200, actually inspect the current settings of the tty to see whether the shell has put it into canonical mode or not. If not canonical, don't do the weird C-d glue thing.

jwz> Martin Buchholz wrote: jwz> http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=82610 jwz> Since upgrading from RH 7.2 to RH 8.0, I can no longer compose long jwz> command lines in tcsh when running in an XEmacs *shell* buffer. I don't jwz> know whether this is a bug in tcsh, or in readline, or what, but it does jwz> not happen with bash. jwz> Aha! Turns out it's xemacs's fault. >> xemacs tries to circumvent the system limit. If the command line is >> longer than 200 or so, it sends the command line in chunks of 200, >> with a '^D' character in between (!). This insanity has mostly worked >> for the past decade. "Usually" the '^D's are discarded. >> csh has always been a little special, because '^D' also does file >> completion. I remember that ten years ago, >> unset filec jwz> Bingo. "unset filec" fixes it. jwz> So if xemacs were either: jwz> - just sending all the bytes at once; or jwz> - sending them in chunks, but without the extra ^D If XEmacs sends more than MAX_CANON (usually 256) bytes all at once to the tty, it typically gets wedged in a most horrible way. Unfortunately, sending them in chunks without ^D has the same effect. EOF (Control-d or ASCII EOT) may be used to generate an end-of-file from a terminal. When received, all the characters waiting to be read are immediately passed to the program, without waiting for a newline, and the EOF is discarded. Thus, if no characters are waiting (that is, the EOF occurred at the beginning of a line) zero characters are passed back, which is the standard end-of-file indication. Unless escaped, the EOF char- acter is not echoed. Because EOT is the default EOF character, this prevents terminals that respond to EOT from hanging up. You can see the effect even today by doing: xterm bash -noediting (or Solaris /bin/sh) hold down the 'x' key. (or paste some text without newlines repeatedly) Eventually (1024 chars on Linux; 256 on Solaris) the xterm will refuse to allow any more input and will just beep at you. jwz> then it would work fine without "unset filec". The above description is only correct if icanon mode is turned on in the tty. That used to be generally true, but these days shells like to process their input characters themselves by default. The correct way to fix this bug is for XEmacs to check whether the tty is in icanon mode and the EOF character is defined. Then, and ONLY then, should the ^D trick be used to avoid MAX_CANON. It is probably also a bug that ICANON is set by XEmacs when it initializes a pty. If a user explicitly inserts a literal ^W into the shell input buffer, it is more likely that they want this passed to the shell than that they want to have the previous word be discarded by the tty. Emacs is a better editor than the tty -- the user already has backward-kill-word. I feel vaguely responsible for this, so I will make a patch. Martin

jwz> It's not merely truncating the command line -- it's also apparently jwz> leaving the un-read characters on the input buffer, and then jwz> interpreting them as commands afterward! More Bogons From the Unixware docs: ---------------------------------------------------------------- In UNIX System V, if the data in the line-discipline buffer exceeds MAX_CANON in canonical mode and IMAXBEL is clear, all the bytes of data saved in the buffer up to that point are discarded without any notice, but if IMAXBEL is set and the data in the line-discipline buffer exceeds MAX_INPUT, the ASCII BEL character is echoed. Further input is not stored, and any data already present in the input-queue remains undisturbed. ... If MAX_CANON is defined for this terminal-device, it is a limit on the number of bytes in a line. The behavior of the system when this limit is exceeded is implementation-dependent. If MAX_CANON is not defined for this terminal-device, there is no such limit. It should be noted that there is a possible inherent deadlock if the program and the implementation conflict on the value of MAX_CANON. With both ICANON and IXOFF set when more than MAX_CANON characters transmitted without a line-feed, transmission is stopped, the line-feed (or carriage-return if ICRLF is set) never arrives, and the read is never satisfied. A program should never set IXOFF if it is using canonical-mode unless it knows that (even in the face of a transmission error) the conditions described previously cannot be met or unless it is prepared to deal with the possible deadlock in some other way, such as time-outs. NOTE: This would only occur if the transmitting side was a communications device (for example, an asynchronous port). This normally will not happen since the transmitting side is a user at a terminal. It should also be noted that this can be made to happen in non-canonical-mode if the number of characters received that would cause IXOFF to be sent is less than VMIN when VTIME equals zero. ... A terminal associated with one of these files ordinarily operates in full-duplex mode. Characters may be typed at any time, even while output is occurring, and are only lost when the character input buffers of the system become completely full, which is rare (for example, if the number of characters in the line discipline buffer exceeds MAX_CANON and IMAXBEL (see below) is not set), or when the user has accumulated MAX_INPUT number of input characters that have not yet been read by some program. When the input limit is reached, all the characters saved in the buffer up to that point are thrown away without notice. ... (why it sometimes works to split the command line into chunks delimited by Ctrl-d) EOF (<Ctrl>D or ASCII EOT) may be used to generate an end-of-file from a terminal. When received, all the characters waiting to be read are immediately passed to the program, without waiting for a newline, and the EOF is discarded. Thus, if no characters are waiting (that is, the EOF occurred at the beginning of a line) zero characters are passed back, which is the standard end-of-file indication. The EOF character is not echoed unless it is escaped or ECHOCTL is set. Because EOT is the default EOF character, this prevents terminals that respond to EOT from hanging up. ---------------------------------------------------------------- Lessons? In addition to the stuff in my previous message, we might want to fiddle with stty imaxbel/stty -imaxbel stty ixoff/stty -ixoff Another thing to note: shells often save/restore tty settings around every command executed, so typing `stty -a' into a shell might not give you the same tty settings that are active while the shell is reading input (rather than executing a command). To get the true tty settings used by a shell while reading input, you must type something like stty < /dev/pts/23 into _another_ shell.