9:15 a.m.
Gnus v5.8.8
XEmacs 21.4 (patch 4) "Artificial Intelligence" [Lucid] (i686-pc-linux, Mule) of
Wed Aug 29 2001 on two-places-at-once.mit.edu
200 senator-bedfellow.mit.edu NNRP Service Ready - usenet(a)mit.edu (posting ok).
Any IMAP command containing a literal that contains newlines will fail
when using nnimap with an nnimap-stream of "gssapi". (APPEND is a good
example of a command that will fail.) This is because the literal
length is miscomputed.
First, the fix; make imap-gssapi-open match imap-kerberos4-open in this way:
--- imap.el 2002/04/03 07:42:18 1.1
+++ imap.el 2002/04/03 07:45:13
＠＠ -520,7 +520,8 ＠＠
response)
(when process
(with-current-buffer buffer
- (setq imap-client-eol "\n")
+ (setq imap-client-eol "\n"
+ imap-calculate-literal-size-first t)
(while (and (memq (process-status process) '(open run))
(goto-char (point-min))
;; cyrus 1.6.x (13? < x <= 22) queries capabilities
Now, the background. "imtest" is a horrible, evil, unholy program that
hates me. It calls up my mother at 3am and swears at her. It
murdered my cat. It's *that* bad.
IMAP specifies that newlines are canonicalized to \r\n. imtest requires
that its input lines be terminated with \n, because it uses fgets().
(I suppose this is actually platform specific and Gnus and imtest would
lose in more interesting ways on non-Unix platforms, if they ran there.)
imtest then does a \n to \r\n translation on what it's read.
This means that any program that uses imtest and sends literals needs
to send any newlines as \n but count them as 2 characters when computing
the literal length.
Gnus likes to construct proper IMAP messages (with \r\n) but is smart
enough to know that imtest wants \n, and translates the buffer before
sending it to imtest. (That's the (setq imap-client-eol "\n") bit.)
However, unless you tell it (setq imap-calculate-literal-size-first
t), it gets the length wrong because it calculates the length after
translating, forgetting that imtest will translate back.
This obviously bit someone long ago on kerberos4 auth, because that
code does the right thing. But that right thing never got copied to
the GSSAPI code.
Oh, by the way, remember how I said that imtest uses fgets()? It uses
a 2k buffer there. Which means that if you ever try to send a line
longer than 2K, imtest will insert spurious newlines in it, and you're
just fucked.
Isn't this fun?
--Jered
12:28 p.m.
On 3 Apr 2002, Jered Floyd wrote:
First, the fix; make imap-gssapi-open match imap-kerberos4-open in
this way:
...
- (setq imap-client-eol "\n")
+ (setq imap-client-eol "\n"
+ imap-calculate-literal-size-first t)
This has been fixed in Oort Gnus, I recomend that the patch be applied to
XEmacs as well.
Oh, by the way, remember how I said that imtest uses fgets()? It
uses
a 2k buffer there. Which means that if you ever try to send a line
longer than 2K, imtest will insert spurious newlines in it, and you're
just fucked.
It depends on your OS, on IRIX the buffer is 256 characters which makes
things rather unusable.
However, recent Cyrus IMAPD imtest uses some other approach, so it works
better. Please try it. (I don't remember whether the approach was to not
use fgets() or set the terminal into returning character-by-characters
instead of line-by-line.)
Isn't this fun?
RMS seems to not like integrating authentication/encryption into core
Emacs, so the situation will not improve. It will likely get worse as new
incompatible versions of the external programs are released.
4:24 p.m.
Simon Josefsson <jas(a)extundo.com> writes:
However, recent Cyrus IMAPD imtest uses some other approach, so it
works
better. Please try it. (I don't remember whether the approach was to not
use fgets() or set the terminal into returning character-by-characters
instead of line-by-line.)
The imtest included in cyrus-imapd 2.1.3 which was released 7 Mar 2002
doesn't seem to do this:
if (fgets(buf, sizeof (buf) - 1, stdin) == NULL) {
printf(LOGOUT);
prot_write(pout, LOGOUT, sizeof (LOGOUT));
FD_CLR(0, &read_set);
} else {
count = strlen(buf);
buf[count - 1] = '\r';
buf[count] = '\n';
buf[count + 1] = '\0';
prot_write(pout, buf, count + 1);
}
However this seems like an easy fix... if what fgets reads in doesn't
end in a \n, don't make the last chars \r\n. I'll send a patch to the
cyrus people later.
--Jered
4:35 p.m.
On 3 Apr 2002, Jered Floyd wrote:
Simon Josefsson <jas(a)extundo.com> writes:
> However, recent Cyrus IMAPD imtest uses some other approach, so it works
> better. Please try it. (I don't remember whether the approach was to not
> use fgets() or set the terminal into returning character-by-characters
> instead of line-by-line.)
The imtest included in cyrus-imapd 2.1.3 which was released 7 Mar 2002
doesn't seem to do this:
if (fgets(buf, sizeof (buf) - 1, stdin) == NULL) {
printf(LOGOUT);
prot_write(pout, LOGOUT, sizeof (LOGOUT));
FD_CLR(0, &read_set);
} else {
count = strlen(buf);
buf[count - 1] = '\r';
buf[count] = '\n';
buf[count + 1] = '\0';
prot_write(pout, buf, count + 1);
}
The solution used the other approach then, not buffering stdin/stoud. Did
you try it?
Index: imtest.c
===================================================================
RCS file: /cvs/src/cyrus/imtest/imtest.c,v
retrieving revision 1.60
retrieving revision 1.61
diff -u -r1.60 -r1.61
--- imtest.c 2001/02/19 19:32:06 1.60
+++ imtest.c 2001/02/28 02:44:32 1.61
＠＠ -1185,6 +1185,11 ＠＠
int run_stress_test=0;
int dotls=0;
int server_supports_tls;
+
+ /* do not buffer */
+ setbuf(stdin, NULL);
+ setbuf(stdout, NULL);
+ setbuf(stderr, NULL);
/* look at all the extra args */
while ((c = getopt(argc, argv, "zvk:l:p:u:a:m:f:r:t:")) != EOF)
However this seems like an easy fix... if what fgets reads in
doesn't
end in a \n, don't make the last chars \r\n. I'll send a patch to the
cyrus people later.
Their current approach isn't too unreasonable, it converts from the
IMAP/TCP eol character to the tty eol character. However, it would be the
wrong thing under Windows, but I doubt it build under Windows anyway.
5:10 p.m.
The solution used the other approach then, not buffering stdin/stoud.
Did
you try it?
[...]
Their current approach isn't too unreasonable, it converts from
the
IMAP/TCP eol character to the tty eol character. However, it would be the
wrong thing under Windows, but I doubt it build under Windows anyway.
Sorry; I'm failing to understand this. I don't see how making stdin
unbuffered changes the behaviour of fgets() at all. fgets() can still
return because it's filled its buffer, at which point the code I quoted
would gleefully replace the last character with \r\n. Am I missing
something?
--Jered
6:47 p.m.
Jered Floyd <jered(a)mit.edu> writes:
> The solution used the other approach then, not buffering
stdin/stoud. Did
> you try it?
[...]
> Their current approach isn't too unreasonable, it converts from the
> IMAP/TCP eol character to the tty eol character. However, it would be the
> wrong thing under Windows, but I doubt it build under Windows anyway.
Sorry; I'm failing to understand this. I don't see how making stdin
unbuffered changes the behaviour of fgets() at all. fgets() can still
return because it's filled its buffer, at which point the code I quoted
would gleefully replace the last character with \r\n. Am I missing
something?
No, I didn't read carefully and thought you referred to the bug where
the OS buffers are filled and long lines are truncated. I guess it
was fixed long time ago though. Either raising the buffer size so
that long lines aren't corrupted, or doing what you suggest seems like
a very good idea.
4:53 a.m.
No, I didn't read carefully and thought you referred to the bug
where
the OS buffers are filled and long lines are truncated. I guess it
was fixed long time ago though. Either raising the buffer size so
that long lines aren't corrupted, or doing what you suggest seems like
a very good idea.
In case this is helpful to others before the next Cyrus release, I'm
cc'ing you guys for the record. The following patch was submitted to
and accepted by the Cyrus IMAPD maintainers.
--Jered
--- cyrus-imapd-2.1.3/imtest/imtest.c 2002/04/03 20:06:25 1.1
+++ cyrus-imapd-2.1.3/imtest/imtest.c 2002/04/03 20:22:18
＠＠ -1139,10 +1139,14 ＠＠
FD_CLR(0, &read_set);
} else {
count = strlen(buf);
- buf[count - 1] = '\r';
- buf[count] = '\n';
- buf[count + 1] = '\0';
- prot_write(pout, buf, count + 1);
+ /* If we read a full line, translate the newline */
+ if (buf[count - 1] == '\n') {
+ buf[count - 1] = '\r';
+ buf[count] = '\n';
+ buf[count + 1] = '\0';
+ count++;
+ }
+ prot_write(pout, buf, count);
}
prot_flush(pout);
} else if (FD_ISSET(sock, &rset)) {
2:29 p.m.
|--==> "SJ" == Simon Josefsson <jas(a)extundo.com> writes:
SJ> On 3 Apr 2002, Jered Floyd wrote:
>First, the fix; make imap-gssapi-open match imap-kerberos4-open in
this way:
SJ> ...
>- (setq imap-client-eol "\n")
>+ (setq imap-client-eol "\n"
>+ imap-calculate-literal-size-first t)
SJ> This has been fixed in Oort Gnus, I recomend that the patch be applied to
SJ> XEmacs as well.
Send the patch with ChangeLog to xemacs-patches(a)xemacs.org and I'll
apply it.
--
|---<Steve Youngs>---------------<GnuPG KeyID: 10D5C9C5>---|
| XEmacs - It's not just an editor. |
| It's a way of life. |
|------------------------------------<youngs(a)xemacs.org>---|
8067
days inactive
8069
days old
7 comments
3 participants
participants (3)
-
Jered Floyd -
Simon Josefsson -
Steve Youngs