1:36 a.m.
Stephen J. Turnbull wrote:
Aidan Kehoe writes:
> Reliably differentiating those [bit-for-bit identical] pairs
> without metadata is not possible.
And unnecessary, precisely because it's not possible. It's only when
you add disambiguating text that it matters, and then the Sufficiently
Smart Program argument applies.
I hate to disturb your conversation -- but I want to add the voice
of one European (German) user: Aidan expresses the problem, and you
seem to reject work on it to get a Really Good Design. But it ain't
so easy, from this user's perspective -- sometimes Worse Is Better,
as we all have learned from Dick Gabriel.
It is highly annoying that I need to explicitly, manually, specify
the buffer-coding-system for a file when I want to insert an EUR
sign and when I want it to end up encoded in ISO-8859-15. And
that's the current state of affairs, AFAICS.
Since many of us use templates for new files anyhow, it's a small
matter to add an appropriate cookie to demand a specific encoding
for a file. That this approach is effectively rejected by you, is
really disturbing for us European users. Maybe one doesn't need it
for Japanese scripts and for UTF-8, but for us poor souls who still
want to use ISO-8859-* for other reasons, it would be a real advantage.
Well, speaking of UTF-8: Since XEmacs is very happy to destroy lots
of my files with its supposedly smart encoding detection -- and
does so WITHOUT any warning -- your demand for more thought about
error recovery is right on spot. But there seems to be other areas
that are in more severe need of that error handling than recovery
from wrong coding cookies (namely, automatic encoding sniffing). I
have had literally dozens of UTF-8 files with one single Latin1
char in them, that got reencoded by XEmacs when I opened and saved
them. (I.e., when I didn't pay enough attention to the modeline in
the process of quickly modifying one or two words.) In all these
cases, reliance on a user-supplied coding cookie would have saved
me untold hours and hours of work to redo the result of XEmacs
automatic encoding detection which Really Really Really Sucks.
Throwing an error if the coding system is not sufficient is much
preferred to the current state of affairs (arbitrarily choosing a
coding system that XEmacs thinks is right).
Sorry about sounding frustrated -- but I am, and your insistence on
Sufficiently Smart Programs(tm) being the Right Way(tm) does not
match my experience with XEmacs, or with programs at all. Please
let us have our way to specify "yes, I want the buffer of this file
in THIS coding system and be DAMNED if you try to switch on another
one beneath my back. I know what I'm doing because I'M THE USER and
I'M ALWAYS RIGHT BY DEFINITION because I know more about MY OWN
FILES than you bloody stupid detection algorithm."
Please.
Joachim
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Joachim Schrod Email: jschrod(a)acm.org
Roedermark, Germany
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
9:04 a.m.
New subject: Coding system sniffing.
Joachim Schrod <jschrod(a)acm.org> writes:
Well, speaking of UTF-8: Since XEmacs is very happy to destroy lots
of my files with its supposedly smart encoding detection -- and does
so WITHOUT any warning -- your demand for more thought about error
recovery is right on spot. But there seems to be other areas that
are in more severe need of that error handling than recovery from
wrong coding cookies (namely, automatic encoding sniffing). I have
had literally dozens of UTF-8 files with one single Latin1 char in
them, that got reencoded by XEmacs when I opened and saved
them. (I.e., when I didn't pay enough attention to the modeline in
the process of quickly modifying one or two words.) In all these
cases, reliance on a user-supplied coding cookie would have saved me
untold hours and hours of work to redo the result of XEmacs
automatic encoding detection which Really Really Really Sucks.
In this case, I think a solution from the coding system implementation
angle should obliterate the problem more reliably than approaching it
from the detection angle. All of the latin-X codings as well as utf-8
have the property that _valid_ characters have exactly one valid
external representation. If Emacs' internal buffer encoding offers a
possibility for encoding "invalid byte with code xx", then loading and
saving a file without changing the encoding will preserve the
contents. Indeed, an "invalid byte with code xx" kind of character
can't be saved in latin-1 since latin-1 does not usually have an
invalid byte with code xx.
So there are a lot of possibilities for detecting uncurable
inconsistencies and maintaining file coherency even across wrong
detections. I don't think that the potential in that area is used to
its full extent. Automatic detection can't _always_ work right
(random bit patterns _always_ have the potential to look like a coding
cookie). But one can make pretty sure that most used encodings will
preserve all the information present in the source file.
Throwing an error if the coding system is not sufficient is much
preferred to the current state of affairs (arbitrarily choosing a
coding system that XEmacs thinks is right).
As long as it can uniquely encode the buffer in the choice taken, and
the buffer uniquely represents the input encoding, at least the damage
is reversible.
Personally, I find that Emacs 22 does a remarkable job at not
corrupting files, even though there are (quite rare for me, but as a
Western European one is somewhat favored) times when the initial
detection goes wrong.
I suppose that if I were to have Escape-code based encodings in my
normal work set of encodings (when there are several different legal
inputs possible for the same sequence of characters) I might get hit
more often by problems, but as things stand, I can't actually
complain.
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
1:19 p.m.
New subject: Coding system sniffing.
Joachim Schrod writes:
you seem to reject work
Hey, please read what I wrote! What I oppose is a half-patch, when
it's known how to deal with the problems of the half-patch.
What I've *asked* for is some design effort to deal with the *known
problem* of cookie-oblivious clients translating to or adding
characters from the wrong charset, and not resetting the cookie
properly. There's also a design issue that XEmacs is not GNU Emacs,
and there are almost surely gotchas in XEmacs that GNU Emacs fixed
already (eg, the tilde problem).
sometimes Worse Is Better, as we all have learned from Dick Gabriel.
Preventing known bugs is Better, period.
Why do you guys resist that so much?
It is highly annoying that I need to explicitly, manually, specify
the buffer-coding-system for a file when I want to insert an EUR
sign and when I want it to end up encoded in ISO-8859-15. And
that's the current state of affairs, AFAICS.
And cookies *should* be 100% unnecessary for that use case, right here
and right now, in XEmacs 21.5 (and possibly XEmacs 21.4 + latin-unity,
too).
In XEmacs 21.5, (prefer-coding-system 'iso-8859-15) and be happy,
AFAIK. Alternatively, use latin-unity; that should detect that the
Euro Sign is present, and prefer Latin-9 to Latin-1. If either of
those doesn't work, file a bug report, please.
For template and HTML files, there's file-coding-system-alist which
allows you to specify encodings according to the file name matching a
regular expression. Again, if you try it, and it doesn't work, it's a
bug, and I'd greatly appreciate a report.
21.4 I don't know about.
Please.
Patches welcome. Right now we're discussing what I'd like to see in a
very hypothetical patch.
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
2:48 p.m.
New subject: Coding system sniffing.
"Stephen J. Turnbull" <stephen(a)xemacs.org> writes:
21.4 I don't know about.
I think it is quite worse than in Emacs 21.1, and the behavior in
Emacs 21.1 is not really satisfactory to most people having to deal
with more than a single encoding.
XEmacs 21.4 is really not something one wants to rely on for work
involved with multiple encodings. I have no personal experience with
21.5 but would think it quite surprising if it was even close to being
as bad as 21.4 (which really has MULE more or less as an afterthought,
if at all).
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-beta
6399
days inactive
6399
days old
3 comments
3 participants
participants (3)
-
David Kastrup -
Joachim Schrod -
Stephen J. Turnbull