Sundry C functions, and the ASCII-compatible assumptions made.

[Ben Wing]

---
I hadn't realised the assumptions of null-termination and ASCII
compatibility for strings were so pervasive. That does make sticking to a
single Ibyte format in both buffers and strings, where that format may
contain embedded zero bytes and ASCII octets that don't represent ASCII
characters, a crazy amount of work.
---

First, the issue of null-termination is not a problem.  It is indeed assumed
in various places that strings are null-terminated, but this is only to find
the length of the strings.  Unicode strings will be null-terminated as well,
just with a double-null.  There is no assumption that "null-termination"
means a single null, other than the places that assume ASCII compatibility.

And anyway, those places are not as pervasive as you may think.  In
particular, I've been very careful to go through and use qxestr*[]
everywhere instead of just str*[] -- the fact that IByte * and char * are
different induces compile warnings, and ensures that these get fixed.  If
you fix up the qxestr*[] functions appropriately, most of these places go
away.  The remaining places can mostly be found by searching for casts to
CIbyte * or char * or Extbyte *, and searching for char constants.  Imo if
this issue is important, it can be fixed.

---
The XEmacs internals and architecture documentation is actually, at the
moment, much better than most internals documentation I've ever had exposure
to. If the buffer_text "beg" pointer hadn't been of type "Ibyte *", or if
I'd read in more detail the various encapsulated functions, I wouldn't have
posted that "Sundry C functions" mail. The documentation isn't really an
issue; I just fucked up. 
---

Thanks:]  (I wrote most of this stuff)


---
I've already been posting pretty detailed plans, and Stephen's been
critiquing them in detail. 
---

What I mean is, you have indeed posted lots of detailed plans of *ideas* but
AFAIK not of the actual process -- what you will change in what order, where
the "checkpoint" steps will be where you test, get it working and sync up,
etc.

-----Original Message-----
From: Aidan Kehoe [mailto:kehoea at parhasard.net] 
Sent: Wednesday, October 06, 2004 6:10 AM
To: Ben Wing
Cc: xemacs-beta at xemacs.org
Subject: Re: Sundry C functions, and the ASCII-compatible assumptions made.



 Ar an cúigiú lá de mí Deireadh Fómhair, scríobh Ben Wing: 

 > Tread softly, Aidan.  There is much you do not know.

Yeah. But hey, I haven't broken anyone's build yet. And the more criticism I
get, the more opportunities to do it are eliminated. Thank you for the
feedback:-) .

 > First, I did go through and Mule-ize most of the code.  That means, it
has  > been checked for problems with internal/external data conversion.
There are  > in fact not very many problems remaining; what looks like a
problem to you  > is often simply a case of you not sufficiently
understanding the code. 

I hadn't realised the assumptions of null-termination and ASCII
compatibility for strings were so pervasive. That does make sticking to a
single Ibyte format in both buffers and strings, where that format may
contain embedded zero bytes and ASCII octets that don't represent ASCII
characters, a crazy amount of work.

 > [... snipping underlining of just how little I'd realised that. ] What  >
I'd suggest is, once you figure out what's really going on, improve the  >
documentation concerning it -- e.g. if you didn't know about the  >
internals manual, put in mentions to it in prominent places where someone  >
like you who's trying to understand the code will encounter it.  Stuff  >
that you've learned in the process of spending the last month or so  >
poring over the code, write it up if it isn't already in the internals  >
manual.  Places that you found particularly confusing, put prominent  >
descriptions of what's going on with appropriate references elsewhere.  >
Cross-reference all over the place so that it's more likely the next  >
person coming along will have an easier time.  > 
 > I'm very serious about this, this would be a great improvement to XEmacs.

The XEmacs internals and architecture documentation is actually, at the
moment, much better than most internals documentation I've ever had exposure
to. If the buffer_text "beg" pointer hadn't been of type "Ibyte *", or if
I'd read in more detail the various encapsulated functions, I wouldn't have
posted that "Sundry C functions" mail. The documentation isn't really an
issue; I just fucked up. 

 > That way we (Stephen and I) can critique you and help you avoid those  >
"gotcha!" problems and lots of fruitless work.

I've already been posting pretty detailed plans, and Stephen's been
critiquing them in detail. 

-- 
Like the early Christians, Marx expected the millennium very soon; like
their successors, his have been disappointed--once more, the world has shown
itself recalcitrant to a tidy formula embodying the hopes of some section of
mankind. (Russell)