>From Unicode's FAQ: (http://www.unicode.org/unicode/faq) """ Q: Will UTF-16 ever be extended to more than a million characters? A: As stated, the goal of Unicode is not to encode glyphs, but characters. Over a million possible codes is far more than enough for this goal. Unicode is *not* designed to encode arbitrary data. If you wanted, for example, to give each "instance of a character on paper throughout history" its own code, you might need trillions or quadrillions of such codes; noble as this effort might be, you would not use Unicode for such an encoding. No proposed extensions of UTF-16 to more than 2 surrogates has a chance of being accepted into the Unicode Standard or ISO/IEC 10646. """ A good example of Unicode encoding characters but not glyphs are the CJK (Chinese, and Japanese, and Korean) Unicode code points. If i recall, for a given unicode character in these ranges its non uncommon for Chinese, Japanese, and Korean to have different glyphs associated with these code points. I don't think any sane person would expect to have a non UTF-16 encodeable character accepted into Unicode for anytime in the upcoming future, unless we suddenly discover several alien races (that also use ideographs) and need to record their documents in Unicode document stores. In other words, the fact that UTF-16 doesn't encode all 4million code points isn't that big of a deal. I will note that other people have told me that glibc defines wchar_t as a UCS-4 type. Bill > From: Ben Wing [mailto:ben＠666.com] > > > surrogates can only encode 1,000,000 chars. ucs-4 encodes > 4,000,000,000 chars. > is there another extension mechanism to handle the rest? > > Bill Tutt wrote: > > > > From: Ben Wing [mailto:ben＠666.com] > > > "Stephen J. Turnbull" wrote: > > > > > > > (3) We may want to be a little bit careful with the > notion of the > > > > default internal representation. I can see that a default > > > > internal representation of UCS-2 (UTF-16, I presume > is what you > > > > really mean?) would be attractive. So what happens > if you have > > > > data that is not representable in the default internal > > > > representation? Do we just tell those users to get lost? > > > > > > > > It would be kind of weird if the default internal > representation > > > > that Eistrings dealt with was UCS-2 but UTF-8 > representation was > > > > available in buffers, which you don't rule out. > > > > > > by its nature, the default int. rep. must be able to > > > represent all chars. that > > > would rule out utf16 if we have more than 1,000,000 and some > > > chars. but it > > > doesn't rule out ucs4, or some utf16 extension that could > > > encode gigs o' chars, > > > etc. > > > > > > > To clarify UTF-16 can represent all characters in UCS-4. > UTF-16, just like > > UTF-8 breaks that annoying simplification that all > characters are fixed > > width. As a happy concidence, the only difference between > UTF-16 and UCS-2 > > is knowing where the character boundaries are. A UTF-16 > encoding of a > > unicode character (e.g. U+000E0020) is itself two valid > UCS-2 characters. > > This is what the surrogate pair range in the Unicode code > space is for. > > > > Making things completly Unicode aware isn't as easy as some > people think, > > have a gander at some of the stuff on www.unicode.org if you haven't > > recently. (esp. the techincal reports) > > e.g. Implementing a regular expression engine that supports > a good chunk of > > Unicode's "features" is very non-trivial, especially if you > don't want it to > > take forever. > > > > Bill