Received: from webgate.proteosys.de (mail.proteosys-ag.com [62.225.9.49]) by lucy.proteosys (8.11.0/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id f1JGio104502 for ; Mon, 19 Feb 2001 17:44:50 +0100 Received: by webgate.proteosys.de (8.11.0/8.11.0) with ESMTP id f1JGiod28070 . for ; Mon, 19 Feb 2001 17:44:50 +0100 Received: from mail.Uni-Mainz.DE (mailserver1.zdv.Uni-Mainz.DE [134.93.8.30]) by mailgate1.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f1JGinH12542 for ; Mon, 19 Feb 2001 17:44:49 +0100 (MET) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C09A93.467F6D00" Received: from mailgate2.zdv.Uni-Mainz.DE (mailgate2.zdv.Uni-Mainz.DE [134.93.8.57]) by mail.Uni-Mainz.DE (8.9.3/8.9.3) with ESMTP id RAA28129 for ; Mon, 19 Feb 2001 17:44:48 +0100 (MET) X-MimeOLE: Produced By Microsoft Exchange V6.5 Received: from mail.listserv.gmd.de (mail.listserv.gmd.de [192.88.97.5]) by mailgate2.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f1JGilQ25513 for ; Mon, 19 Feb 2001 17:44:47 +0100 (MET) Received: from mail.listserv.gmd.de (192.88.97.5) by mail.listserv.gmd.de (LSMTP for OpenVMS v1.1a) with SMTP id <9.181F8659@mail.listserv.gmd.de>; Mon, 19 Feb 2001 17:44:38 +0100 Received: from RELAY.URZ.UNI-HEIDELBERG.DE by RELAY.URZ.UNI-HEIDELBERG.DE (LISTSERV-TCP/IP release 1.8b) with spool id 490559 for LATEX-L@RELAY.URZ.UNI-HEIDELBERG.DE; Mon, 19 Feb 2001 17:44:44 +0100 Received: from ix.urz.uni-heidelberg.de (mail.urz.uni-heidelberg.de [129.206.119.234]) by relay.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id RAA28139 for ; Mon, 19 Feb 2001 17:44:43 +0100 (MET) Received: from relay.uni-heidelberg.de (relay.uni-heidelberg.de [129.206.100.212]) by ix.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id RAA25766 for ; Mon, 19 Feb 2001 17:44:43 +0100 Received: from abel.math.umu.se (abel.math.umu.se [130.239.20.139]) by relay.uni-heidelberg.de (8.10.2+Sun/8.10.2) with ESMTP id f1JGihx15304 for ; Mon, 19 Feb 2001 17:44:43 +0100 (MET) Received: from [130.239.20.144] (mac144.math.umu.se [130.239.20.144]) by abel.math.umu.se (8.9.2/8.9.2) with ESMTP id RAA15498 for ; Mon, 19 Feb 2001 17:42:46 +0100 (CET) In-Reply-To: <14991.50325.136586.919754@gargle.gargle.HOWL> References: <14991.47029.909613.454490@spqr2.oucs.ox.ac.uk> <200102131655.QAA05656@penguin.nag.co.uk> <14985.27933.205182.551235@gargle.gargle.HOWL> <14990.45178.192666.234476@istrati.zdv.uni-mainz.de> <14991.47029.909613.454490@spqr2.oucs.ox.ac.uk> Return-Path: X-Sender: lars@abel.math.umu.se x-mime-autoconverted: from quoted-printable to 8bit by relay.urz.uni-heidelberg.de id RAA28140 Content-class: urn:content-classes:message Subject: Re: Multilingual Encodings Summary 2.0 Date: Mon, 19 Feb 2001 17:44:41 +0100 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: From: =?iso-8859-1?Q?Lars_Hellstr=F6m?= Sender: "Mailing list for the LaTeX3 project" To: "Multiple recipients of list LATEX-L" Reply-To: "Mailing list for the LaTeX3 project" Status: R X-Status: X-Keywords: X-UID: 3984 This is a multi-part message in MIME format. ------_=_NextPart_001_01C09A93.467F6D00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable In http://na.uni-tuebingen.de/~oliver/latex/encodings-2.tex, Marcel = Oliver wrote (edited?): [Section 2] >\item It is rather hard to make a new font available under LaTeX. > Essentially one must create a virtual font which has all the > character slots in the places where hyphenation expects them to be. > > FM disagrees. > > Can we have more comments from people who have actually dealt with > making fonts available? I believe what Frank disagreed about here is primarily that it should be = a big problem that, with the current TeX engine, the hyphenation patterns encoding has to be the same as the font encoding. It certainly is a = design flaw in TeX which should be fixed in its successor, but it is nothing = you cannot live with as a user. Even when you have to use one \language code for each language+encoding pair you have you don't reach TeX's limit for the number of different languages, because there are never more than a handfull of font encodings that are useful for any given language = (possibly with the exception of the a--z only languages, but then you can reuse = the same hyphenation patterns anyway). The problems that exist are more at = an administative level, which means things get more complicated for those = that implement language support in LaTeX, but those are manageable. [Subsection 3.5] >Biased judgment: Cross-platform support is probably nothing to worry >about at this point. As far as commercial TeX goes: vendors should >find it easy to port to Omega. If not, they must have already created >a significant code fork, and should therefore fall outside our >consideration. (Diverging opinions, anyone?) I'm not so sure the port is that easy. In the case of Textures, the main extra feature not present in normal TeX seems to be the "flash mode", in which the text is typeset and shown in a different window as you are = typing it. Something like this has to involve a major rewrite of TeX as a = program, so don't count on them doing it all over again for Omega in the = foreseeable future. (OTOH this never seemed to work satisfactory with LaTeX when I = used Textures back in 95/96; this was a major reason why I switched to OzTeX. But there are a lot of people who still use Textures, so we can expect = that they will not switch to Omega unless Blue Sky reconstructs it = similarly.) [Section 4:] >\item Could type a math paper without saying \verb+Schr\"o\-din\-ger+ > all the time. (Yes, I know it's easy to get around this, but it > requires active intervention on part of the user when it's logically > not necessary.) I don't get this argument. What's your point? [Still Section 4:] >\item Availability of different fonts may increase as they would > typically not need to be VF re-encoded. > > FM: ``i guess this is a red herring. you need to provide support for > a font in any encoding to make it accessible to a typesetting > system. it might be easier, perhaps but in any case it is a onetime > effort.'' It's _not_ hard to make a virtual font, provided you use some tool like fontinst to do all the file format conversions and shuffling numbers = around (without such a tool I can agree it is a nightmare). I certainly found = it much easier to use fontinst "out of the box" than I found babel = (although I may be a bad example as I have now ended up being the maintainer of fontinst). What has been missing though is the basic support for = non-latin scripts, but there the situation has improved lately with Vladimir Volovich's t2 package (currently in CTAN:macros/latex/contrib/supported/t2/) which covers the cyrillic = script. Support for other scripts (even if it is no more than the basic descriptions of some encoding as an .etx file) is welcome and will be = added to the distribution if only someone contributes it. I suspect a partial reason noone seems to have done so for the other scripts is that you = need to understand a lot of esoterical matters (many of which are poorly documented, or at least very hard to find the documentation for, if documented at all) to develop support for new encodings (whether for = LaTeX or for fontinst) and thus the set of people who are qualified to do it = may in many cases be almost empty. But in any case it is a onetime effort; = if you do it for one font for a script, there's very little work involved = in later doing it for any other font for that script (unless you're a font freak which absolutely has to produce the very best results with the = glyphs available in the base fonts). My experience is that the driver map files are troublesome much more = often than the VF files are, but that problem should also become less = important, as fontinst v1.9 can now write the necessary map files for e.g. dvips as well. Concerning the appendices: Appendix A and C really adress the same matter, so they should be = merged. The reason Achim's suggestion is unfeasible is the mere size of the automaton you would have to implement in TeX's mouth to handle all the emulation necessary; it would have to completely take over tasks whose current implementation ranges over about a dozen of the 55 sections (including index) that comprise TeX (the program). Most of Appendix B can be removed. With the exception of the JB remark = at the end I don't think it is relevant for the discussion. Concerning the inputenc problems discussed in Appendix D I don't think = the way to go is to reimplement \inputencoding to select an OCP, due to the problems this raises with switching encoding inside a paragraph that are described. If anything, the use of the inputenc package should select an OCP that does something equivalent to the current active character mechanism, and then as today the exact meaning of a character should = depend on the current definition of a macro. Although imperfect, this approach doesn't introduce any new problems, and those who need something better = can always use Unicode instead of the old 8-bit encodings. Question raised by this: Can OCPs output control sequences, or do they = just produce characters? Can one specify what catcode the characters should = have? Another question regarding OCPs: Is the OCP model general enough to = support a reasonable size OCP that translates UTF-8 to 32-bit Unicode, or is it necessary to implement the entire translation as a gigantic (2^31 = entries) table? Lars Hellstr=F6m ------_=_NextPart_001_01C09A93.467F6D00 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: Multilingual Encodings Summary 2.0

In http://= na.uni-tuebingen.de/~oliver/latex/encodings-2.tex, Marcel = Oliver
wrote (edited?):

[Section 2]
>\item It is rather hard to make a new font = available under LaTeX.
>  Essentially one must create a virtual font = which has all the
>  character slots in the places where = hyphenation expects them to be.
>
>  FM disagrees.
>
>  Can we have more comments from people who = have actually dealt with
>  making fonts available?

I believe what Frank disagreed about here is primarily = that it should be a
big problem that, with the current TeX engine, the = hyphenation patterns
encoding has to be the same as the font encoding. It = certainly is a design
flaw in TeX which should be fixed in its successor, = but it is nothing you
cannot live with as a user. Even when you have to use = one \language code
for each language+encoding pair you have you don't = reach TeX's limit for
the number of different languages, because there are = never more than a
handfull of font encodings that are useful for any = given language (possibly
with the exception of the a--z only languages, but = then you can reuse the
same hyphenation patterns anyway). The problems that = exist are more at an
administative level, which means things get more = complicated for those that
implement language support in LaTeX, but those are = manageable.

[Subsection 3.5]
>Biased judgment: Cross-platform support is = probably nothing to worry
>about at this point.  As far as commercial = TeX goes: vendors should
>find it easy to port to Omega.  If not, they = must have already created
>a significant code fork, and should therefore = fall outside our
>consideration.  (Diverging opinions, = anyone?)

I'm not so sure the port is that easy. In the case of = Textures, the main
extra feature not present in normal TeX seems to be = the "flash mode", in
which the text is typeset and shown in a different = window as you are typing
it. Something like this has to involve a major = rewrite of TeX as a program,
so don't count on them doing it all over again for = Omega in the foreseeable
future. (OTOH this never seemed to work satisfactory = with LaTeX when I used
Textures back in 95/96; this was a major reason why I = switched to OzTeX.
But there are a lot of people who still use Textures, = so we can expect that
they will not switch to Omega unless Blue Sky = reconstructs it similarly.)

[Section 4:]
>\item Could type a math paper without saying = \verb+Schr\"o\-din\-ger+
>  all the time.  (Yes, I know it's easy = to get around this, but it
>  requires active intervention on part of = the user when it's logically
>  not necessary.)

I don't get this argument. What's your point?

[Still Section 4:]
>\item Availability of different fonts may = increase as they would
>  typically not need to be VF = re-encoded.
>
>  FM: ``i guess this is a red herring. you = need to provide support for
>  a font in any encoding to make it = accessible to a typesetting
>  system. it might be easier, perhaps but in = any case it is a onetime
>  effort.''

It's _not_ hard to make a virtual font, provided you = use some tool like
fontinst to do all the file format conversions and = shuffling numbers around
(without such a tool I can agree it is a nightmare). = I certainly found it
much easier to use fontinst "out of the = box" than I found babel (although I
may be a bad example as I have now ended up being the = maintainer of
fontinst). What has been missing though is the basic = support for non-latin
scripts, but there the situation has improved lately = with Vladimir
Volovich's t2 package (currently in
CTAN:macros/latex/contrib/supported/t2/) which covers = the cyrillic script.

Support for other scripts (even if it is no more than = the basic
descriptions of some encoding as an .etx file) is = welcome and will be added
to the distribution if only someone contributes it. I = suspect a partial
reason noone seems to have done so for the other = scripts is that you need
to understand a lot of esoterical matters (many of = which are poorly
documented, or at least very hard to find the = documentation for, if
documented at all) to develop support for new = encodings (whether for LaTeX
or for fontinst) and thus the set of people who are = qualified to do it may
in many cases be almost empty. But in any case it is = a onetime effort; if
you do it for one font for a script, there's very = little work involved in
later doing it for any other font for that script = (unless you're a font
freak which absolutely has to produce the very best = results with the glyphs
available in the base fonts).

My experience is that the driver map files are = troublesome much more often
than the VF files are, but that problem should also = become less important,
as fontinst v1.9 can now write the necessary map = files for e.g. dvips as
well.

Concerning the appendices:
Appendix A and C really adress the same matter, so = they should be merged.
The reason Achim's suggestion is unfeasible is the = mere size of the
automaton you would have to implement in TeX's mouth = to handle all the
emulation necessary; it would have to completely take = over tasks whose
current implementation ranges over about a dozen of = the 55 sections
(including index) that comprise TeX (the = program).

Most of Appendix B can be removed. With the exception = of the JB remark at
the end I don't think it is relevant for the = discussion.

Concerning the inputenc problems discussed in Appendix = D I don't think the
way to go is to reimplement \inputencoding to select = an OCP, due to the
problems this raises with switching encoding inside a = paragraph that are
described. If anything, the use of the inputenc = package should select an
OCP that does something equivalent to the current = active character
mechanism, and then as today the exact meaning of a = character should depend
on the current definition of a macro. Although = imperfect, this approach
doesn't introduce any new problems, and those who = need something better can
always use Unicode instead of the old 8-bit = encodings.

Question raised by this: Can OCPs output control = sequences, or do they just
produce characters? Can one specify what catcode the = characters should have?

Another question regarding OCPs: Is the OCP model = general enough to support
a reasonable size OCP that translates UTF-8 to 32-bit = Unicode, or is it
necessary to implement the entire translation as a = gigantic (2^31 entries)
table?

Lars Hellstr=F6m

------_=_NextPart_001_01C09A93.467F6D00--