Received: from webgate.proteosys.de (mail.proteosys-ag.com [62.225.9.49]) by lucy.proteosys (8.11.0/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id f4QIQgf22891 for ; Sat, 26 May 2001 20:26:42 +0200 Received: by webgate.proteosys.de (8.11.0/8.11.0) with ESMTP id f4QIQg723569 . for ; Sat, 26 May 2001 20:26:42 +0200 Received: from mail.Uni-Mainz.DE (mailserver1.zdv.Uni-Mainz.DE [134.93.8.30]) by mailgate2.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f4QIQf025377 for ; Sat, 26 May 2001 20:26:41 +0200 (MET DST) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C0E611.69309D00" Received: from mailgate1.zdv.Uni-Mainz.DE (mailgate1.zdv.Uni-Mainz.DE [134.93.8.56]) by mail.Uni-Mainz.DE (8.9.3/8.9.3) with ESMTP id UAA09108 for ; Sat, 26 May 2001 20:26:41 +0200 (MEST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Received: from mail.listserv.gmd.de (mail.listserv.gmd.de [192.88.97.5]) by mailgate1.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f4QIQeU27757 for ; Sat, 26 May 2001 20:26:41 +0200 (MET DST) Received: from mail.listserv.gmd.de (192.88.97.5) by mail.listserv.gmd.de (LSMTP for OpenVMS v1.1a) with SMTP id <10.5C0FF4C9@mail.listserv.gmd.de>; Sat, 26 May 2001 20:24:42 +0200 Received: from RELAY.URZ.UNI-HEIDELBERG.DE by RELAY.URZ.UNI-HEIDELBERG.DE (LISTSERV-TCP/IP release 1.8b) with spool id 497048 for LATEX-L@RELAY.URZ.UNI-HEIDELBERG.DE; Sat, 26 May 2001 20:26:37 +0200 Received: from ix.urz.uni-heidelberg.de (mail.urz.uni-heidelberg.de [129.206.119.234]) by relay.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id UAA02493 for ; Sat, 26 May 2001 20:26:35 +0200 (MET DST) Received: from relay.uni-heidelberg.de (relay.uni-heidelberg.de [129.206.100.212]) by ix.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id UAA27308 for ; Sat, 26 May 2001 20:26:36 +0200 Received: from naf1.mathematik.uni-tuebingen.de (naf1.mathematik.uni-tuebingen.de [134.2.161.197]) by relay.uni-heidelberg.de (8.10.2+Sun/8.10.2) with ESMTP id f4QIQa105333 for ; Sat, 26 May 2001 20:26:36 +0200 (MET DST) Received: from na13.mathematik.uni-tuebingen.de (na13 [134.2.161.180]) by naf1.mathematik.uni-tuebingen.de (8.9.3+Sun/8.9.3) with ESMTP id UAA20579; Sat, 26 May 2001 20:26:33 +0200 (MET DST) Received: (from oliver@localhost) by na13.mathematik.uni-tuebingen.de (8.9.3+Sun/8.9.1) id UAA20854; Sat, 26 May 2001 20:26:32 +0200 (MET DST) Return-Path: X-Mailer: VM 6.88 under Emacs 20.7.2 x-mime-autoconverted: from quoted-printable to 8bit by relay.urz.uni-heidelberg.de id UAA02494 X-Authentication-Warning: na13.mathematik.uni-tuebingen.de: oliver set sender to oliver@na13 using -f Content-class: urn:content-classes:message Subject: \InputTranslation Date: Sat, 26 May 2001 19:26:32 +0100 Message-ID: <15119.62808.151690.192812@gargle.gargle.HOWL> X-MS-Has-Attach: X-MS-TNEF-Correlator: From: "Marcel Oliver" Sender: "Mailing list for the LaTeX3 project" To: "Multiple recipients of list LATEX-L" Reply-To: "Mailing list for the LaTeX3 project" Status: R X-Status: X-Keywords: X-UID: 4106 This is a multi-part message in MIME format. ------_=_NextPart_001_01C0E611.69309D00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I'd like to bring the discussion back to the ICR issue, in particular how a hypothetical successor to TeX should handle input encodings. I think the point that Omega does not do it the "right" way has been made pretty clearly. But what should be the "right" way? I repost some thoughts from last week that seem to have been lost among the \(var)epsilons. --Marcel Frank Mittelbach writes: > > In fact, \InputEncoding was not intended for that, but only for > > "technical" translations which applies to the whole document > > as one byte -> two byte or little endian -> big endian. The main > > problem of it is that it doesn't translate macros: > > \def\myE{=C9} > > \InputEncoding > > =C9\myE > > \InputEncoding is the point where one need to go from external > source encoding to OICR that is precisely the wound: the current > \InputEncoding isn't doing this job fully (and that it is not clear > how to do it properly (to be fair)) How about this: - There is one default \InputTranslation (this, rather than \InputEncoding, is the official name of the Omega command) which may need to be specified at the time of format creation. This encoding is the one that all macro names need to be in, as well as the encoding initially selected for text (I think it does not make any sense to allow for multiply encoded macro names in a single document). As there is no legacy cruft with regard to macro names, we may as well force this default encoding to be UTF-8. - Changes in the \InputTranslation follow the usual TeX scoping rules (this is obviously not how Omega currently does it), and take effect immediately during the initial tokenization. This would mean that the characters \ { } must be in their expected position in every permissible encoding, but I guess that's not any more restrictive than what we currently have. I also assume that TeX (Omega) always knows whether it is parsing code or text, so that it can select the default for code, and the top of the encoding stack for text. - Regarding Javier's above example: I think this is the correct and expected behavior. I want to be able to able to write: \begin{chinese} \newcommand{\foo}{***something chinese***} \newcommand{\bar}{***and some more chinese***} \end{chinese} The chinese characters \foo\ and \bar\ are not easy to enter on a western keyboard. If you need to frequently use \foo\ in your scholarly discussion of Chinese literature, it is better to first define macros for all the chinese characters you need, and then just write \verb|\foo| whenever you need \foo. (I don't know if this babel-like begin-end of a language selection would actually be legal in the document preamble, but I think the strategy is very natural at least.) - It may be more of a problem how to deal with \'e and the like. Would it be possible to force immediate expansion into the corresponding internal Unicode token? ------_=_NextPart_001_01C0E611.69309D00 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable \InputTranslation

I'd like to bring the discussion back to the ICR = issue, in particular
how a hypothetical successor to TeX should handle = input encodings.  I
think the point that Omega does not do it the = "right" way has been
made pretty clearly.

But what should be the "right" way?  I = repost some thoughts from last
week that seem to have been lost among the = \(var)epsilons.

--Marcel

Frank Mittelbach writes:
 >  > In fact, \InputEncoding was not = intended for that, but only for
 >  > "technical" = translations which applies to the whole document
 >  > as one byte -> two byte or = little endian -> big endian. The main
 >  > problem of it is that it = doesn't translate macros:
 >  > \def\myE{=C9}
 >  > \InputEncoding <an = encoding>
 >  > =C9\myE
 >
 > \InputEncoding is the point where one need = to go from external
 > source encoding to OICR that is precisely = the wound: the current
 > \InputEncoding isn't doing this job fully = (and that it is not clear
 > how to do it properly (to be fair))

How about this:

- There is one default \InputTranslation (this, rather = than
  \InputEncoding, is the official name of the = Omega command) which may
  need to be specified at the time of format = creation.  This encoding
  is the one that all macro names need to be in, = as well as the
  encoding initially selected for text (I think = it does not make any
  sense to allow for multiply encoded macro = names in a single
  document).  As there is no legacy cruft = with regard to macro names,
  we may as well force this default encoding to = be UTF-8.

- Changes in the \InputTranslation follow the usual = TeX scoping rules
  (this is obviously not how Omega currently = does it), and take effect
  immediately during the initial = tokenization.  This would mean that
  the characters \ { } must be in their expected = position in every
  permissible encoding, but I guess that's not = any more restrictive
  than what we currently have.  I also = assume that TeX (Omega) always
  knows whether it is parsing code or text, so = that it can select the
  default for code, and the top of the encoding = stack for text.

- Regarding Javier's above example: I think this is = the correct and
  expected behavior.  I want to be able to = able to write:

  \begin{chinese}
    \newcommand{\foo}{***something = chinese***}
    \newcommand{\bar}{***and some more = chinese***}
  \end{chinese}

  The chinese characters \foo\ and \bar\ are not = easy to enter on a
  western keyboard.  If you need to = frequently use \foo\ in your
  scholarly discussion of Chinese literature, it = is better to first
  define macros for all the chinese characters = you need, and then just
  write \verb|\foo| whenever you need = \foo.

  (I don't know if this babel-like begin-end of a = language selection
  would actually be legal in the document = preamble,  but I think the
  strategy is very natural at least.)

- It may be more of a problem how to deal with \'e and = the like.
  Would it be possible to force immediate = expansion into the
  corresponding internal Unicode token?

------_=_NextPart_001_01C0E611.69309D00--