Received: from webgate.proteosys.de (mail.proteosys-ag.com [62.225.9.49]) by lucy.proteosys (8.11.0/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id f1DNm8H29205 for ; Wed, 14 Feb 2001 00:48:08 +0100 Received: by webgate.proteosys.de (8.11.0/8.11.0) with ESMTP id f1DNm8d03705 . for ; Wed, 14 Feb 2001 00:48:08 +0100 Received: from mail.Uni-Mainz.DE (mailserver1.zdv.Uni-Mainz.DE [134.93.8.30]) by mailgate2.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f1DNm7718713 for ; Wed, 14 Feb 2001 00:48:07 +0100 (MET) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C09617.6A682C00" Received: from mailgate2.zdv.Uni-Mainz.DE (mailgate2.zdv.Uni-Mainz.DE [134.93.8.57]) by mail.Uni-Mainz.DE (8.9.3/8.9.3) with ESMTP id AAA15625 for ; Wed, 14 Feb 2001 00:48:07 +0100 (MET) Received: from mail.listserv.gmd.de (mail.listserv.gmd.de [192.88.97.5]) by mailgate2.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f1DNm6718709 for ; Wed, 14 Feb 2001 00:48:07 +0100 (MET) X-MimeOLE: Produced By Microsoft Exchange V6.5 Received: from mail.listserv.gmd.de (192.88.97.5) by mail.listserv.gmd.de (LSMTP for OpenVMS v1.1a) with SMTP id <0.3DA41DEB@mail.listserv.gmd.de>; Wed, 14 Feb 2001 0:47:59 +0100 Received: from RELAY.URZ.UNI-HEIDELBERG.DE by RELAY.URZ.UNI-HEIDELBERG.DE (LISTSERV-TCP/IP release 1.8b) with spool id 487416 for LATEX-L@RELAY.URZ.UNI-HEIDELBERG.DE; Wed, 14 Feb 2001 00:48:02 +0100 Received: from ix.urz.uni-heidelberg.de (mail.urz.uni-heidelberg.de [129.206.119.234]) by relay.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id AAA12002 for ; Wed, 14 Feb 2001 00:47:58 +0100 (MET) Received: from relay.uni-heidelberg.de (relay.uni-heidelberg.de [129.206.100.212]) by ix.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id AAA24180 for ; Wed, 14 Feb 2001 00:47:58 +0100 Received: from knatte.tninet.se (knatte.tninet.se [195.100.94.10]) by relay.uni-heidelberg.de (8.10.2+Sun/8.10.2) with SMTP id f1DNlwg29037 for ; Wed, 14 Feb 2001 00:47:58 +0100 (MET) Received: (qmail 6343 invoked from network); 14 Feb 2001 00:47:57 +0100 Received: from garibaldi.tninet.se (HELO algonet.se) (195.100.94.103) by knatte.tninet.se with SMTP; 14 Feb 2001 00:47:57 +0100 Received: from [195.100.226.152] (du131-226.ppp.su-anst.tninet.se [195.100.226.131]) by garibaldi.tninet.se (BLUETAIL Mail Robustifier 2.2.1) with ESMTP id 351337.108076.982garibaldi-s0 for ; Wed, 14 Feb 2001 00:47:56 +0100 In-Reply-To: <200102131843.SAA06366@penguin.nag.co.uk> References: (message from Hans Aberg on Tue, 13 Feb 2001 18:55:22 +0100) Hans Aberg's message of "Tue, 13 Feb 2001 15:10:54 +0100" Return-Path: X-Sender: haberg@pop.matematik.su.se Content-class: urn:content-classes:message Subject: Re: Side remarks about TeX input sequence Date: Wed, 14 Feb 2001 00:42:35 +0100 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: From: "Hans Aberg" Sender: "Mailing list for the LaTeX3 project" To: "Multiple recipients of list LATEX-L" Reply-To: "Mailing list for the LaTeX3 project" Status: R X-Status: X-Keywords: X-UID: 3908 This is a multi-part message in MIME format. ------_=_NextPart_001_01C09617.6A682C00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable At 18:43 +0000 2001/02/13, David Carlisle wrote: >> What happens if a command in the middle of a line changes the = catcodes > >makes no difference: the notion of line for the input buffer is = hardwired >into the implementation it is not changable via TeX commands and does >not depend on catcodes or the value of \endlinechar. > >> or contains a macro that expands to a \input ? > >The rest of the line of the original file sits buffered in one of those >input streams until the input file finishes. OK. >Incidentally one reason why xmltex can not support utf16 is that >TeX buffers to ^J (or ^M) and throws away any bytes with value 32 that >occur at the end of this buffer, which might just be half of a 16bit >quantity that you'd rather keep. there's no way to control this >behaviour from within TeX. So TeX is a lot less sophisticated than it appears at first sight. >> How can this be true? > >By magic, or the will of Knuth, or something. Well, it's not magic, so it must be the other then. >At 14:44 -0500 2001/02/13, Michael John Downes wrote: >>Sorry, I didn't use the terminology very well. TeX input first goes = into >>a string buffer, one line at a time. This string buffer is the only >>place where TeX deals with ASCII chars as input; all other "input >>streams" are streams of tokens. Tokenization occurs by scanning >>substrings from this string buffer and adding the corresponding token = to >>the current input stream (which if we call it a "buffer", is a = different >>buffer, not the one that contains simple 8-bit characters as first = read >>from a file). >> >>If you get an error "TeX capacity exceeded: buffer size" it means >>that a line of the input file was too long to be read into the string >>buffer. TeX really is a program from another age... Hans Aberg * Email: Hans Aberg * Home Page: * AMS member listing: ------_=_NextPart_001_01C09617.6A682C00 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: Side remarks about TeX input sequence

At 18:43 +0000 2001/02/13, David Carlisle = wrote:
>> What happens if a command in the middle of a = line changes the catcodes
>
>makes no difference: the notion of line for the = input buffer is hardwired
>into the implementation it is not changable via = TeX commands and does
>not depend on catcodes or the value of = \endlinechar.
>
>>  or contains a macro that expands to a = \input <filename>?
>
>The rest of the line of the original file sits = buffered in one of those
>input streams until the input file = finishes.

OK.

>Incidentally one reason why xmltex can not support = utf16 is that
>TeX buffers to ^J (or ^M) and throws away any = bytes with value 32 that
>occur at the end of this buffer, which might just = be half of a 16bit
>quantity that you'd rather keep. there's no way = to control this
>behaviour from within TeX.

So TeX is a lot less sophisticated than it appears at = first sight.

>> How can this be true?
>
>By magic, or the will of Knuth, or = something.

Well, it's not magic, so it must be the other = then.

>At 14:44 -0500 2001/02/13, Michael John Downes = wrote:
>>Sorry, I didn't use the terminology very = well. TeX input first goes into
>>a string buffer, one line at a time. This = string buffer is the only
>>place where TeX deals with ASCII chars as = input; all other "input
>>streams" are streams of tokens. = Tokenization occurs by scanning
>>substrings from this string buffer and adding = the corresponding token to
>>the current input stream (which if we call it = a "buffer", is a different
>>buffer, not the one that contains simple = 8-bit characters as first read
>>from a file).
>>
>>If you get an error "TeX capacity = exceeded: buffer size" it means
>>that a line of the input file was too long to = be read into the string
>>buffer.

TeX really is a program from another age...

  Hans Aberg
          &nbs= p;       * Email: Hans Aberg <mailto:haberg@member.ams.org>= ;
          &nbs= p;       * Home Page: <http://www.matematik.su.se/~= haberg/>
          &nbs= p;       * AMS member listing: <http://www.ams.org/cml/>

------_=_NextPart_001_01C09617.6A682C00--