Received: from webgate.proteosys.de (mail.proteosys-ag.com [62.225.9.49]) by lucy.proteosys (8.11.0/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id f1DJixH28004 for ; Tue, 13 Feb 2001 20:44:59 +0100 Received: by webgate.proteosys.de (8.11.0/8.11.0) with ESMTP id f1DJixd02659 . for ; Tue, 13 Feb 2001 20:44:59 +0100 Received: from mail.Uni-Mainz.DE (mailserver1.zdv.Uni-Mainz.DE [134.93.8.30]) by mailgate1.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f1DJiwM19328 for ; Tue, 13 Feb 2001 20:44:58 +0100 (MET) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C095F5.72AF6F80" Received: from mailgate2.zdv.Uni-Mainz.DE (mailgate2.zdv.Uni-Mainz.DE [134.93.8.57]) by mail.Uni-Mainz.DE (8.9.3/8.9.3) with ESMTP id UAA03223 for ; Tue, 13 Feb 2001 20:44:58 +0100 (MET) X-MimeOLE: Produced By Microsoft Exchange V6.5 Received: from mail.listserv.gmd.de (mail.listserv.gmd.de [192.88.97.5]) by mailgate2.zdv.Uni-Mainz.DE (8.11.0/8.10.2) with ESMTP id f1DJiv703157 for ; Tue, 13 Feb 2001 20:44:57 +0100 (MET) Received: from mail.listserv.gmd.de (192.88.97.5) by mail.listserv.gmd.de (LSMTP for OpenVMS v1.1a) with SMTP id <6.45D3EE6C@mail.listserv.gmd.de>; Tue, 13 Feb 2001 20:44:50 +0100 Received: from RELAY.URZ.UNI-HEIDELBERG.DE by RELAY.URZ.UNI-HEIDELBERG.DE (LISTSERV-TCP/IP release 1.8b) with spool id 488931 for LATEX-L@RELAY.URZ.UNI-HEIDELBERG.DE; Tue, 13 Feb 2001 20:44:54 +0100 Received: from ix.urz.uni-heidelberg.de (mail.urz.uni-heidelberg.de [129.206.119.234]) by relay.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id UAA09220 for ; Tue, 13 Feb 2001 20:44:52 +0100 (MET) Received: from relay.uni-heidelberg.de (relay.uni-heidelberg.de [129.206.100.212]) by ix.urz.uni-heidelberg.de (8.8.8/8.8.8) with ESMTP id UAA27608 for ; Tue, 13 Feb 2001 20:44:51 +0100 Received: from ams.org (sun06.ams.org [130.44.1.6]) by relay.uni-heidelberg.de (8.10.2+Sun/8.10.2) with ESMTP id f1DJing14774 for ; Tue, 13 Feb 2001 20:44:50 +0100 (MET) Received: (from mjd@localhost) by ams.org (8.11.1/8.11.1) id f1DJikE13194; Tue, 13 Feb 2001 14:44:46 -0500 (EST) In-Reply-To: Hans Aberg's message of "Tue, 13 Feb 2001 18:55:22 +0100" Lines: 23 References: Return-Path: X-Mailer: Gnus v5.7/Emacs 20.7 Content-class: urn:content-classes:message Subject: Re: Side remarks about TeX input sequence Date: Tue, 13 Feb 2001 20:44:46 +0100 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: From: "Michael John Downes" Sender: "Mailing list for the LaTeX3 project" To: "Multiple recipients of list LATEX-L" Reply-To: "Mailing list for the LaTeX3 project" Status: R X-Status: X-Keywords: X-UID: 3893 This is a multi-part message in MIME format. ------_=_NextPart_001_01C095F5.72AF6F80 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hans Aberg writes: > >> The >> tokens at need. > > > >TeX reads into the buffer one line at a time. > > How can this be true? What happens if a command in the middle of a = line > changes the catcodes, or contains a macro that expands to a \input > ? Sorry, I didn't use the terminology very well. TeX input first goes into a string buffer, one line at a time. This string buffer is the only place where TeX deals with ASCII chars as input; all other "input streams" are streams of tokens. Tokenization occurs by scanning substrings from this string buffer and adding the corresponding token to the current input stream (which if we call it a "buffer", is a different buffer, not the one that contains simple 8-bit characters as first read from a file). If you get an error "TeX capacity exceeded: buffer size" it means that a line of the input file was too long to be read into the string buffer. ------_=_NextPart_001_01C095F5.72AF6F80 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: Side remarks about TeX input sequence

Hans Aberg <haberg@MATEMATIK.SU.SE> = writes:

> >> The <not yet gulped up ASCII (or = 8-bit) buffer is read converted into
> >> tokens at need.
> >
> >TeX reads into the buffer one line at a = time.
>
> How can this be true? What happens if a command = in the middle of a line
> changes the catcodes, or contains a macro that = expands to a \input
> <filename>?

Sorry, I didn't use the terminology very well. TeX = input first goes into
a string buffer, one line at a time. This string = buffer is the only
place where TeX deals with ASCII chars as input; all = other "input
streams" are streams of tokens. Tokenization = occurs by scanning
substrings from this string buffer and adding the = corresponding token to
the current input stream (which if we call it a = "buffer", is a different
buffer, not the one that contains simple 8-bit = characters as first read
from a file).

If you get an error "TeX capacity exceeded: = buffer size" it means
that a line of the input file was too long to be read = into the string
buffer.

------_=_NextPart_001_01C095F5.72AF6F80--