MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <88ED1D30-FF25-423D-B73B-485982A3CB69@guerilla.net.au>
            <5.1.0.14.0.20051110085835.024b6700@pop3.web.de>
            <20051110100945.GA3265@m0A023274.vpn.uni-freiburg.de>
            <17267.14888.849329.473348@isostar.informatik.uni-stuttgart.de>
            <8BDE465E-4169-40F3-A72C-60A331A9092C@guerilla.net.au>
Message-ID:  <17272.53237.920872.634154@pussy.r.npc.de>
Date:         Mon, 14 Nov 2005 18:57:09 +0100
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
Sender: Mailing list for the LaTeX3 project <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
From: Joachim Schrod <jschrod@ACM.ORG>
Subject: Re: LaTeX3 access, pdfTeX extensions, &c.
To: LATEX-L@LISTSERV.UNI-HEIDELBERG.DE
In-Reply-To:  <8BDE465E-4169-40F3-A72C-60A331A9092C@guerilla.net.au>
Precedence: list
Status: R

>>>>> "WR" == Will Robertson <will@GUERILLA.NET.AU> writes:

WR> The token stream won't change retrospectively, so in theory it
WR> should be able to be examined.

But it does change. That's the whole point of macros, to change the
token stream.

With a definition of 

    \newcommand\a[1]{f1\b{#1}}
    \newcommand\b[2]{\par #2 #1}

and an input of

    ab\a12

this results in the following token stream [*]

    (char "a") (char "b") (cseq "a") (other "1") (other "2")

which is in turn transformed into the token stream

    (char "a") (char "b") (cseq "b") (bgroup "{") (other "1")
    (egroup "}") (other "2")

and then into the token stream

    (char "a") (char "b") (cseq "par") (other "2") (other "1")

Which of those do you want to keep? There are good arguments for all
of them.

	Joachim

[*] The notation per token is (TOKEN-TYPE NAME), where the token types
are hopefully self-explained. Please note that TOKEN-TYPE is not
identical to catcodes, as catcodes are attributes of input characters
for configuration of lexical analysis (aka scanning), and not
attributes of tokens (which are the result of the scan process). There
are some catcodes that have a one-to-one mapping to token types, of
course, when the token is created from one input char according to
that catcode.

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Joachim Schrod					Email: jschrod@acm.org
Roedermark, Germany

	``How do we persuade new users that spreading fonts across the page
	like peanut butter across hot toast is not necessarily the route to
	typographic excellence?''			-- Peter Flynn