Message-ID:  <199703021739.LAA27835@dcdrjh.fnal.gov>
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@RELAY.URZ.UNI-HEIDELBERG.DE>
Date:         Sun, 2 Mar 1997 11:39:00 -0600
From: "Randolph J. Herber" <herber@DCDRJH.FNAL.GOV>
Sender: Mailing list for the LaTeX3 project
              <LATEX-L@RELAY.URZ.UNI-HEIDELBERG.DE>
To: Multiple recipients of list LATEX-L
              <LATEX-L@RELAY.URZ.UNI-HEIDELBERG.DE>
Subject:      Re: Shortref mechanism
Status: R

The following header lines retained to affect attribution:
|Date:         Sun, 2 Mar 1997 13:48:57 +0100
|From: Hans Aberg <haberg@MATEMATIK.SU.SE>
|Subject:      Re: Shortref mechanism

|Frank Mittelbach <Frank.Mittelbach@UNI-MAINZ.DE> wrote:

|>The above means that the shortref mechanism either has to be fully
|>expandable or that one can't use it to produce glyphs that might play
|>a part in ligature or kerning tables.

|  I cannot follow the details in your reasoning, but I can note that with
|deterministic parsing, the method generally used in LaTeX, conditional
 ^^^^^^^^^^^^^ ^^^^^^^                                      ^^^^^^^^^^^
|parsing have such limits.
 ^^^^^^^
|  But with non-deterministic parsing more general things can be done:
            ^^^^^^^^^^^^^^^^^
|  For example, I just made a definition command that can produce commands
|having optional arguments; in this general approach, I had to switch from
|LaTeX style deterministic parsing to non-deterministic parsing.
             ^^^^^^^^^^^^^ ^^^^^^^    ^^^^^^^^^^^^^^^^^

        ...

|  Hans Aberg

To someone that has written several small compilers and has studied automata
theory at the doctorate level, your word choice as high-lighted above is
quite jarring.  By using a power automaton, a non-deterministic automaton
can be reduced to a deterministic automaton.  Therefore, one does not gain
any power of expression by using a non-deterministic automaton, rather one
only gains compaction of the description.

I believe that what you intended is the distinction of context free and
context sensitive languages.  From what I have read in the TeX book, the
tokenizer of TeX is context sensitive with a single character look-ahead
and the TeX language based on the recognized tokens is context free.

It is a significant change in the behavior of the TeX language to change
it from being context free to being context sensitive.  But, it may be
a necessary change.  Most modern computer languages are context sensitive
with a single token look ahead.  A few look ahead two tokens in some
situations.  I imagine that some look ahead three tokens.  Parser generators
for single token look ahead readily are availed.

What you are proposing is a change from zero token look ahead to one
token look ahead.

Randolph J. Herber, herber@dcdrjh.fnal.gov, +1 630 840 2966,
CD/OSS/CDF CDF-PK-149O Mail Stop 234
Fermilab, Kirk & Pine Rds., P.O. Box 500, Batavia, IL 60190-0500.
(Speaking for myself and not for US, US DOE, FNAL nor URA.)
(Product, trade, or service marks herein belong to their respective owners.)
N 41 50 26.3 W 88 14 54.4 and altitude 700' approximately, WGS84 datum.