MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Message-ID:  <CAFUtaNEvBktQbJytj8MhWuZtOBODbT+AQ_eBHqxhh7DXu29BnQ@mail.gmail.com>
Date:         Thu, 19 Jul 2012 18:33:38 -0400
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@listserv.uni-heidelberg.de>
Sender: Mailing list for the LaTeX3 project <LATEX-L@listserv.uni-heidelberg.de>
From: "Joel C. Salomon" <joelcsalomon@GMAIL.COM>
Subject: Peek ahead for next token not in token-list
To: LATEX-L@listserv.uni-heidelberg.de
Precedence: list
Status: R

Cross-posted to <http://tex.stackexchange.com/q/63971/2966>, modulo
some details.

I've been developing my xpeek package with lots of help from you
folks; see <https://github.com/jcsalomon/xpeek>. It’s intended to help
write commands which, like the familiar `\xspace`, look ahead into the
input stream and choose what to do based on what follows.  Although
the interface needs to change, it has enough functionality to
implement something like xspace, or the intelligent italic-correction
from LaTeX's \textit.

Which means it has the same familiar limitation as \textit, best
illustrated by example:

    \textit{foof}\xspace. \textit{foof}\xspace!  % bad italic
correction before "."
    \def\nocorrlist{.,\xspace}
    \textit{foof}\xspace. \textit{foof}\xspace!  % no italic
correction before "!"

Thinking about the problem, it seems I need the ability to scan ahead
in the input stream, ignoring tokens from one list while looking for
tokens from another.  In Expl3 terms, I’m hoping to define something
like `\peek_inlist_ignore_auxlist:nnTF`.

As an initial step toward this end, I’m trying to simply scan ahead
for the next input token not on the ignore list. For example:

    \ExplSyntaxOn
    \tl_const:Nn \ignorelist {~,.;:!}
    \NewDocumentCommand { \nextnonpunc } { }
      {
        \peek_ignore_list:N \ignorelist
        `\l_peek_token'
      }
    \ExplSyntaxOff
    The next non-punctuation mark is \nextnonpunc.,;:! xyz

yielding

> The next non-punctuation mark is ‘x’.,;:! xyz

But how can I implement `\peek_ignore_list:N`?

The direction I’m considering is to read ahead, consuming tokens. Each
token read is added to a save-list and compared to the ignore-list. If
it’s on the ignore-list, continue; otherwise put the save-list back on
the input stream and stop.

Does this sound reasonable so far?

To consume tokens one-by-one, I built this function:

    \cs_new_protected:Npn \peek_meaning_really_remove:NTF #1 #2 #3
      {
        \peek_meaning_remove:NTF #1
          { #2 }
          {
            \peek_meaning_remove:NT \l_peek_token
              { #3 }
          }
      }

(This should be created via \prg_new_conditional, but I haven’t yet
figured that out.)

On TeX.SX, I just asked for some hand-holding; here, I want to discuss
the topic.

Is the direction I'm taking appropriate for what I'm trying to do?

Is there some existing functionality that would help that I'm overlooking?

--Joel