User-Agent: Thunderbird 2.0.0.22 (Windows/20090605)
MIME-Version: 1.0
References: <4A7921CF.5020803@morningstar2.co.uk>                      
            <4A86949D.3090500@morningstar2.co.uk>                      
            <4A886BA8.2000209@morningstar2.co.uk>           
            <0417DF73-EC19-4262-B9DF-5C870D47BFCE@gmail.com>
            <4A893816.2090807@residenset.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Message-ID:  <4A89610D.8060108@morningstar2.co.uk>
Date:         Mon, 17 Aug 2009 14:54:21 +0100
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
Sender: Mailing list for the LaTeX3 project <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
From: Joseph Wright <joseph.wright@MORNINGSTAR2.CO.UK>
Subject: Re: xparse
To: LATEX-L@LISTSERV.UNI-HEIDELBERG.DE
In-Reply-To:  <4A893816.2090807@residenset.net>
Precedence: list
Status: R

Lars Hellström wrote:
> Haven't had time to review the code yet, I've only just downloaded it.
> Will get back to you when I have had a look at it. (I'm especially
> curious to how on earth you managed to do optional arguments at
> expand-time.)

That is done basically following etextools. For each argument, grab #1
and see (in an expandable way) if it is the same as the opening
delimiter. If it is, assume that it was the only thing grabbed and call
an auxiliary function to find the entire delimited argument. On the
other hand, if #1 is not the open-argument token, insert the appropriate
stuff and re-introduce #1 as the *next* thing to be processed.

All I've added to that is some argument re-arrangement so that there is
a chain of functions which do this and pass along previous arguments.
This uses the usual delimited argument method:

\cs_set:Npn { \token_to_str:N \foo <arg-number> }
  #1 \q_xparse_stop #2 {
  % #1 is previous argument
  % #2 is the current one to test
}

The auxiliary then calls the next function, re-inserting the marker and
the arguments to carry forward. The very last one is defined to remove
the \q_xparse_stop marker and actually call the code we're trying set
up. In that way, <code> is in a function which has the correct number of
arguments (\\foo, following the non-expandable pattern).

> Have latex'ed xparse.dtx, though, and it throws a few errors in the
> index. I think those are due to \xparse_add_type_>:, as the ">" is
> special to makeindex, forcing an extra index level where none was
> intended. Harmless character sequences avoids that trap. ;-)

Well, now that we have a mechanism available l3doc can be altered to use
it :-)

I'd missed this before as I'd not run makeindex on xparse. I've
corrected my batch file so this happens!

> In the meantime, here are some technical issues to consider.
> 
> 1. What happens if I do this:
> 
>   \DeclareDocumentCommand{\foo}{ +O{\par} m +O{\par} }{ ... }
> 
> ? One technical problem I saw in xdoc2l3 was that of simultaneously
> having \par tokens in defaults and non-long arguments.

I hope this should work.  The method I've adopted is to use two
functions to grab each argument. The first is \long, and gets hold of
the stuff remaining to be processed. It then sets up the real grabber,
so m args are grabbed something like:

\cs_new:Npn \xparse_grab_m:w #1 \l_xparse_args_toks {
  \cs_set_nopar:Nn \xparse_grab_m_aux:n {
    \toks_put_right:Nn \l_xparse_args_toks { {##1} }
    #1 \l_xparse_args_toks
  }
}

> 2. What happens (for the various argument types) when TeX fails to grab
> an argument? (Failures happen for long arguments when encountering a
> mismatched right brace, and for short argument also when encountering a
> \par token.) To keep in line with TeX, everything up to the ungrabbable
> token should disappear, but an uncareful implementation could leave
> other tokens in place as well, which would subject the user to
> mysterious additional errors.

The basic idea of xparse is not mine: ask the more senior team members!
Seriously, my tests suggest things are okay. As illustrated above, what
will happen is the aux function will complain, perhaps put something odd
onto the toks then get back to business with grabbing any more
arguments. Of course, if the problem is a run away long argument,
everything in the file will disappear and throw out to the TeX prompt.

Any suggestions for improvement very welcome, of course (or indeed any
bug reports: I've already fixed a couple while doing tests for this
e-mail!).
-- 
Joseph Wright