References: <5701B0C1.9080407@free.fr>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101
            Thunderbird/38.7.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Message-ID:  <57022D60.6050803@morningstar2.co.uk>
Date:         Mon, 4 Apr 2016 10:01:20 +0100
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
Sender: Mailing list for the LaTeX3 project <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
From: Joseph Wright <joseph.wright@MORNINGSTAR2.CO.UK>
Subject: Re: Bug in xparse r, or R, or d or D argument specifiers
To: LATEX-L@LISTSERV.UNI-HEIDELBERG.DE
In-Reply-To:  <5701B0C1.9080407@free.fr>
Precedence: list
Envelope-To: <rainer.schoepf@GMX.NET>
Status: R

On 04/04/2016 01:09, Julien RIVAUD (_FrnchFrgg_) wrote:
> Hello,
> 
> A command created with
> 
> \NewDocumentCommand \foo { r\OPEN\CLOSE } { <code> }
> 
> seems to work fine, and grabs correctly its argument (even when there is
> nesting involved), but leaves OPEN at the beginning of the argument. The
> problem is in \__xparse_grab_D_aux:NNnN where you do:
> 
>  \token_if_eq_catcode:NNTF + #1
>  {
>  <snip>
>  }
>  {
>    \exp_after:wN \l__xparse_fn_tl
>    \token_to_str:N #1
>  }
> 
> But of course \token_to_str:N #1 is several characters long. Suppose now
> that there was no nesting, i.e. we used \foo\OPEN hello\CLOSE.
> 
> Then \l__xparse_fn_tl goes to
> 
>  \tl_if_blank:oTF { \use_none:n ####1 }
>    { \__xparse_add_arg:o { \use_none:n ####1 } }
>    {
>      \str_if_eq_x:nnTF
>        { \exp_not:o { \use_none:n ####1 } }
>        { { \exp_not:o { \use_ii:nnn ####1 \q_nil } } }
>        { \__xparse_add_arg:o { \use_ii:nn ####1 } }
>        { \__xparse_add_arg:o { \use_none:n ####1 } }
>    }
>  #3 \l__xparse_args_tl
> 
> where the blank test cannot succeed (####1 is \OPEN hello where \ is a
> letter so \use_none:n will only gobble that), and the \str_if_eq_x is
> also false, because
> 
> \use_none:n ####1 gives OPEN hello (all catcode 12)
> 
> and
> 
> \use_ii:nnn ####1 \q_nil gives OEN hello\q_nil (all catcode 12)

The intention of d/D/r/R was always as a generalisation of LaTeX2e's
optional arg/co-ordinate arg syntax. As such, my idea at least was that
the are intended for handling single character tokens (also true for
t-type args, etc.). Certainly xparse is meant to provide 'LaTeX2e-like
interfaces' not 'a completely general argument parser', so it's not
unreasonable to have a restriction here. I'll look to tighten up the
docs but will first wait to see if there are strong views that the
behaviour should change.

> As a side note, I don't understand what this test is about: it can be
> true only when ####1 is two tokens, so only one token, and I didn't find
> a situation where \use_none:n didn't cut it (I tried loosing braces
> prevention, or loosing space prevention, I never made the test return
> true *and* the \use_none:n call be wrong). Of course here it doesn't
> matter, the many letters of the opening token make these fine mechanics
> go awry.

This was added by me in r4462
(https://github.com/latex3/latex3/commit/5aa6ab78f1e035abd73daf41a92b30501bcd27a4).
As noted there, the case we are worrying about is the behaviour of the
two uses

    \foo[{bar}]{baz}
    \foo[bar]{baz}

The first use is what is currently required with LaTeX2e if the optional
argument itself contains something with an optional argument, which
xparse handles by looking correctly for nesting. However, xparse needs
to work in a 'mixed' environment and it is also not unreasonable that
the two are parsed in the same way. That's what the more complicated
test is all about.

> [1] iirc, it was to make enumitem overlay-aware in beamer while
> respecting the already existing syntax. So
> \begin{enumerate}[<overlay>][enumitem options], where of course any of
> them can be present (in that order only according to what beamer does
> for its enumerate). I had to resort to grabbing the first optional
> argument, check if it began with <, and if so remove the trailing > and
> grab another optional argument. A bit ugly, whereas
> 
> \NewDocumentEnvironment {enumerate} { d{[<}{>]} o } would have been
> beautifully clean.

Till's interface choice here is perhaps a bit unusual but that doesn't
mean xparse has to handle it directly. (As noted above, that's not
really what xparse is about.)

> P.S.: I also encountered a "quirk" in \NewDocumentEnvironment, in that
> the environment close is not align-safe (probably because of the
> retrieving of arguments, but I didn't check; or just because it is
> protected?): it starts a new row (and that's the least worry because
> then the closing stuff is in a box group and TeX frowns upon the
> \endgroup of \end). I didn't need closing args, so I just used
> \NewDocumentCommand.

This looks like it needs a separate post with an example!

Joseph