User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-GB; rv:1.9.2.9)
            Gecko/20100915 Thunderbird/3.1.4
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Message-ID:  <4C990EE9.9050905@morningstar2.co.uk>
Date:         Tue, 21 Sep 2010 21:00:41 +0100
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@listserv.uni-heidelberg.de>
Sender: Mailing list for the LaTeX3 project <LATEX-L@listserv.uni-heidelberg.de>
From: Joseph Wright <joseph.wright@MORNINGSTAR2.CO.UK>
Subject: LaTeX3 engine requirements and \tl_if_eq:nn(TF)
To: LATEX-L@listserv.uni-heidelberg.de
Precedence: list
Status: R

Hello all,

Prompted in part by the issues raised by category code tables in LuaTeX, 
the team have been considering engine variations in LaTeX3/expl3.  While 
expl3 is 'stable', it is important that problems continue to be 
addressed when they are identified. This includes revising existing 
ideas, functions and implementations when issues arise.

\pdfstrcmp
----------

In particular, the code provided a family of functions \tl_if_eq:nn(TF). 
With a sufficiently new engine, this family was implemented using the 
\pdfstrcmp primitive in pdfTeX, the equivalent \strcmp primitive in 
XeTeX, or using Lua when loaded with LuaTeX. Under these conditions, 
\tl_if_eq:nn(TF) was expandable. However, with older pdfTeX and XeTeX 
versions the code instead used a different implementation which was not 
expandable.

The danger with this approach was that other functions are also built on 
top of the basic comparison, for example \IfNoValueTF from xparse. The 
possibility of unexpected errors when moving from an expandable to 
non-expandable version is clear, particularly in the context of the 
category code table issue already mentioned.

Engine requirements
-------------------

The team has therefore looked again at what is desirable here. The 
\pdfstrcmp primitive was introduced in pdfTeX 1.30 in 2005. It is 
therefore very likely that users who want to use expl3 will have 
\pdfstrcmp available when using pdfTeX. XeTeX has only had the primitive 
since version 0.9994, meaning from TeX Live 2009 onward. For LuaTeX, the 
very active development means that there are many reasons to expect 
users to make use of a recent release.

Based on the above, the team have decided to require the availability of 
\pdfstrcmp (or equivalent) for loading expl3. This is clearly a 
significant change sense, although hopefully will not cause issues for 
most expl3 users. As a result, it will only work with engines:

  - pdfTeX 1.30 or later
  - XeTeX 0.9994 or later
  - LuaTeX 0.40 or later (due to the syntax of \directlua), although
    only v0.60 or later is used for testing.

String versus token list comparisons
------------------------------------

There is a second issue with \tl_if_eq:nn(TF) as previously provided. 
The function is '\tl...', and so should compare on a token by token 
basis. However, the implement compared as strings (i.e. considering 
character codes only). This was clearly misleading. As part of the 
update in this area, the following have now been implemented:

  - \str_if_eq:nn(TF), a family of expandable comparisons which
    work on a purely character basis. These are implemented using
    \pdfstrcmp internally. This includes \str_if_eq:xx(TF), which
    remains expandable as \pdfstrcmp performs full expansion while
    itself being expandable.

  - \tl_if_eq:nn(TF), a family of unexpandable comparison which
    work on a token basis. These are implemented by storing the input
    and using an \ifx comparison internally.

The result is that the behaviour of \tl_if_eq:nn(TF) is altered, and 
that the expansion variants provided will change. In many cases 
programmers will want to alter existing uses of \tl_if_eq:nn(TF) to use 
the new \str_if_eq:nn(TF) functions.

There will be a CTAN update of expl3 and xpackages soon (by the weekend, 
I hope). This will include these changes (along with others: see my 
other e-mails).
-- 
Joseph Wright