User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0)
            Gecko/20120713 Thunderbird/14.0
MIME-Version: 1.0
References: <CAFUtaNGNykpg-mguMyiBv5kaCHBV9CsTYkt2EOeVg9JfJ0kPZw@mail.gmail.com> <5022BFB4.1090800@latex-project.org>  
            <CAFUtaNHy+gYuBCJ0OaXv7CCu6TR+Ht+kqah6ga7eFrnPnqdS=Q@mail.gmail.com>
            <CANQYN6xEos-Rqcuhp6=PwiHncshFnDOLqKEFC6qK9_FUUqOzWA@mail.gmail.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Message-ID:  <502428CD.1050208@morningstar2.co.uk>
Date:         Thu, 9 Aug 2012 22:17:01 +0100
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@listserv.uni-heidelberg.de>
Sender: Mailing list for the LaTeX3 project <LATEX-L@listserv.uni-heidelberg.de>
From: Joseph Wright <joseph.wright@MORNINGSTAR2.CO.UK>
Subject: Re: Examples of l3doc & unit testing?
To: LATEX-L@listserv.uni-heidelberg.de
In-Reply-To:  <CANQYN6xEos-Rqcuhp6=PwiHncshFnDOLqKEFC6qK9_FUUqOzWA@mail.gmail.com>
Precedence: list
Status: R

On 09/08/2012 18:30, Bruno Le Floch wrote:
> We run our test suite using a Makefile (or make.bat on Windows), which
> - calls the appropriate TeX engine the appropriate number of times, then
> - calls a Perl script to remove paths and other parts of the log file
> specific to a given installation,
> - calls diff to compare the result with a saved result.
> The drawbacks are that it is os-dependent, it uses perl, which may not
> be installed everywhere.
> 
> It should be possible to use a texlua script in place of the Makefile,
> either LPeg on the Lua side or l3regex on the TeX side to strip the
> log file, then comparing again either on the Lua or TeX side.  Would
> it be useful?  Is copying the current test system the right approach?
I have certainly wondered about replacing the Perl script with a Lua
one, but I am not 100% that makes things more accessible (if you look
beyond people with a 'recent' TeX distro, Perl is likely to be more, not
less, 'available' than Lua). It would certainly be handy to remove the
need to use the system file comparison tool at the end of the process,
as "diff" and "fc" give rather different output. It would also be good
to have everything available in one 'package'.

The current LaTeX3 test system works pretty well, provided the tests are
written to actually test behaviour correctly :-) Checking log file info
seems to work well both for 'programmatic' information (writing the
result to the log), and for 'typesetting' (by using \box_show:N to again
write to the log). So it does not seem like a bad model. It might be
worth looking at the TeX part of the process (the test .tex file) to see
if it needs any tidying up to be more generally usable.

Of course, if you do want to look at this then it would also be worth
looking at the alternatives (e.g. qstest).
--
Joseph Wright