MIME-Version: 1.0
References: <CAFUtaNGNykpg-mguMyiBv5kaCHBV9CsTYkt2EOeVg9JfJ0kPZw@mail.gmail.com> <5022BFB4.1090800@latex-project.org>
            <CAFUtaNHy+gYuBCJ0OaXv7CCu6TR+Ht+kqah6ga7eFrnPnqdS=Q@mail.gmail.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit
Message-ID:  <CANQYN6xEos-Rqcuhp6=PwiHncshFnDOLqKEFC6qK9_FUUqOzWA@mail.gmail.com>
Date:         Thu, 9 Aug 2012 19:30:16 +0200
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@listserv.uni-heidelberg.de>
Sender: Mailing list for the LaTeX3 project <LATEX-L@listserv.uni-heidelberg.de>
From: Bruno Le Floch <blflatex@GMAIL.COM>
Subject: Re: Examples of l3doc & unit testing?
To: LATEX-L@listserv.uni-heidelberg.de
In-Reply-To:  <CAFUtaNHy+gYuBCJ0OaXv7CCu6TR+Ht+kqah6ga7eFrnPnqdS=Q@mail.gmail.com>
Precedence: list
Status: R

On 8/8/12, Joel C. Salomon <joelcsalomon@gmail.com> wrote:
> On Wed, Aug 8, 2012 at 3:36 PM, Frank Mittelbach
> <frank.mittelbach@latex-project.org> wrote:
>> Am 08.08.2012 19:34, schrieb Joel C. Salomon:
>>
>>> Is there a simple example somewhere of the use of l3doc’s \TestFiles,
>>> \UnitTested, & \TestMissing commands? What sort of unit-tests are
>>> these meant to document?
>>>
>>> (Or are these just meant to be dumb pointers to arbitrary kinds of
>>> unit testing?)
>>
>> they are meant to document how much testing has been written in the
>> regression test suite. The idea being that each interface function gets
>> tested on standard and boundary cases (in the ./testfiles directories).
>>
>> but nothing scientific about really. It always depended on how well
>> people
>> wrote and write tests. Now at some point we did some attempt to sort out
>> where we have tests and where we are still missing them.
>>
>> There is some support in the make file for this "make checktest"
>>
>> but we didn't quite got this properly going (yet)
>
> And on Wed, Aug 8, 2012 at 3:54 PM, Joseph Wright
> <joseph.wright@morningstar2.co.uk> wrote:
>> Frank has covered what the idea is here. To see the tests themselves,
>> take a look at the GitHub site: the tests don't go to CTAN.
>
> Thank you both. I'll have to see whether & how to adapt that model to
> my own programming style.
>
> And since there's nothing l3-specific going on, I can get tips on
> writing effective unit tests elsewhere (read: on TeX.SX) without
> distracting this list. ;-)
>
> (There shouldn't be any funny interactions between l3doc and qstest,
> right?)

I've been thinking of writing an xtest (or l3test) for some time now,
to cater for my own package's need for testing.

We run our test suite using a Makefile (or make.bat on Windows), which
- calls the appropriate TeX engine the appropriate number of times, then
- calls a Perl script to remove paths and other parts of the log file
specific to a given installation,
- calls diff to compare the result with a saved result.
The drawbacks are that it is os-dependent, it uses perl, which may not
be installed everywhere.

It should be possible to use a texlua script in place of the Makefile,
either LPeg on the Lua side or l3regex on the TeX side to strip the
log file, then comparing again either on the Lua or TeX side.  Would
it be useful?  Is copying the current test system the right approach?

Regards,
Bruno