Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Message-ID:  <8422310776248060.WA.kellysmith12.21gmail.com@listserv.uni-heidelberg.de>
Date:         Mon, 16 Mar 2020 23:54:12 +0100
Reply-To:     Mailing list for the LaTeX3 project
              <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
Sender:       Mailing list for the LaTeX3 project <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
From:         Kelly Smith <kellysmith12.21@GMAIL.COM>
Subject: Re: Using Lua to Preprocess Unicode Data
To:           LATEX-L@LISTSERV.UNI-HEIDELBERG.DE
Precedence: list
Envelope-To: <rainer.schoepf@gmx.net>
Content-Transfer-Encoding: 8bit
Status: R

>As format-building is all about saving time for 'normal' runs, I'm not
>seeing there is a massive need to speed up the process. I know there is
>one engine in development that doesn't use format files, so that might
>be a place to consider things, but I think we'd need a strong case to
>alter the approach for XeTeX/LuaTeX (pdfTeX, ...).
>
>Joseph

Sorry, I should’ve clarified: the point of preprocessing the data wouldn’t
be to speed up anything, instead, the point would be to do complex
processing that would be very difficult or even impossible in LaTeX.

For example, if the l3regex module were extended so that precompiled
regexes could be used as parts of other regexes, then Unicode properties
could be simply implemented by referring to precompiled regexes whose
content was created by running filters over the Unicode character database.

Another example would be processing the very complex XML files that are
used in supplementary Unicode files, like the Common Locale Data Repository,
which could help with localization and language-specific date and number
parsing/formatting.

This idea of preprocessing could be applied to any complex data set that
LaTeX3 may need to work with, but I used the example of Unicode data
because that’s the one that immediately came to mind.