User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0)
            Gecko/20100101 Thunderbird/24.4.0
MIME-Version: 1.0
References: <533473A9.3020401@residenset.net>                      
            <53387AD3.4060902@morningstar2.co.uk>                      
            <533965E4.2060205@residenset.net>           
            <53398E5F.8020507@morningstar2.co.uk>
            <53399A18.9040401@residenset.net>
Content-Type: text/plain; charset=ISO-8859-1
Message-ID:  <5339D1AA.3020709@morningstar2.co.uk>
Date:         Mon, 31 Mar 2014 21:35:54 +0100
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
Sender: Mailing list for the LaTeX3 project <LATEX-L@LISTSERV.UNI-HEIDELBERG.DE>
From: Joseph Wright <joseph.wright@MORNINGSTAR2.CO.UK>
Subject: Re: harmless package public beta
To: LATEX-L@LISTSERV.UNI-HEIDELBERG.DE
In-Reply-To:  <53399A18.9040401@residenset.net>
Precedence: list
Content-Transfer-Encoding: quoted-printable
Envelope-To: <rainer.schoepf@GMX.NET>
Status: R

On 31/03/2014 17:38, Lars Hellstr=F6m wrote:
>>>> I'd also suggest the 'sheep and goats' separation of all commands in=
to
>>>> fully expandable or \protected.
>>>
>>> Not sure there could only be those two, but I can certainly change so=
me
>>> \newcommand's into \DeclareRobustCommand's.
>>
>> There are very few commands that don't seem to fall into one of the tw=
o
>> categories. (I've perhaps got one set for siunitx, but in a very unusu=
al
>> use case.) Most commands either:
>>
>>   - Should/can work inside \edef =3D>  expandable
>>   - Don't (assignments, ...) =3D>  \protected
>=20
> Well, I wonder how that would interact with \harmless_secure_expand:w.

Haven't read all of the code: just did an overview. The observation of
the team to date, which also seems to match with the ConTeXt experience,
is that a division does appear to work.

>> Note this is nothing to do with \DeclareRobustCommand, as those are no=
t
>> engine-robust.
>=20
> For the time being, I still support TeX 3.

Your decision: you do know e-TeX was finalised 15 years ago ;-)

>> A general impression, not least in that you've coded things by hand th=
at
>> would be done using expl3 kernel functions. The other very obvious one
>> is that we don't use toks :-)
>=20
> Not at all?! Or just not the likes of \newtoks?

At an interface level, no toks at all (there are a few internal special
cases). The reasoning is that toks are hard to explain: two types of
similar variable (macros and toks), except ... With e-TeX, we can say tha=
t:

  - Anything can be stored in a macro with

      \edef\foo{\unexpanded{<stuff>}}

 - Expansion can always be controlled with

      \unexpanded\expandafter{\macro}

Wrapping that up inside some interfaces, we don't need toks at all.

(Bruno uses toks by number for l3regex within a group structure, and of
course at a base layer there are some raw things to do with \every...
and so on. However, none of this shows up in the interfaces, which all
use macro-based storage of tokens.)


>>> You mean a token with character code 229 (U+00E5) is more canonical t=
han
>>> \r{a} as encoding of that character? That's actually close to the way
>>> harmless would do it (under the \HighCharsUnicode setting). For
>>> typesetting, there is conversely \UnicodeCharUsesChar.
>>
>> In that area, yes. Thinking is currently as follows: for UTF-8 work,
>> there are two engines which can do the job. Supporting non-UTF-8 input
>> for chars outside of the 8-bit range is something that 'new' code real=
ly
>> should avoid. That doesn't negate use of inputenc, etc., for 'real
>> world' cases now but does suggest that for new code we might want to
>> take a different take. For example, the current thinking is that for
>> 'text level' case-changing functions we will support only 'engine
>> native' input, which implies that at some stage we'd want to as you sa=
y
>> do mappings for UTF-8 engines going in the \r{a} =3D>  U+00E5 directio=
n.
>> (At a user level, things like \r{a} for a 'one-off' accent remain usef=
ul
>> even with a UTF-8 engine.)
>=20
> As a data-point here, I should perhaps mention that I have for some
> years been using the \r and \" accents even when I write swedish text i=
n
> LaTeX -- the reason being that I've remapped the =C5=D6=C4 keys to \{} =
since
> the latter are more frequently needed. And when you don't have a
> character conveniently available on the keyboard, it doesn't make that
> much difference that UTF-8 can encode it prefecty fine, if you cannot
> conveniently type it!

As I said, there is still discussion to have in this area and at the
user level things like \r{A} make sense. However, that's not the same as
saying they should be handled by e.g. case-changing functions, which
with a UTF-8 engine can deal with e.g. =C5 readily.
--
Joseph Wright