User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:12.0)
            Gecko/20120428 Thunderbird/12.0.1
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Message-ID:  <4FCE2B42.2090006@morningstar2.co.uk>
Date:         Tue, 5 Jun 2012 16:52:34 +0100
Reply-To: Mailing list for the LaTeX3 project
              <LATEX-L@listserv.uni-heidelberg.de>
Sender: Mailing list for the LaTeX3 project <LATEX-L@listserv.uni-heidelberg.de>
From: Joseph Wright <joseph.wright@MORNINGSTAR2.CO.UK>
Subject: Separating out 'public' and 'internal' functions/variables in LaTeX3 modules
To: LATEX-L@listserv.uni-heidelberg.de
Precedence: list
Status: R

*Introduction*

One of the aims of ongoing work on LaTeX3 is to enable a much clearer
separation of different 'layers' of the system. The idea of separating
out 'user' commands, 'design' commands and 'code' commands is one which
is already present to some extent in LaTeX2e. The use of "@" macros in
LaTeX2e provides a separate 'code' space, but without sufficiently clear
separation from the 'user' level (as many internal macros are tied to
the user-level syntax). The xparse module provides a method to deal with
the separation of 'user' and 'code' functions, while the 'expl3' syntax
is more formalised than LaTeX2e at the 'code' level

However, within the 'code' layer there is actually further subdivision.
Some functions and variables are intended to act as the user interface
for a module. These should be documented with some fixed and reliable
syntax/semantics. At the same time, there are functions and variables
which are internal to a module. These are necessary for the
implementation of functionality but may be varied by the programmer. At
the same time, they should not be used directly by 'others' (ideally not
even by the same programmer from a separate module).

Using 'internal' material to alter behaviour is something which is
common in LaTeX2e programming. However, it means that packages come to
rely on the implementation detail of other packages, making changes over
time difficult or even impossible. Documentation of public code-level
functions/variables is the approach that has been taken to date by the
team to address this issue. That works best where internal code is
clearly internal ("\foo_function_aux:"), but less well when the internal
functions have apparently 'usable' names.

*An additional syntax convention*

What the team feel may be needed is a way to enable LaTeX3 programmers
to see which functions they are 'allowed' to use from other modules with
having to refer at all stages to the documentation. That does not of
course mean that the documentation should not be written, but that we
are looking for additional steps and support for good practice.

Marking material as 'internal' to a module can be done by picking some
naming convention which shows this is the case. The team have discussed
a number of possible approaches, bearing in mind the fact that LaTeX3
code is in use and that it is therefore important not to make any
breaking changes in this regard. The convention we propose is use two
"_" at the beginning of the module to indicate internal material:

  % Public
  \foo_function:nn
  \l_foo_variable_tl

  % Internal
  \__foo_function:nn
  \l__foo_variable_tl

*Extending DocStrip*

The 'two underscore' convention can of course be used directly in code.
However, as it extends the name of every code-level name, this may not
be desirable. To support this further, we have also explored the idea of
extending DocStrip such that a 'marker' can be used for internal
material, such that this can be replaced by the "__foo" part.

The approach that we have taken for this is to use "@@" as the indicator
for internal code:

  \@@_function:nn
  \l_@@_variable_tl

To tell the extended l3docstrip program what is the appropriate
replacement for "@@", an additional guard-like line is also required:

  %<@@=foo>
  \@@_function:nn   % Will be converted to \__foo_function:nn
  \l_@@_variable_tl % Will be converted to \l__foo_variable_tl

  %<@@=bar>
  \@@_function:nn % Will be converted to \__bar_function:nn

  %<@@=>
  \@@_function:nn % Will be extracted unchanged

(It is important to note that this 'extended DocStrip' idea additional
to the idea of an 'internal' syntax.)

*Feedback*

At this stage, we have not implemented any of these ideas in the main
development code. We have added some 'example' material to the
development system, making copies of several LaTeX3 modules as
demonstrations. These are available from

  https://github.com/latex3/svn-mirror/tree/master/l3trial/l3docstrip

If a separation of internal and public functions outlined here is to be
implemented, it is important that there is general agreement on the
approach, as a mixture of styles is likely to be unsuccessful. We
therefore need to know how others see this: do the suggestions make
sense, does the detail work for other people and so on.

It is important to notice that as we are dealing purely with internal
code: any change here should not require adjustment of release code from
others.
--
Joseph Wright