Skip to content

Commit 45ca7e3

Browse files
aphillipscatamorphism
aphillips
and
catamorphism
authored
[DESIGN] Maintaining the Standard, Optional and Unicode Namespace Function Sets (#634)
* Design doc to capture registry maintenance * Update maintaining-registry.md * Update exploration/maintaining-registry.md Co-authored-by: Tim Chevalier <[email protected]> * Update exploration/maintaining-registry.md Co-authored-by: Tim Chevalier <[email protected]> * Add user stories, small updates to RGI * Update exploration/maintaining-registry.md * Adding additional detail * Remove machine readable registry; update prose * Update maintaining-registry.md * Further development work * Update to change format and naming Per the 2024-08-19 call, we decided to switch towards a specification-per-function model, with statuses. This commit includes the initial set of changes to try and implement this. * Address some comments. --------- Co-authored-by: Tim Chevalier <[email protected]>
1 parent 749a502 commit 45ca7e3

File tree

1 file changed

+316
-0
lines changed

1 file changed

+316
-0
lines changed

‎exploration/maintaining-registry.md

Lines changed: 316 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,316 @@
1+
# Maintaining and Registering Functions
2+
3+
Status: **Proposed**
4+
5+
<details>
6+
<summary>Metadata</summary>
7+
<dl>
8+
<dt>Contributors</dt>
9+
<dd>@aphillips</dd>
10+
<dt>First proposed</dt>
11+
<dd>2024-02-12</dd>
12+
<dt>Pull Requests</dt>
13+
<dd>#634</dd>
14+
</dl>
15+
</details>
16+
17+
## Objective
18+
19+
_What is this proposal trying to achieve?_
20+
21+
Describe how to manage the registration of functions and options under the
22+
auspices of MessageFormat 2.0.
23+
This includes the Standard Functions which are normatively required by MF2.0,
24+
functions or options in the Unicode `u:` namespace,
25+
and functions/options that are recommended for interoperability.
26+
27+
## Background
28+
29+
_What context is helpful to understand this proposal?_
30+
31+
MessageFormat v2 originally included the concept of "function registries",
32+
including a "default function registry" required of conformant implementations.
33+
34+
The terms "registry" and "default registry" suggest machine-readbility
35+
and various relationships between function sets that the working group decided
36+
was not appropriate.
37+
38+
MessageFormat v2 includes a standard set of functions.
39+
Implementations are required to implement all of the _selectors_
40+
and _formatters_ in this set,
41+
including _operands_, _options_, and option values.
42+
Our goal is to be as universal as possible,
43+
making MFv2's message syntax available to developers in many different
44+
runtimes in a wholly consistent manner.
45+
Because we want broad adoption in many different programming environments
46+
and because the capabilities
47+
and functionality available in these environments vary widely,
48+
this standard set of functions must be conservative in its requirements
49+
such that every implementation can reasonably implement it.
50+
51+
Promoting message interoperability can and should go beyond this.
52+
Even when a given feature or function cannot be adopted by all platforms,
53+
diversity in the function names, operands, options, error behavior,
54+
and so forth remains undesirable.
55+
Another way to say this is that, ideally, there should be only one way to
56+
do a given formatting or selection operation in terms of the syntax of a message.
57+
58+
This suggests that there exist a set of functions and options that
59+
extends the standard set of functions.
60+
Such a set contains the "templates" for functions that go beyond those every implementation
61+
must provide or which contain additional, optional features (options, option values)
62+
that implementations can provide if they are motivated and capable of doing so.
63+
These specifications are normative for the functionality that they provide,
64+
but are optional for implementaters.
65+
66+
There also needs to be a mechanism and process by which functions in the default namespace
67+
can be incubated for future inclusion in either the standard set of functions
68+
or in this extended, optional set.
69+
70+
### Examples
71+
72+
_Function Incubation_
73+
74+
CLDR and ICU have defined locale data and formatting for personal names.
75+
This functionality is new in CLDR and ICU.
76+
Because it is new, few, if any, non-ICU implementations are currently prepared to implement
77+
a function such as a `:person` formatter or selector.
78+
Implementation and usage experience is limited in ICU.
79+
Where functionality is made available, we don't want it to vary from
80+
platform to platform.
81+
82+
_Option Incubation_
83+
84+
In the Tech Preview (LDML45) release, options for `:number` (and friends)
85+
and `:datetime` (and friends) were omitted, including `currency` for `:number`
86+
and `timeZone` for `:datetime`.
87+
The options and their values were reserved, possibly for the LDML46 release as required,
88+
but they also might be retained at a lower level of maturity.
89+
90+
## Use-Cases
91+
92+
_What use-cases do we see? Ideally, quote concrete examples._
93+
94+
As an implementer, I want to know what functions, options, and option values are
95+
required to claim support for MF2:
96+
- I want to know what options I am required to implement.
97+
- I want to know what the values of each option are.
98+
- I want to know what the options and their values mean.
99+
- I want to be able to implement all of the required functions using my runtime environment
100+
without difficulty.
101+
- I want to be able to use my local I18N APIs, which might use an older release of CLDR
102+
or might not be based on CLDR data at all.
103+
This could mean that my output might not match that of an CLDR-based implementation.
104+
105+
As an implementer, user, translator, or tools author I expect functions, options
106+
and option values to be stable.
107+
The meaning and use of these, once established, should never change.
108+
Messages that work today must work tomorrow.
109+
This doesn't mean that the output is stabilized or that selectors won't
110+
produce different results for a given input or locale.
111+
112+
As an implementer, I want to track best practices for newer I18N APIs
113+
(such as implementing personal name formatting/selection)
114+
without being required to implement any such APIs that I'm not ready for.
115+
116+
As an implementer, I want to be assured that functions or options added in the future
117+
will not conflict with functions or options that I have created for my local users.
118+
119+
As a developer, I want to be able to implement my own local functions or local options
120+
and be assured that these do not conflict with future additions by the core standard.
121+
122+
As a tools developer, I want to track both required and optional function development
123+
so that I can produce consistent support for messages that use these features.
124+
125+
As a translator, I want messages to be consistent in their meaning.
126+
I want functions and options to work consistently.
127+
I want to selection and formatting rules to be consistent so that I only have
128+
to learn them once and so that there are no local quirks.
129+
130+
As a user, I want to be able to use required functions and their options in my messages.
131+
I want to be able to quickly adopt new additions as my implementation supports them
132+
or be able to choose plug-in or shim implementations.
133+
I never want to have to rewrite a message because a function or its options have changed.
134+
135+
As an implementer or user, I want to be able to suggest useful additions to MF2 functionality
136+
so that users can benefit from consistent, standardized features.
137+
I want to understand the status of my proposal (and those of others) and know that a public,
138+
structured, well-managed process has been applied.
139+
140+
## Requirements
141+
142+
_What properties does the solution have to manifest to enable the use-cases above?_
143+
144+
The Standard Function Set needs to describe the minimum set of selectors and formatters
145+
needed to create messages effectively.
146+
This must be compatible with ICU MessageFormat 1 messages.
147+
148+
There must be a clear process for the creation of new selectors that are required
149+
by the Standard Function Set,
150+
which includes a maturation process that permits implementer feedback.
151+
152+
There must be a clear process for the creation of new formatters that are required
153+
by the Standard Function Set,
154+
which includes a maturation process that permits implementer feedback.
155+
156+
There must be a clear process for the addition of options or option values that are required
157+
by the Standard Function Set,
158+
which includes a maturation process that permits implementer feedback.
159+
160+
There must be a clear process for the deprecation of any functions, options, or option values
161+
that are no longer I18N best practices.
162+
The stability guarantees of our standard do not permit removal of any of these.
163+
164+
## Constraints
165+
166+
_What prior decisions and existing conditions limit the possible design?_
167+
168+
## Proposed Design
169+
170+
_Describe the proposed solution. Consider syntax, formatting, errors, registry, tooling, interchange._
171+
172+
The MessageFormat WG will release a set of specifications
173+
that standardize the implementation of functions and options in the default namespace of
174+
MessageFormat v2 beginning with the LDML46 release.
175+
Implementations and users are strongly discouraged from defining
176+
their own functions or options that use the default namespace
177+
Future updates to these sets of functions and options will coincide with LDML releases.
178+
179+
Each _function_ is described by a single specification document.
180+
Each such document will use a common template.
181+
A _function_ can be a _formatting function_,
182+
a _selector_,
183+
or both.
184+
185+
The specification will indicate if the _formatting function_,
186+
the _selector function_, or, where applicable, both are `Standard` or `Optional`.
187+
The specification must describe operands, including literal representations.
188+
189+
The specification includes all defined _options_ for the function.
190+
Each _option_ must define which values it accepts.
191+
An _option_ is either `Standard` or `Optional`.
192+
193+
_Functions_ or _options_ that have an `Optional` status
194+
must have a maturity level assigned.
195+
The maturity levels are:
196+
- **Proposed**
197+
- **Accepted**
198+
- **Released**
199+
- **Deprecated**
200+
201+
_Functions_ and _options_ that have a `Standard` status have only the
202+
`Released` and `Deprecated` statuses.
203+
204+
* An _option_ can be `Standard` for an `Optional` function.
205+
This means that the function is optional to implement, but that, when implemented, must include the option.
206+
* An _option_ can be `Optional` for a `Standard` function.
207+
This means that the function is required, but implementations are not required to implement the option.
208+
* An _option_ can be `Optional` for an `Optional` function.
209+
This means that the function is optional to implement and the option is optional when implementing the function.
210+
211+
A function specification describes the functions _operand_ or _operands_,
212+
its formatting options (if any) and their values,
213+
its selection options (if any) and their values,
214+
its formatting behavior (if any),
215+
its selection behavior (if any),
216+
and its resolved value behavior.
217+
218+
`Standard` functions are stable and subject to stability guarantees.
219+
Such entries will be limited in scope to functions that can reasonably be
220+
implemented in nearly any programming environment.
221+
> Examples: `:string`, `:number`, `:datetime`, `:date`, `:time`
222+
223+
224+
`Optional` functions are stable and subject to stability guarantees once they
225+
reach the status of **Released**.
226+
Implmentations are not required to implement _functions_ or _options_ with an `Optional` status
227+
when claiming MF2 conformance.
228+
Implementations MUST NOT implement functions or options that conflict with `Optional` functions or options.
229+
230+
`Optional` values may have their status changed to `Standard`,
231+
but not vice-versa.
232+
233+
> Option Examples `:datetime` might have a `timezone` option in LDML46.
234+
> Function Examples: We don't currently have any, but potential work here
235+
> might includes personal name formatting, gender-based selectors, etc.
236+
237+
The CLDR-TC reserves the `u:` namespace for use by the Unicode Consortium.
238+
This namespace can contain _functions_ or _options_.
239+
Implementations are not required to implement these _functions_ or _options_
240+
and may adopt or ignore them at their discretion,
241+
but are encouraged to implement these items.
242+
243+
Items in the Unicode Reserved Namespace are stable and subject to stability guarantees.
244+
This namespace might sometimes be used to incubate functionality before
245+
promotion to the default namespace in a future release.
246+
In such cases, the `u:` namespace version is retained, but deprecated.
247+
> Examples: Number and date skeletons are an example of Unicode extension
248+
> possibilities.
249+
> Providing a well-documented shorthand to augment "option bags" is
250+
> popular with some developers,
251+
> but it is not universally available and could represent a barrier to adoption
252+
> if normatively required.
253+
254+
All `Standard`, `Optional`, and Unicode namespace function or option specifications goes through
255+
a development process that includes these levels of maturity:
256+
257+
1. **Proposed** The _function_ or _option_, along with necessary documentation,
258+
has been proposed for inclusion in a future release.
259+
2. **Accepted** The _function_ or _option_ has been accepted but is not yet released.
260+
During this period, changes can still be made.
261+
3. **Released** The _function_ or _option_ is accepted as of a given LDML release that MUST be specified.
262+
4. **Deprecated** The _function_ or _option_ was previously _released_ but has been deprecated.
263+
Implementations are still required to support `Standard` functions or options that are deprecated.
264+
5. **Rejected** The _function_ or _option_ was considered and rejected by the MF2 WG and/or the CLDR-TC.
265+
Such items are not part of any standard, but might be maintained for historical reference.
266+
267+
A proposal can seek to modify an existing function.
268+
For example, if a _function_ `:foo` were an `Optional` function in the LDMLxx release,
269+
a proposal to add an _option_ `bar` to this function would take the form
270+
of a proposal to alter the existing specification of `:foo`.
271+
Multiple proposals can exist for a given _function_ or _option_.
272+
273+
### Process
274+
275+
Proposals for additions are made via pull requests in a unicode-org github repo
276+
using a specific template TBD.
277+
Proposals for changes are made via pull requests in a unicode-org github repo
278+
using a specific template TBD against the existing specification for the function or option.
279+
280+
Proposals must be made at least _x months_ prior to the release date to be included
281+
in a specific LDML release.
282+
The CLDR-TC will consider each proposal using _process details here_ and make a determination.
283+
The CLDR-TC may delegate approval to the MF2 WG.
284+
Decisions by the MF2 WG may be appealed to the CLDR-TC.
285+
Decisions by the CLDR-TC may be appealed using _existing process_.
286+
287+
Technical discussion during the approval process is strongly encouraged.
288+
Changes to the proposal,
289+
such as in response to comments or implementation experience, are permitted
290+
until the proposal has been approved.
291+
Once approved, changes require re-approval (how?)
292+
293+
294+
The timing of official releases of the Standard Function Set and Optional Set is the same as CLDR/LDML.
295+
Each LDML release will include:
296+
- **Released** specifications in the Standard Function Set
297+
- **Released** specifications in the Unicode reserved namespace
298+
- a section of the MF2 specification specifically incorporating versions of the above
299+
- **Accepted** entries for each of the above available for testing and feedback
300+
301+
Proposals for additions to any of the above include the following:
302+
- a design document, which MUST contain:
303+
- the exact text to include in the MF2 specification using a template to be named later
304+
305+
Each proposal is stored in a directory indicating indicating its maturity level.
306+
The maturity levels are:
307+
- **Accepted** Items waiting for the next CLDR release.
308+
- **Released** Complete designs that are released.
309+
- **Proposed** Proposals that have not yet been considered by the MFWG or which are under active development.
310+
- **Rejected** Proposals that have been rejected by the MFWG in the past.
311+
312+
## Alternatives Considered
313+
314+
_What other solutions are available?_
315+
_How do they compare against the requirements?_
316+
_What other properties they have?_

0 commit comments

Comments
 (0)