From d850cf49ac095cbb9afa4009b2d7a1136887b74b Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 13 Nov 2023 12:11:35 -0800 Subject: [PATCH 01/39] Implementing namespacing --- spec/message.abnf | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/spec/message.abnf b/spec/message.abnf index bd340fe9f2..e6c620aac7 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -22,7 +22,7 @@ function-expression = "{" [s] annotation [s] "}" annotation = (function *(s option)) / reserved / private-use literal = quoted / unquoted -variable = "$" name +variable = "$" name-body function = (":" / "+" / "-") name option = name [s] "=" [s] (literal / variable) @@ -70,9 +70,11 @@ reserved-char = %x00-08 ; omit HTAB and LF / %x7E-D7FF ; omit surrogates / %xE000-10FFFF -; based on https://www.w3.org/TR/xml/#NT-Name, -; but cannot start with U+003A COLON ":" -name = name-start *name-char +; based on https://www.w3.org/TR/REC-xml-names/#NT-QName +name = [namespace] name-body +namespace = name-start *name-char namespace-sep +namespace-sep = ":" +name-body = name-start *name-char name-start = ALPHA / "_" / %xC0-D6 / %xD8-F6 / %xF8-2FF / %x370-37D / %x37F-1FFF / %x200C-200D From 1cf7f3952b1ab911cf509096de624fbf24fa32b8 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 13 Nov 2023 12:27:37 -0800 Subject: [PATCH 02/39] Update syntax.md --- spec/syntax.md | 39 ++++++++++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/spec/syntax.md b/spec/syntax.md index 1b68da8e60..0361219f93 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -677,16 +677,45 @@ unquoted-start = name-start / DIGIT / "." A **_name_** is an identifier for a _variable_ (prefixed with `$`), for a _function_ (prefixed with `:`, `+` or `-`), or for an _option_ (these have no prefix). -The namespace for _names_ is based on XML's [Name](https://www.w3.org/TR/xml/#NT-Name), -with the restriction that it MUST NOT start with `:`, -as that would conflict with the _function_ start character. +The namespace for _names_ is based on Namespaces in XML 1.0's +[NCName](https://www.w3.org/TR/xml-names/#NT-NCName). +This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name) +in that it MUST NOT start with `:`. Otherwise, the set of characters allowed in names is large. +_Functions_ and _options_ may be preceded by a _namespace_ identifier +which is separated from the body of the _name_ by a `: U+003A COLON`. +Built-in _functions_ and _options_ do not have a _namespace_ identifier. + +Examples: +> A variable: +>``` +>This has a {$variable} +>``` +>A function: +> ``` +> This has a {:function} +> ``` +> An add-on function from the `icu` namespace: +> ``` +> This has a {:icu:function} +> ``` +> An option and an add-on option: +> ``` +> This has {:options option=value icu:option=add_on} +> ``` + +Support for _namespaces_ and their interpretation is implementation-defined +in this release. + ```abnf -variable = "$" name +variable = "$" name-body function = (":" / "+" / "-") name -name = name-start *name-char +name = [namespace] name-body +namespace = name-start *name-char namespace-sep +namespace-sep = ":" +name-body = name-start *name-char name-start = ALPHA / "_" / %xC0-D6 / %xD8-F6 / %xF8-2FF / %x370-37D / %x37F-1FFF / %x200C-200D From 30cdac6486488b46b37bd077cfa5a23936ab7a8b Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 09:34:12 -0800 Subject: [PATCH 03/39] Update formatting.md --- spec/formatting.md | 68 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 59 insertions(+), 9 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 85ea0288d1..621a07841d 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -197,17 +197,19 @@ the following steps are taken: 1. If the _expression_ includes an _operand_, resolve its value. If this fails, use a _fallback value_ for the _expression_. -2. Based on the _function_ starting sigil and _name_, +2. _Resolve the name_ of the _function_ and, based on the starting sigil, find the appropriate function implementation from the _function registry_. If the registry does not define an implementation for this _name_, emit an Unknown Function error and use a _fallback value_ for the _expression_. -3. Resolve the _option_ values to a mapping of string identifiers to values. +4. If the _expression_ includes _options_, resolve the _options_ to a mapping + of string identifiers to values. For each _option_: + - _Resolve the name_ of the option. - If its right-hand side successfully resolves to a value, bind the _name_ of the _option_ to the resolved value in the mapping. - Otherwise, do not bind the _name_ of the _option_ to any value in the mapping. -4. Call the function implementation with the following arguments: +6. Call the function implementation with the following arguments: - The current _locale_. - The resolved mapping of _options_. @@ -218,16 +220,64 @@ the following steps are taken: An implementation MAY pass additional arguments to the function, as long as reasonable precautions are taken to keep the function interface simple and minimal, and avoid introducing potential security vulnerabilities. - - As implementations MAY allow custom functions to be defined by users, - their access to the _formatting context_ SHOULD be minimal and read-only, - and their execution time SHOULD be limited. - -5. If the call succeeds, + Implementations SHOULD use an implementation-defined _namesapce_ for + any additional arguments exposed to users as _options_. + + An implementations MAY define its own functions or allow custom functions + to be defined by users. + Function access to the _formatting context_ SHOULD be minimal and read-only, + and execution time SHOULD be limited. + Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_. + User-defined _functions_ SHOULD be permitted to define the _namespace_ to use + in name resolution. + +8. If the call succeeds, resolve the value of the _expression_ as the result of that function call. If the call fails or does not return a valid value, emit a Resolution error and use a _fallback value_ for the _expression_. +### Name Resolution + +**_Name resolution_** is the determination of +the _name_ and the _namespace_ of the name for a given _function_ +or _option_. + +To determine the _name_ of an item, the following steps are taken: + +1. Let the _name_ be the string to be processed, less any preceeding + sigils or markers. + For example, in the _expression_ `{:function}`, the string `function` + is the _name_ string. +2. Determine if _name_ contains the character U+003A COLON `:`: + - If _name_ contains a colon, let _namespace_ be the + substring of _name_ up to (but not including) + the colon and _name_ be the substring of _name_ following the colon; + - Else let _namespace_ be an empty value (null, void, or empty string). +4. If the _name_ is empty, return a Syntax error. +5. Return _namespace_ and _name_. + +Implementations are not required to support the installation or resolution +of different namespaces. +How namespaces are defined, provisioned, or handled is also implementation defined. +It is important to note that the _namespace_ is not considered part of the _name_ +after resolution. +The _namespace_ MAY affect which function is called. + +Example: +> Suppose the ICU implementation of MessageFormat added a custom option `skeleton` +> to the `:datetime` function. +> Users of that implementation could invoke the datetime formatter with an +> expression similar to: +> ``` +> {{Today is {$date :datetime icu:skeleton=yMMMd}.}} +> ``` +> The `name` of the `datetime` function is resolved as: +> > `namespace`: (blank) +> > `name`: `datetime` +> The `name` of the option in the expression is resolved as: +> > `namespace`: `icu` +> > `name`: `skelelton` + ### Fallback Resolution A **_fallback value_** is the resolved value emitted when an _expression_ cannot be resolved. From d875dd3c29afe6c3847c46bd8a57be669575421b Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 09:39:14 -0800 Subject: [PATCH 04/39] Fix one instance of normative language --- spec/syntax.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/syntax.md b/spec/syntax.md index 0361219f93..a0bb0111fc 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -683,8 +683,8 @@ This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name) in that it MUST NOT start with `:`. Otherwise, the set of characters allowed in names is large. -_Functions_ and _options_ may be preceded by a _namespace_ identifier -which is separated from the body of the _name_ by a `: U+003A COLON`. +_Functions_ and _options_ MAY be preceded by a _namespace_ identifier +which is separated from the body of the _name_ by a U+003A COLON `:`. Built-in _functions_ and _options_ do not have a _namespace_ identifier. Examples: From a75908a1351f28f7b84a6abba15a593e9b6c1f19 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 10:44:39 -0800 Subject: [PATCH 05/39] Update spec/formatting.md --- spec/formatting.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 621a07841d..e25544bea3 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -223,8 +223,8 @@ the following steps are taken: Implementations SHOULD use an implementation-defined _namesapce_ for any additional arguments exposed to users as _options_. - An implementations MAY define its own functions or allow custom functions - to be defined by users. +An implementation MAY define its own functions. +An implementation MAY allow custom functions to be defined by users. Function access to the _formatting context_ SHOULD be minimal and read-only, and execution time SHOULD be limited. Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_. From 7f099a2589496b136ed53de6f0b8c5880c7d7a79 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 10:45:20 -0800 Subject: [PATCH 06/39] Update spec/formatting.md Co-authored-by: Eemeli Aro --- spec/formatting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index e25544bea3..82ac0290ec 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -225,7 +225,7 @@ the following steps are taken: An implementation MAY define its own functions. An implementation MAY allow custom functions to be defined by users. - Function access to the _formatting context_ SHOULD be minimal and read-only, + Function access to the _formatting context_ MUST be minimal and read-only, and execution time SHOULD be limited. Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_. User-defined _functions_ SHOULD be permitted to define the _namespace_ to use From bcff52e20f268e981662f939252b6d46e4883706 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 11:31:03 -0800 Subject: [PATCH 07/39] Update message.abnf From a suggestion by @eemeli but accounting for the use of `name-part` in `variable` --- spec/message.abnf | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/spec/message.abnf b/spec/message.abnf index e6c620aac7..36f1af0b9c 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -22,7 +22,7 @@ function-expression = "{" [s] annotation [s] "}" annotation = (function *(s option)) / reserved / private-use literal = quoted / unquoted -variable = "$" name-body +variable = "$" name-part function = (":" / "+" / "-") name option = name [s] "=" [s] (literal / variable) @@ -71,10 +71,9 @@ reserved-char = %x00-08 ; omit HTAB and LF / %xE000-10FFFF ; based on https://www.w3.org/TR/REC-xml-names/#NT-QName -name = [namespace] name-body -namespace = name-start *name-char namespace-sep -namespace-sep = ":" -name-body = name-start *name-char +name = [namespace ":"] name-body +namespace = name-part +name-part = name-start *name-char name-start = ALPHA / "_" / %xC0-D6 / %xD8-F6 / %xF8-2FF / %x370-37D / %x37F-1FFF / %x200C-200D From 7e73ab7b3ace0b5a34d45350d1a0d0e1db7aaf46 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 11:34:16 -0800 Subject: [PATCH 08/39] Adjust to match syntax changes --- spec/syntax.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/spec/syntax.md b/spec/syntax.md index a0bb0111fc..df23e25aec 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -677,7 +677,7 @@ unquoted-start = name-start / DIGIT / "." A **_name_** is an identifier for a _variable_ (prefixed with `$`), for a _function_ (prefixed with `:`, `+` or `-`), or for an _option_ (these have no prefix). -The namespace for _names_ is based on Namespaces in XML 1.0's +The namespace for _names_ is based on Namespaces in XML 1.0's [NCName](https://www.w3.org/TR/xml-names/#NT-NCName). This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name) in that it MUST NOT start with `:`. @@ -709,13 +709,12 @@ Support for _namespaces_ and their interpretation is implementation-defined in this release. ```abnf -variable = "$" name-body +variable = "$" name-part function = (":" / "+" / "-") name -name = [namespace] name-body -namespace = name-start *name-char namespace-sep -namespace-sep = ":" -name-body = name-start *name-char +name = [namespace ":"] name-part +namespace = name-part +name-part = name-start *name-char name-start = ALPHA / "_" / %xC0-D6 / %xD8-F6 / %xF8-2FF / %x370-37D / %x37F-1FFF / %x200C-200D From 732ab83d0dfc20d56494be7b8689306b0bdb651a Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 11:49:00 -0800 Subject: [PATCH 09/39] Fixing list numbering and name-handling wording --- spec/formatting.md | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 82ac0290ec..2f4863370a 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -202,15 +202,16 @@ the following steps are taken: If the registry does not define an implementation for this _name_, emit an Unknown Function error and use a _fallback value_ for the _expression_. -4. If the _expression_ includes _options_, resolve the _options_ to a mapping +3. If the _expression_ includes _options_, resolve the _options_ to a mapping of string identifiers to values. For each _option_: - - _Resolve the name_ of the option. - - If its right-hand side successfully resolves to a value, - bind the _name_ of the _option_ to the resolved value in the mapping. + - _Resolve the name_ of the _option_. + - If the _option_'s _name-part_ already exists in the resolved mapping of _options_, + emit a Duplicate Option Name error. + - If the _option_'s right-hand side successfully resolves to a value, + bind the _name-part_ of the _option_'s to the resolved value in the mapping. - Otherwise, do not bind the _name_ of the _option_ to any value in the mapping. -6. Call the function implementation with the following arguments: - +4. Call the function implementation with the following arguments: - The current _locale_. - The resolved mapping of _options_. - If the _expression_ includes an _operand_, its resolved value. @@ -220,18 +221,20 @@ the following steps are taken: An implementation MAY pass additional arguments to the function, as long as reasonable precautions are taken to keep the function interface simple and minimal, and avoid introducing potential security vulnerabilities. - Implementations SHOULD use an implementation-defined _namesapce_ for - any additional arguments exposed to users as _options_. -An implementation MAY define its own functions. -An implementation MAY allow custom functions to be defined by users. + Implementations MAY expose _options_ in an implementation-defined _namespace_ + in addition to or superceding those found in the default registry. + + An implementation MAY define its own functions. + An implementation MAY allow custom functions to be defined by users. + Function access to the _formatting context_ MUST be minimal and read-only, and execution time SHOULD be limited. Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_. User-defined _functions_ SHOULD be permitted to define the _namespace_ to use in name resolution. -8. If the call succeeds, +5. If the call succeeds, resolve the value of the _expression_ as the result of that function call. If the call fails or does not return a valid value, emit a Resolution error and use a _fallback value_ for the _expression_. @@ -239,10 +242,10 @@ An implementation MAY allow custom functions to be defined by users. ### Name Resolution **_Name resolution_** is the determination of -the _name_ and the _namespace_ of the name for a given _function_ +the _name-part_ and, where present, the _namespace_ of the _name_ for a given _function_ or _option_. -To determine the _name_ of an item, the following steps are taken: +To resolve the _name_ of an item, the following steps are taken: 1. Let the _name_ be the string to be processed, less any preceeding sigils or markers. @@ -251,10 +254,11 @@ To determine the _name_ of an item, the following steps are taken: 2. Determine if _name_ contains the character U+003A COLON `:`: - If _name_ contains a colon, let _namespace_ be the substring of _name_ up to (but not including) - the colon and _name_ be the substring of _name_ following the colon; - - Else let _namespace_ be an empty value (null, void, or empty string). -4. If the _name_ is empty, return a Syntax error. -5. Return _namespace_ and _name_. + the colon and _name-part_ be the substring of _name_ following the colon; + - Else let _namespace_ be an empty value (null, void, or empty string) + and _name-part_ be the _name_. +3. If the _name-part_ is empty, return a Syntax error. +4. Return _namespace_ and _name-part_. Implementations are not required to support the installation or resolution of different namespaces. From d69a927a3a75d7d32ccbbf21f19d7f837731b05d Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 16:22:48 -0800 Subject: [PATCH 10/39] Update spec/formatting.md Co-authored-by: Tim Chevalier --- spec/formatting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index 2f4863370a..dfef690f5e 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -223,7 +223,7 @@ the following steps are taken: simple and minimal, and avoid introducing potential security vulnerabilities. Implementations MAY expose _options_ in an implementation-defined _namespace_ - in addition to or superceding those found in the default registry. + in addition to or superseding those found in the default registry. An implementation MAY define its own functions. An implementation MAY allow custom functions to be defined by users. From 1dc97bfcd95494e80de5a2b1e5ee62681f4c92e4 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 16:23:40 -0800 Subject: [PATCH 11/39] Update spec/formatting.md Co-authored-by: Tim Chevalier --- spec/formatting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index dfef690f5e..4fdd7f9e95 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -280,7 +280,7 @@ Example: > > `name`: `datetime` > The `name` of the option in the expression is resolved as: > > `namespace`: `icu` -> > `name`: `skelelton` +> > `name`: `skeleton` ### Fallback Resolution From 85d7eadcc60df05691200acc36a2f10362d3d4b1 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 16:36:38 -0800 Subject: [PATCH 12/39] Update spec/formatting.md Co-authored-by: Tim Chevalier --- spec/formatting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index 4fdd7f9e95..32ec5ec9f2 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -271,7 +271,7 @@ Example: > Suppose the ICU implementation of MessageFormat added a custom option `skeleton` > to the `:datetime` function. > Users of that implementation could invoke the datetime formatter with an -> expression similar to: +> pattern similar to: > ``` > {{Today is {$date :datetime icu:skeleton=yMMMd}.}} > ``` From c3ad9d93ee5db89af2249fc064df80c026e3fe92 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 14 Nov 2023 16:38:13 -0800 Subject: [PATCH 13/39] Update spec/formatting.md Co-authored-by: Richard Gibson --- spec/formatting.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 32ec5ec9f2..5b1c7c32ee 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -276,11 +276,12 @@ Example: > {{Today is {$date :datetime icu:skeleton=yMMMd}.}} > ``` > The `name` of the `datetime` function is resolved as: -> > `namespace`: (blank) -> > `name`: `datetime` +> * `namespace`: (blank) +> * `name`: `datetime` +> > The `name` of the option in the expression is resolved as: -> > `namespace`: `icu` -> > `name`: `skeleton` +> * `namespace`: `icu` +> * `name`: `skeleton` ### Fallback Resolution From 9e40a9c0f02bbe91ee5efaf8eb401482bd9bf3f0 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Wed, 15 Nov 2023 09:03:54 -0800 Subject: [PATCH 14/39] Remove `:` from `name-char` This makes `name` equivalent to xml-names's NCName (just like the spec text says) --- spec/message.abnf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/message.abnf b/spec/message.abnf index 36f1af0b9c..5366e21e43 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -79,7 +79,7 @@ name-start = ALPHA / "_" / %x370-37D / %x37F-1FFF / %x200C-200D / %x2070-218F / %x2C00-2FEF / %x3001-D7FF / %xF900-FDCF / %xFDF0-FFFD / %x10000-EFFFF -name-char = name-start / DIGIT / "-" / "." / ":" +name-char = name-start / DIGIT / "-" / "." / %xB7 / %x300-36F / %x203F-2040 text-escape = backslash ( backslash / "{" / "}" ) From 233c2174a18a159212eb667fc93a42dce01e90bd Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 20 Nov 2023 08:28:20 -0800 Subject: [PATCH 15/39] Clean up references to NCName and clarify no colons --- spec/syntax.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/syntax.md b/spec/syntax.md index df23e25aec..8372dcac60 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -680,7 +680,7 @@ or for an _option_ (these have no prefix). The namespace for _names_ is based on Namespaces in XML 1.0's [NCName](https://www.w3.org/TR/xml-names/#NT-NCName). This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name) -in that it MUST NOT start with `:`. +in that it MUST NOT contain a U+003A COLON `:`. Otherwise, the set of characters allowed in names is large. _Functions_ and _options_ MAY be preceded by a _namespace_ identifier From 98307853a85357023096f424f9e5d83045b33303 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 20 Nov 2023 08:42:08 -0800 Subject: [PATCH 16/39] Update formatting based on comments --- spec/formatting.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 5b1c7c32ee..a52204eee9 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -199,19 +199,21 @@ the following steps are taken: If this fails, use a _fallback value_ for the _expression_. 2. _Resolve the name_ of the _function_ and, based on the starting sigil, find the appropriate function implementation from the _function registry_. - If the registry does not define an implementation for this _name_, + If the implementation does not support the _namespace_ of the _name_ + or the _namespace_ does not support the _name-part_ named _function_ + or the registry does not otherwise support the _name_, emit an Unknown Function error and use a _fallback value_ for the _expression_. -3. If the _expression_ includes _options_, resolve the _options_ to a mapping +4. If the _expression_ includes _options_, resolve the _options_ to a mapping of string identifiers to values. For each _option_: - _Resolve the name_ of the _option_. - - If the _option_'s _name-part_ already exists in the resolved mapping of _options_, + - If the _option_'s _name_ already exists in the resolved mapping of _options_, emit a Duplicate Option Name error. - If the _option_'s right-hand side successfully resolves to a value, - bind the _name-part_ of the _option_'s to the resolved value in the mapping. + bind the _name_ of the _option_ to the resolved value in the mapping. - Otherwise, do not bind the _name_ of the _option_ to any value in the mapping. -4. Call the function implementation with the following arguments: +5. Call the function implementation with the following arguments: - The current _locale_. - The resolved mapping of _options_. - If the _expression_ includes an _operand_, its resolved value. @@ -234,7 +236,7 @@ the following steps are taken: User-defined _functions_ SHOULD be permitted to define the _namespace_ to use in name resolution. -5. If the call succeeds, +6. If the call succeeds, resolve the value of the _expression_ as the result of that function call. If the call fails or does not return a valid value, emit a Resolution error and use a _fallback value_ for the _expression_. @@ -263,8 +265,6 @@ To resolve the _name_ of an item, the following steps are taken: Implementations are not required to support the installation or resolution of different namespaces. How namespaces are defined, provisioned, or handled is also implementation defined. -It is important to note that the _namespace_ is not considered part of the _name_ -after resolution. The _namespace_ MAY affect which function is called. Example: @@ -276,12 +276,15 @@ Example: > {{Today is {$date :datetime icu:skeleton=yMMMd}.}} > ``` > The `name` of the `datetime` function is resolved as: +> * `name` : `datetime` > * `namespace`: (blank) -> * `name`: `datetime` +> * `name-part`: `datetime` > > The `name` of the option in the expression is resolved as: +> * `name` : `icu:skeleton` > * `namespace`: `icu` -> * `name`: `skeleton` +> * `name-part`: `skeleton` + ### Fallback Resolution From 341717c5f754c76631eb33757bfb74e4d09b94b6 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 20 Nov 2023 09:26:00 -0800 Subject: [PATCH 17/39] Update spec/formatting.md Co-authored-by: Eemeli Aro --- spec/formatting.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index a52204eee9..a7ac848854 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -200,8 +200,7 @@ the following steps are taken: 2. _Resolve the name_ of the _function_ and, based on the starting sigil, find the appropriate function implementation from the _function registry_. If the implementation does not support the _namespace_ of the _name_ - or the _namespace_ does not support the _name-part_ named _function_ - or the registry does not otherwise support the _name_, + or the _namespace_ does not support the _name-part_ named _function_, emit an Unknown Function error and use a _fallback value_ for the _expression_. 4. If the _expression_ includes _options_, resolve the _options_ to a mapping From 290a7c49c1fc00514becd352feea5f3b81d18c50 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 20 Nov 2023 12:33:12 -0800 Subject: [PATCH 18/39] Address comments --- spec/formatting.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index a7ac848854..b880eef1be 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -223,17 +223,12 @@ the following steps are taken: as long as reasonable precautions are taken to keep the function interface simple and minimal, and avoid introducing potential security vulnerabilities. - Implementations MAY expose _options_ in an implementation-defined _namespace_ - in addition to or superseding those found in the default registry. - An implementation MAY define its own functions. An implementation MAY allow custom functions to be defined by users. Function access to the _formatting context_ MUST be minimal and read-only, and execution time SHOULD be limited. Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_. - User-defined _functions_ SHOULD be permitted to define the _namespace_ to use - in name resolution. 6. If the call succeeds, resolve the value of the _expression_ as the result of that function call. From 2c471a5a549e5f0fce3c95342a9913234e634ca9 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Sat, 25 Nov 2023 10:23:44 -0800 Subject: [PATCH 19/39] Address part of @eemeli's feedback Removing "resolve the name" to the syntax spec. --- spec/formatting.md | 65 ++++++++++++---------------------------------- 1 file changed, 16 insertions(+), 49 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index b880eef1be..f2d10af6a2 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -198,11 +198,17 @@ the following steps are taken: 1. If the _expression_ includes an _operand_, resolve its value. If this fails, use a _fallback value_ for the _expression_. 2. _Resolve the name_ of the _function_ and, based on the starting sigil, - find the appropriate function implementation from the _function registry_. - If the implementation does not support the _namespace_ of the _name_ - or the _namespace_ does not support the _name-part_ named _function_, - emit an Unknown Function error + find the appropriate function implementation to call. + If the implementation cannot find the function, emit an Unknown Function error and use a _fallback value_ for the _expression_. + + An implementation MAY emit an Unknown Function error if the implementation + does not support the _namespace_ of the _name_ + or the _namespace_ does not support the _name-part_ named _function_. + + Implementations are not required to implement _namespaces_ or installable + _function registries_. + 4. If the _expression_ includes _options_, resolve the _options_ to a mapping of string identifiers to values. For each _option_: @@ -226,59 +232,20 @@ the following steps are taken: An implementation MAY define its own functions. An implementation MAY allow custom functions to be defined by users. + An implementation MAY perform additional processing before calling the function. + For example, it can modify the _name_ of an _option_ to match the function's + input expectations (such as by removing the _namespace_ prefix from the _name-part_). + Function access to the _formatting context_ MUST be minimal and read-only, and execution time SHOULD be limited. + Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_. -6. If the call succeeds, +7. If the call succeeds, resolve the value of the _expression_ as the result of that function call. If the call fails or does not return a valid value, emit a Resolution error and use a _fallback value_ for the _expression_. -### Name Resolution - -**_Name resolution_** is the determination of -the _name-part_ and, where present, the _namespace_ of the _name_ for a given _function_ -or _option_. - -To resolve the _name_ of an item, the following steps are taken: - -1. Let the _name_ be the string to be processed, less any preceeding - sigils or markers. - For example, in the _expression_ `{:function}`, the string `function` - is the _name_ string. -2. Determine if _name_ contains the character U+003A COLON `:`: - - If _name_ contains a colon, let _namespace_ be the - substring of _name_ up to (but not including) - the colon and _name-part_ be the substring of _name_ following the colon; - - Else let _namespace_ be an empty value (null, void, or empty string) - and _name-part_ be the _name_. -3. If the _name-part_ is empty, return a Syntax error. -4. Return _namespace_ and _name-part_. - -Implementations are not required to support the installation or resolution -of different namespaces. -How namespaces are defined, provisioned, or handled is also implementation defined. -The _namespace_ MAY affect which function is called. - -Example: -> Suppose the ICU implementation of MessageFormat added a custom option `skeleton` -> to the `:datetime` function. -> Users of that implementation could invoke the datetime formatter with an -> pattern similar to: -> ``` -> {{Today is {$date :datetime icu:skeleton=yMMMd}.}} -> ``` -> The `name` of the `datetime` function is resolved as: -> * `name` : `datetime` -> * `namespace`: (blank) -> * `name-part`: `datetime` -> -> The `name` of the option in the expression is resolved as: -> * `name` : `icu:skeleton` -> * `namespace`: `icu` -> * `name-part`: `skeleton` - ### Fallback Resolution From 4507a4e34a02261c8981e097e0a438c69ca2639c Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Sat, 25 Nov 2023 10:29:51 -0800 Subject: [PATCH 20/39] Update syntax.md --- spec/syntax.md | 51 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 4 deletions(-) diff --git a/spec/syntax.md b/spec/syntax.md index 8372dcac60..f2e76ad5e6 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -687,6 +687,11 @@ _Functions_ and _options_ MAY be preceded by a _namespace_ identifier which is separated from the body of the _name_ by a U+003A COLON `:`. Built-in _functions_ and _options_ do not have a _namespace_ identifier. +> [!NOTE] +> _External variables_ can be passed in that are not valid _names_. +> Such variables cannot be referenced in a _message_, +> but are not otherwise errors. + Examples: > A variable: >``` @@ -724,10 +729,48 @@ name-char = name-start / DIGIT / "-" / "." / ":" / %xB7 / %x300-36F / %x203F-2040 ``` -> [!NOTE] -> _External variables_ can be passed in that are not valid _names_. -> Such variables cannot be referenced in a _message_, -> but are not otherwise errors. +#### Name resolution + +**_Resolve the name_** means the determination of +the _name-part_ and, where present, the _namespace_ of the _name_ for a given _function_ +or _option_. + +To resolve the _name_ of an item, the following steps are taken: + +1. Let the _name_ be the string to be processed, less any preceeding + sigils or markers. + For example, in the _expression_ `{:function}`, the string `function` + is the _name_ string. +2. Determine if _name_ contains the character U+003A COLON `:`: + - If _name_ contains a colon, let _namespace_ be the + substring of _name_ up to (but not including) + the colon and _name-part_ be the substring of _name_ following the colon; + - Else let _namespace_ be an empty value (null, void, or empty string) + and _name-part_ be the _name_. +3. Return _namespace_ and _name-part_. + +Implementations are not required to support the installation or resolution +of different namespaces. +How namespaces are defined, provisioned, or handled is also implementation defined. + +Example: +> Suppose the ICU implementation of MessageFormat added a custom option `skeleton` +> to the `:datetime` function. +> Users of that implementation could invoke the datetime formatter with an +> pattern similar to: +> ``` +> {{Today is {$date :datetime icu:skeleton=yMMMd}.}} +> ``` +> The `name` of the `datetime` function is resolved as: +> * `name` : `datetime` +> * `namespace`: (blank) +> * `name-part`: `datetime` +> +> The `name` of the option in the expression is resolved as: +> * `name` : `icu:skeleton` +> * `namespace`: `icu` +> * `name-part`: `skeleton` + ### Escape Sequences From 6b15f1044b97e7002bd71afce09c5a8878608527 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Sun, 26 Nov 2023 07:37:21 -0800 Subject: [PATCH 21/39] Update spec/message.abnf Co-authored-by: Eemeli Aro --- spec/message.abnf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/message.abnf b/spec/message.abnf index 5366e21e43..a8dd3bd4ae 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -71,7 +71,7 @@ reserved-char = %x00-08 ; omit HTAB and LF / %xE000-10FFFF ; based on https://www.w3.org/TR/REC-xml-names/#NT-QName -name = [namespace ":"] name-body +name = [namespace ":"] name-part namespace = name-part name-part = name-start *name-char name-start = ALPHA / "_" From cccb7adee05a6de8faa4fc87cf994941345de8dc Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Sun, 26 Nov 2023 07:38:45 -0800 Subject: [PATCH 22/39] Update spec/formatting.md Co-authored-by: Eemeli Aro --- spec/formatting.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index f2d10af6a2..1f6bef7741 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -218,7 +218,8 @@ the following steps are taken: - If the _option_'s right-hand side successfully resolves to a value, bind the _name_ of the _option_ to the resolved value in the mapping. - Otherwise, do not bind the _name_ of the _option_ to any value in the mapping. -5. Call the function implementation with the following arguments: +4. Call the function implementation with the following arguments: + - The current _locale_. - The resolved mapping of _options_. - If the _expression_ includes an _operand_, its resolved value. From ef83110d60d9a7d2bf9ac6eee28684ef451ef3bc Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Sun, 26 Nov 2023 07:39:39 -0800 Subject: [PATCH 23/39] Update spec/formatting.md Co-authored-by: Eemeli Aro --- spec/formatting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index 1f6bef7741..5728a7d520 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -242,7 +242,7 @@ the following steps are taken: Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_. -7. If the call succeeds, +5. If the call succeeds, resolve the value of the _expression_ as the result of that function call. If the call fails or does not return a valid value, emit a Resolution error and use a _fallback value_ for the _expression_. From e9fb373f31b88afe0366732b25733c4b8b68c893 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Sun, 26 Nov 2023 12:13:54 -0800 Subject: [PATCH 24/39] Update spec/formatting.md Co-authored-by: Eemeli Aro --- spec/formatting.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 5728a7d520..37230da2f4 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -233,10 +233,6 @@ the following steps are taken: An implementation MAY define its own functions. An implementation MAY allow custom functions to be defined by users. - An implementation MAY perform additional processing before calling the function. - For example, it can modify the _name_ of an _option_ to match the function's - input expectations (such as by removing the _namespace_ prefix from the _name-part_). - Function access to the _formatting context_ MUST be minimal and read-only, and execution time SHOULD be limited. From 17d714ecf865487f760d04c1d42b52dbd6d17b18 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 08:13:36 -0800 Subject: [PATCH 25/39] Update spec/syntax.md Co-authored-by: Eemeli Aro --- spec/syntax.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/spec/syntax.md b/spec/syntax.md index f2e76ad5e6..32d145dfa4 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -683,9 +683,11 @@ This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name) in that it MUST NOT contain a U+003A COLON `:`. Otherwise, the set of characters allowed in names is large. -_Functions_ and _options_ MAY be preceded by a _namespace_ identifier -which is separated from the body of the _name_ by a U+003A COLON `:`. -Built-in _functions_ and _options_ do not have a _namespace_ identifier. +A **_name_** is a character sequence that is used as +an identifier for _functions_ and _options_. +This MAY include a non-empty _namespace_ identifier at its start +which is separated from the rest of the _name_ by a U+003A COLON `:`. +Built-in _functions_ and their _options_ do not have a _namespace_ identifier. > [!NOTE] > _External variables_ can be passed in that are not valid _names_. From 714b4fedd08ecf119edb666eeec71724f0bfabfa Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 08:23:20 -0800 Subject: [PATCH 26/39] Address naming of `identifier` and `name` --- spec/message.abnf | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/spec/message.abnf b/spec/message.abnf index a8dd3bd4ae..88be80759e 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -22,9 +22,9 @@ function-expression = "{" [s] annotation [s] "}" annotation = (function *(s option)) / reserved / private-use literal = quoted / unquoted -variable = "$" name-part -function = (":" / "+" / "-") name -option = name [s] "=" [s] (literal / variable) +variable = "$" name +function = (":" / "+" / "-") identifier +option = identifier [s] "=" [s] (literal / variable) ; reserved keywords are always lowercase input = %s"input" @@ -71,9 +71,9 @@ reserved-char = %x00-08 ; omit HTAB and LF / %xE000-10FFFF ; based on https://www.w3.org/TR/REC-xml-names/#NT-QName -name = [namespace ":"] name-part -namespace = name-part -name-part = name-start *name-char +identifier = [namespace ":"] name +namespace = name +name = name-start *name-char name-start = ALPHA / "_" / %xC0-D6 / %xD8-F6 / %xF8-2FF / %x370-37D / %x37F-1FFF / %x200C-200D From 0c50ab7baf28a6b027825347f54625bc89dfc5ce Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 08:35:44 -0800 Subject: [PATCH 27/39] Implementing `identifier` and `name` in syntax.md --- spec/syntax.md | 94 ++++++++++++++++---------------------------------- 1 file changed, 30 insertions(+), 64 deletions(-) diff --git a/spec/syntax.md b/spec/syntax.md index 32d145dfa4..b87e1bd25e 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -169,7 +169,7 @@ external input value does not appear in a _declaration_. > [!Note] > These restrictions only apply to _declarations_. > A _placeholder_ or _selector_ can apply a different annotation to a _variable_ -> than one applied to the same _variable_ name in a _declaration_. +> than one applied to the same _variable_ named in a _declaration_. > For example, this message is _valid_: > ``` > {{ @@ -482,7 +482,7 @@ and vice versa. > {+button}Submit{-button} or {+link}cancel{-link}. > ``` -A _function_ consists of a prefix sigil followed by a _name_. +A _function_ consists of a prefix sigil followed by an _identifier_. The following sigils are used for _functions_: - `:` for a _standalone_ function @@ -497,8 +497,8 @@ _Options_ are not required. An **_option_** is a key-value pair containing a named argument that is passed to a _function_. -An _option_ has a _name_ and a _value_. -The _name_ is separated from the _value_ by an U+003D EQUALS SIGN `=` along with +An _option_ has an _identifier_ and a _value_. +The _identifier_ is separated from the _value_ by an U+003D EQUALS SIGN `=` along with optional whitespace. The value of an _option_ can be either a _literal_ or a _variable_. @@ -506,7 +506,7 @@ Multiple _options_ are permitted in an _annotation_. Each _option_ is separated by whitespace. ```abnf -option = name [s] "=" [s] (literal / variable) +option = identifier [s] "=" [s] (literal / variable) ``` > Examples of _functions_ with _options_ @@ -672,22 +672,30 @@ unquoted-start = name-start / DIGIT / "." / %xB7 / %x300-36F / %x203F-2040 ``` -### Names +### Names and Identifiers -A **_name_** is an identifier for a _variable_ (prefixed with `$`), +An **_identifier_** is a character sequence that +identifies a _function_ or _option_. +Each _identifier_ consists of a _name_ optionally preceeded by +a _namespace_. +When present, the _namespace_ is separated from the _name_ by a +U+003A COLON `:`. +Built-in _functions_ and their _options_ do not have a _namespace_ identifier. + +_Function_ _identifiers_ are prefixed with `:`, `+`, or `-`. +_Option_ _identifiers_ have no prefix. + +A **_name_** is a character sequence used in an _identifier_ +or as the name for for a _variable_. + +_Variable_ names are prefixed with `$`. for a _function_ (prefixed with `:`, `+` or `-`), -or for an _option_ (these have no prefix). -The namespace for _names_ is based on Namespaces in XML 1.0's + +Valid content for _names_ is based on Namespaces in XML 1.0's [NCName](https://www.w3.org/TR/xml-names/#NT-NCName). This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name) in that it MUST NOT contain a U+003A COLON `:`. -Otherwise, the set of characters allowed in names is large. - -A **_name_** is a character sequence that is used as -an identifier for _functions_ and _options_. -This MAY include a non-empty _namespace_ identifier at its start -which is separated from the rest of the _name_ by a U+003A COLON `:`. -Built-in _functions_ and their _options_ do not have a _namespace_ identifier. +Otherwise, the set of characters allowed in a _name_ is large. > [!NOTE] > _External variables_ can be passed in that are not valid _names_. @@ -716,12 +724,13 @@ Support for _namespaces_ and their interpretation is implementation-defined in this release. ```abnf -variable = "$" name-part -function = (":" / "+" / "-") name +variable = "$" name +function = (":" / "+" / "-") identifier +option = identifier [s] "=" [s] (literal / variable) -name = [namespace ":"] name-part -namespace = name-part -name-part = name-start *name-char +identifier = [namespace ":"] name-part +namespace = name +name = name-start *name-char name-start = ALPHA / "_" / %xC0-D6 / %xD8-F6 / %xF8-2FF / %x370-37D / %x37F-1FFF / %x200C-200D @@ -731,49 +740,6 @@ name-char = name-start / DIGIT / "-" / "." / ":" / %xB7 / %x300-36F / %x203F-2040 ``` -#### Name resolution - -**_Resolve the name_** means the determination of -the _name-part_ and, where present, the _namespace_ of the _name_ for a given _function_ -or _option_. - -To resolve the _name_ of an item, the following steps are taken: - -1. Let the _name_ be the string to be processed, less any preceeding - sigils or markers. - For example, in the _expression_ `{:function}`, the string `function` - is the _name_ string. -2. Determine if _name_ contains the character U+003A COLON `:`: - - If _name_ contains a colon, let _namespace_ be the - substring of _name_ up to (but not including) - the colon and _name-part_ be the substring of _name_ following the colon; - - Else let _namespace_ be an empty value (null, void, or empty string) - and _name-part_ be the _name_. -3. Return _namespace_ and _name-part_. - -Implementations are not required to support the installation or resolution -of different namespaces. -How namespaces are defined, provisioned, or handled is also implementation defined. - -Example: -> Suppose the ICU implementation of MessageFormat added a custom option `skeleton` -> to the `:datetime` function. -> Users of that implementation could invoke the datetime formatter with an -> pattern similar to: -> ``` -> {{Today is {$date :datetime icu:skeleton=yMMMd}.}} -> ``` -> The `name` of the `datetime` function is resolved as: -> * `name` : `datetime` -> * `namespace`: (blank) -> * `name-part`: `datetime` -> -> The `name` of the option in the expression is resolved as: -> * `name` : `icu:skeleton` -> * `namespace`: `icu` -> * `name-part`: `skeleton` - - ### Escape Sequences An **_escape sequence_** is a two-character sequence starting with From 3625ac46d650f367d9d2c708633c4c947ce29886 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 08:39:11 -0800 Subject: [PATCH 28/39] Implement `identifier` vs. `name` in formatting.md --- spec/formatting.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 37230da2f4..3998540618 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -197,14 +197,14 @@ the following steps are taken: 1. If the _expression_ includes an _operand_, resolve its value. If this fails, use a _fallback value_ for the _expression_. -2. _Resolve the name_ of the _function_ and, based on the starting sigil, +2. Resolve the _identifier_ of the _function_ and, based on the starting sigil, find the appropriate function implementation to call. If the implementation cannot find the function, emit an Unknown Function error and use a _fallback value_ for the _expression_. An implementation MAY emit an Unknown Function error if the implementation - does not support the _namespace_ of the _name_ - or the _namespace_ does not support the _name-part_ named _function_. + does not support the _namespace_ of the _identifier_ + or the _namespace_ does not support the named _function_. Implementations are not required to implement _namespaces_ or installable _function registries_. @@ -212,12 +212,12 @@ the following steps are taken: 4. If the _expression_ includes _options_, resolve the _options_ to a mapping of string identifiers to values. For each _option_: - - _Resolve the name_ of the _option_. - - If the _option_'s _name_ already exists in the resolved mapping of _options_, + - Resolve the _identifier_ of the _option_. + - If the _option_'s _identifier_ already exists in the resolved mapping of _options_, emit a Duplicate Option Name error. - If the _option_'s right-hand side successfully resolves to a value, - bind the _name_ of the _option_ to the resolved value in the mapping. - - Otherwise, do not bind the _name_ of the _option_ to any value in the mapping. + bind the _identifier_ of the _option_ to the resolved value in the mapping. + - Otherwise, do not bind the _identifier_ of the _option_ to any value in the mapping. 4. Call the function implementation with the following arguments: - The current _locale_. @@ -271,13 +271,13 @@ The _fallback value_ depends on the contents of the _expression_: > Example: `$user` - _expression_ with no _operand_: - the _function_ starting sigil followed by its _name_ + the _function_ starting sigil followed by its _identifier_ > Examples: `:platform`, `+tag`, `-tag` - Otherwise: The U+FFFD REPLACEMENT CHARACTER `�` character -_Option_ names and values are not included in the _fallback value_. +_Option_ identifiers and values are not included in the _fallback value_. When an error occurs in an _expression_ with a _variable_ _operand_ and the _variable_ refers to a local _declaration_, @@ -833,7 +833,7 @@ These are divided into the following categories: > }} > ``` - - A **Duplicate Option Name error** occurs when the same _name_ + - A **Duplicate Option Name error** occurs when the same _identifier_ appears on the left-hand side of more than one _option_ in the same _expression_. From 94dfbd427aa0679b6d1136862fded23294bf195c Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 09:18:26 -0800 Subject: [PATCH 29/39] Update spec/syntax.md Co-authored-by: Eemeli Aro --- spec/syntax.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/syntax.md b/spec/syntax.md index b87e1bd25e..9d2a3f9b94 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -728,7 +728,7 @@ variable = "$" name function = (":" / "+" / "-") identifier option = identifier [s] "=" [s] (literal / variable) -identifier = [namespace ":"] name-part +identifier = [namespace ":"] name namespace = name name = name-start *name-char name-start = ALPHA / "_" From 4692f7218a5d5f4a9a6e2058c82230049ba81cd1 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 12:10:11 -0800 Subject: [PATCH 30/39] Changes per 2023-11-27 call --- spec/message.abnf | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/spec/message.abnf b/spec/message.abnf index 88be80759e..97b39cb6e9 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -45,12 +45,8 @@ quoted-char = %x0-5B ; omit \ / %x7D-D7FF ; omit surrogates / %xE000-10FFFF -; based on https://www.w3.org/TR/xml/#NT-Nmtoken, -; but cannot start with U+002D HYPHEN-MINUS or U+003A COLON ":" -unquoted = unquoted-start *name-char +unquoted = unquoted-start *name-char unquoted-start = name-start / DIGIT / "." - / %xB7 / %x300-36F / %x203F-2040 - ; reserve sigils for private-use by implementations private-use = private-start reserved-body From 3adb4aa7b5a1a411ca6f6cc40b34122b1ec20c78 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 12:35:57 -0800 Subject: [PATCH 31/39] Update message.abnf --- spec/message.abnf | 1 + 1 file changed, 1 insertion(+) diff --git a/spec/message.abnf b/spec/message.abnf index 97b39cb6e9..707b08aaf1 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -47,6 +47,7 @@ quoted-char = %x0-5B ; omit \ unquoted = unquoted-start *name-char unquoted-start = name-start / DIGIT / "." + / %xB7 / %x300-36F / %x203F-2040 ; reserve sigils for private-use by implementations private-use = private-start reserved-body From 0b7042b619609dbfd7a5b2d1a1a931bd215f9b30 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Mon, 27 Nov 2023 14:22:18 -0800 Subject: [PATCH 32/39] Add `:` to unquoted --- spec/message.abnf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/message.abnf b/spec/message.abnf index 707b08aaf1..5e98aefd02 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -45,7 +45,7 @@ quoted-char = %x0-5B ; omit \ / %x7D-D7FF ; omit surrogates / %xE000-10FFFF -unquoted = unquoted-start *name-char +unquoted = unquoted-start *(name-char / ":") unquoted-start = name-start / DIGIT / "." / %xB7 / %x300-36F / %x203F-2040 From a4920faf8aa81117ee1a9b810ded1c0e9c304d40 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 28 Nov 2023 12:25:59 -0800 Subject: [PATCH 33/39] Update spec/formatting.md Co-authored-by: Richard Gibson --- spec/formatting.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 3998540618..91d1ef5fab 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -217,8 +217,10 @@ the following steps are taken: emit a Duplicate Option Name error. - If the _option_'s right-hand side successfully resolves to a value, bind the _identifier_ of the _option_ to the resolved value in the mapping. - - Otherwise, do not bind the _identifier_ of the _option_ to any value in the mapping. -4. Call the function implementation with the following arguments: + - Otherwise, emit an appropriate error (e.g., an Unresolved Variable error) and + bind the _identifier_ of the _option_ to an unresolved value in the mapping. +4. Remove from the resolved mapping of _options_ any binding for which the value is an unresolved value. +5. Call the function implementation with the following arguments: - The current _locale_. - The resolved mapping of _options_. From d3e0e584eebb5b85c17cf08aa1e9cd03a4802a21 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 28 Nov 2023 12:26:32 -0800 Subject: [PATCH 34/39] Update spec/message.abnf Co-authored-by: Richard Gibson --- spec/message.abnf | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/spec/message.abnf b/spec/message.abnf index 5e98aefd02..f4b610fe5c 100644 --- a/spec/message.abnf +++ b/spec/message.abnf @@ -67,7 +67,8 @@ reserved-char = %x00-08 ; omit HTAB and LF / %x7E-D7FF ; omit surrogates / %xE000-10FFFF -; based on https://www.w3.org/TR/REC-xml-names/#NT-QName +; identifier matches https://www.w3.org/TR/REC-xml-names/#NT-QName +; name matches https://www.w3.org/TR/REC-xml-names/#NT-NCName identifier = [namespace ":"] name namespace = name name = name-start *name-char From 938d7d5bdbc7a84dcbefab30a732c17967512a08 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 28 Nov 2023 12:28:39 -0800 Subject: [PATCH 35/39] Apply suggestions from code review Co-authored-by: Richard Gibson --- spec/formatting.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 91d1ef5fab..9670313338 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -202,15 +202,16 @@ the following steps are taken: If the implementation cannot find the function, emit an Unknown Function error and use a _fallback value_ for the _expression_. - An implementation MAY emit an Unknown Function error if the implementation - does not support the _namespace_ of the _identifier_ - or the _namespace_ does not support the named _function_. +An implementation MUST emit an Unknown Function error +if the implementation does not support _namespaces_ +or if the _namespace_ of the _identifier_ is not recognized by the implementation +or the _namespace_ does not support the named _function_. Implementations are not required to implement _namespaces_ or installable _function registries_. -4. If the _expression_ includes _options_, resolve the _options_ to a mapping - of string identifiers to values. +3. Resolve the _options_ to a mapping of string identifiers to values. + If _options_ is missing, the mapping will be empty. For each _option_: - Resolve the _identifier_ of the _option_. - If the _option_'s _identifier_ already exists in the resolved mapping of _options_, From fd62645c1cf53c53fc524c3579c9b8ed45f433f8 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 28 Nov 2023 12:30:52 -0800 Subject: [PATCH 36/39] Clean formatting --- spec/formatting.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 9670313338..34a1a91783 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -202,10 +202,10 @@ the following steps are taken: If the implementation cannot find the function, emit an Unknown Function error and use a _fallback value_ for the _expression_. -An implementation MUST emit an Unknown Function error -if the implementation does not support _namespaces_ -or if the _namespace_ of the _identifier_ is not recognized by the implementation -or the _namespace_ does not support the named _function_. + An implementation MUST emit an Unknown Function error + if the implementation does not support _namespaces_ + or if the _namespace_ of the _identifier_ is not recognized by the implementation + or the _namespace_ does not support the named _function_. Implementations are not required to implement _namespaces_ or installable _function registries_. From 60a17404dcf7001691b635349edea7319c8ac5c1 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 28 Nov 2023 12:35:24 -0800 Subject: [PATCH 37/39] Address double-emission of unresolved variable --- spec/formatting.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 34a1a91783..3c373afdad 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -218,8 +218,8 @@ the following steps are taken: emit a Duplicate Option Name error. - If the _option_'s right-hand side successfully resolves to a value, bind the _identifier_ of the _option_ to the resolved value in the mapping. - - Otherwise, emit an appropriate error (e.g., an Unresolved Variable error) and - bind the _identifier_ of the _option_ to an unresolved value in the mapping. + - Otherwise, bind the _identifier_ of the _option_ to an unresolved value in the mapping. + (Note that an Unresolved Variable error will have been emitted.) 4. Remove from the resolved mapping of _options_ any binding for which the value is an unresolved value. 5. Call the function implementation with the following arguments: From 023b5a8e2eebf0fc2fda912b0e47daa3842f87f3 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 28 Nov 2023 13:58:53 -0800 Subject: [PATCH 38/39] Update spec/formatting.md Co-authored-by: Eemeli Aro --- spec/formatting.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index 3c373afdad..40feb453d2 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -199,7 +199,9 @@ the following steps are taken: If this fails, use a _fallback value_ for the _expression_. 2. Resolve the _identifier_ of the _function_ and, based on the starting sigil, find the appropriate function implementation to call. - If the implementation cannot find the function, emit an Unknown Function error + If the implementation cannot find the function, + or if the _identifier_ includes a _namespace_ that the implementation does not support, + emit an Unknown Function error and use a _fallback value_ for the _expression_. An implementation MUST emit an Unknown Function error From 1e9e6442d452738da7e0b49da1a25b959ea68d0a Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Wed, 29 Nov 2023 05:37:20 -0800 Subject: [PATCH 39/39] Remove extra MUSTard Co-authored-by: Eemeli Aro --- spec/formatting.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 40feb453d2..c8179ef12d 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -204,11 +204,6 @@ the following steps are taken: emit an Unknown Function error and use a _fallback value_ for the _expression_. - An implementation MUST emit an Unknown Function error - if the implementation does not support _namespaces_ - or if the _namespace_ of the _identifier_ is not recognized by the implementation - or the _namespace_ does not support the named _function_. - Implementations are not required to implement _namespaces_ or installable _function registries_.