From 1a27ccd80023a1afd6288a4890fefa9291e47c6b Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Mon, 19 Dec 2022 19:18:03 +0200 Subject: [PATCH 01/11] Add error handling --- spec/formatting.md | 54 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/spec/formatting.md b/spec/formatting.md index 7fe7e4a42a..2daa0b6693 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -18,3 +18,57 @@ the local variable takes precedence. It is an error for a local variable definition to refer to a local variable that's defined after it in the message. + +## Error Handling + +During the formatting of a message, +various errors may be encountered. +These are divided to the following categories: + +- **Syntax errors** occur when the syntax representation of a message is invalid. +- **Resolution errors** occur when the runtime value of a part of a message + cannot be determined. +- **Unresolved variable errors** are a sub-category of resolution errors, + and occur when a variable reference cannot be resolved. +- **Formatting errors** are thrown by expression handlers, + for example when encountering a value with an unsupported type + or an internally inconsistent set of options. + +In all cases, when encountering an error, +a message formatter must provide some representation of the message. +An informative error must also be separately provided. + +When an error occurs in the syntax or resolution of an Expression or MarkupStart Option, +the Expression or MarkupStart in question is processed as if the option was not defined. +This may allow for the fallback handling described below to be avoided, +though an error must still be emitted. + +When an error occurs within a Selector, +the selector may only match the catch-all VariantKey `*`. + +When an error occurs in a Placeholder that is being formatted, +the fallback string representation of the Placeholder +always starts with U+007B LEFT CURLY BRACKET `{` +and ends with U+007D RIGHT CURLY BRACKET `}`. +Between the brackets, the following contents are used: + +- Expression with Literal Operand: U+0028 LEFT PARENTHESIS `(` + followed by the value of the Literal, + and then by U+0029 RIGHT PARENTHESIS `)` +- Expression with Variable Operand: U+0024 DOLLAR SIGN `$` + followed by the Variable Name of the Operand +- Expression with no Operand: U+003A COLON `:` followed by the Expression Name +- Markup start: U+002B PLUS SIGN `+` followed by the MarkupStart Name +- Markup end: U+002D HYPHEN-MINUS `-` followed by the MarkupEnd Name +- Otherwise: Three U+003F QUESTION MARK `?` characters, i.e. `???` + +For example, the formatted string representation of the expression `{$foo :bar}` +would be `{$foo}` if the variable could not be resolved. + +The formatted string representation of a message with an unrecoverable syntax error +is the concatenation of U+007B LEFT CURLY BRACKET `{`, +a string identifier for the message, +and U+007D RIGHT CURLY BRACKET `}`. +If an identifier is not available, +it is replaced with three U+003F QUESTION MARK `?` characters, +resulting in the string `{???}`. From 4389ae502131d93e0663465ca80efecccfe9af49 Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Thu, 22 Dec 2022 15:18:00 +0200 Subject: [PATCH 02/11] Address review comments from @zbraniecki and @aphillips --- spec/formatting.md | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 2daa0b6693..f993586135 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -28,15 +28,25 @@ These are divided to the following categories: - **Syntax errors** occur when the syntax representation of a message is invalid. - **Resolution errors** occur when the runtime value of a part of a message cannot be determined. -- **Unresolved variable errors** are a sub-category of resolution errors, - and occur when a variable reference cannot be resolved. -- **Formatting errors** are thrown by expression handlers, + + - **Unresolved Variable** errors occur when a variable reference cannot be resolved. + +- **Selection errors** cover failures encountered during selection. + + - **Selector errors** are failures in the matching of a key to a specific selector. + - **Missing Fallback** errors occur when no Variant is selected + due to the message not including a Variant with only catch-all keys. + +- **Formatting errors** occur during the formatting of a resolved value, for example when encountering a value with an unsupported type or an internally inconsistent set of options. +During selection, an expression handler must only emit Resolution and Selection errors. +During formatting, an expression handler must only emit Resolution and Formatting errors. + In all cases, when encountering an error, a message formatter must provide some representation of the message. -An informative error must also be separately provided. +An informative error or errors must also be separately provided. When an error occurs in the syntax or resolution of an Expression or MarkupStart Option, the Expression or MarkupStart in question is processed as if the option was not defined. @@ -44,7 +54,11 @@ This may allow for the fallback handling described below to be avoided, though an error must still be emitted. When an error occurs within a Selector, -the selector may only match the catch-all VariantKey `*`. +the selector must not match any VariantKey other than the catch-all `*` +and a Selector error is emitted. +When selection fails to match any Variant, +an empty string is used as the formatted string representation of the message +and a Missing Fallback error is emitted. When an error occurs in a Placeholder that is being formatted, the fallback string representation of the Placeholder From 55890b991ef0713b765b448051cf17d222ab48c5 Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Mon, 2 Jan 2023 20:00:47 +0200 Subject: [PATCH 03/11] Apply suggestions from code review --- spec/formatting.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index f993586135..eee407bf36 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -23,18 +23,18 @@ refer to a local variable that's defined after it in the message. During the formatting of a message, various errors may be encountered. -These are divided to the following categories: +These are divided into the following categories: - **Syntax errors** occur when the syntax representation of a message is invalid. - **Resolution errors** occur when the runtime value of a part of a message cannot be determined. - - **Unresolved Variable** errors occur when a variable reference cannot be resolved. + - **Unresolved Variable errors** occur when a variable reference cannot be resolved. -- **Selection errors** cover failures encountered during selection. +- **Selection errors** occur when message selection fails. - **Selector errors** are failures in the matching of a key to a specific selector. - - **Missing Fallback** errors occur when no Variant is selected + - **Missing Fallback errors** occur when no Variant is selected due to the message not including a Variant with only catch-all keys. - **Formatting errors** occur during the formatting of a resolved value, @@ -49,7 +49,7 @@ a message formatter must provide some representation of the message. An informative error or errors must also be separately provided. When an error occurs in the syntax or resolution of an Expression or MarkupStart Option, -the Expression or MarkupStart in question is processed as if the option was not defined. +the Expression or MarkupStart in question is processed as if the option were not defined. This may allow for the fallback handling described below to be avoided, though an error must still be emitted. @@ -74,7 +74,7 @@ Between the brackets, the following contents are used: - Expression with no Operand: U+003A COLON `:` followed by the Expression Name - Markup start: U+002B PLUS SIGN `+` followed by the MarkupStart Name - Markup end: U+002D HYPHEN-MINUS `-` followed by the MarkupEnd Name -- Otherwise: Three U+003F QUESTION MARK `?` characters, i.e. `???` +- Otherwise: The U+FFFD REPLACEMENT CHARACTER `�` character For example, the formatted string representation of the expression `{$foo :bar}` would be `{$foo}` if the variable could not be resolved. @@ -84,5 +84,5 @@ is the concatenation of U+007B LEFT CURLY BRACKET `{`, a string identifier for the message, and U+007D RIGHT CURLY BRACKET `}`. If an identifier is not available, -it is replaced with three U+003F QUESTION MARK `?` characters, -resulting in the string `{???}`. +it is replaced with the U+FFFD REPLACEMENT CHARACTER `�` character, +resulting in the string `{�}`. From 99ce4a13740f0b3413e0864c370b18d608a8b13a Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Tue, 17 Jan 2023 12:46:43 +0200 Subject: [PATCH 04/11] Add Data Model errors & apply suggestions from code review --- spec/formatting.md | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index eee407bf36..5687484402 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -25,7 +25,15 @@ During the formatting of a message, various errors may be encountered. These are divided into the following categories: -- **Syntax errors** occur when the syntax representation of a message is invalid. +- **Syntax errors** occur when the syntax representation of a message is not well-formed. +- **Data Model errors** occur when a message is invalid due to + violating one of the following semantic requirements on its structure: + + - **Variant Key Mismatch errors** occur when the number of keys on a Variant + does not equal the number of Selectors. + - **Missing Fallback Variant errors** occur when the message + does not include a Variant with only catch-all keys. + - **Resolution errors** occur when the runtime value of a part of a message cannot be determined. @@ -34,22 +42,22 @@ These are divided into the following categories: - **Selection errors** occur when message selection fails. - **Selector errors** are failures in the matching of a key to a specific selector. - - **Missing Fallback errors** occur when no Variant is selected - due to the message not including a Variant with only catch-all keys. - **Formatting errors** occur during the formatting of a resolved value, for example when encountering a value with an unsupported type or an internally inconsistent set of options. -During selection, an expression handler must only emit Resolution and Selection errors. -During formatting, an expression handler must only emit Resolution and Formatting errors. +Syntax and Data Model errors must be emitted as soon as possible. + +During selection, an Expression handler must only emit Resolution and Selection errors. +During formatting, an Expression handler must only emit Resolution and Formatting errors. In all cases, when encountering an error, a message formatter must provide some representation of the message. An informative error or errors must also be separately provided. -When an error occurs in the syntax or resolution of an Expression or MarkupStart Option, -the Expression or MarkupStart in question is processed as if the option were not defined. +When an error occurs in the resolution of an Expression or Markup Option, +the Expression or Markup in question is processed as if the option were not defined. This may allow for the fallback handling described below to be avoided, though an error must still be emitted. @@ -79,10 +87,10 @@ Between the brackets, the following contents are used: For example, the formatted string representation of the expression `{$foo :bar}` would be `{$foo}` if the variable could not be resolved. -The formatted string representation of a message with an unrecoverable syntax error +The formatted string representation of a message with a Syntax or Data Model error is the concatenation of U+007B LEFT CURLY BRACKET `{`, -a string identifier for the message, +a fallback string, and U+007D RIGHT CURLY BRACKET `}`. -If an identifier is not available, -it is replaced with the U+FFFD REPLACEMENT CHARACTER `�` character, +If a fallback string is not defined, +the U+FFFD REPLACEMENT CHARACTER `�` character is used, resulting in the string `{�}`. From da1ecaf3e49f9f4fdc2ab56e55f48126d99a6cbd Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Mon, 23 Jan 2023 16:30:09 +0200 Subject: [PATCH 05/11] Add examples --- spec/formatting.md | 117 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 115 insertions(+), 2 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 5687484402..a96097897f 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -26,27 +26,124 @@ various errors may be encountered. These are divided into the following categories: - **Syntax errors** occur when the syntax representation of a message is not well-formed. + + Example invalid messages resulting in a Syntax error: + + ``` + {Missing end brace + ``` + + ``` + {Unknown {?placeholder}} + ``` + + ``` + let $var = {(no message body)} + ``` + - **Data Model errors** occur when a message is invalid due to violating one of the following semantic requirements on its structure: - **Variant Key Mismatch errors** occur when the number of keys on a Variant does not equal the number of Selectors. + + Example invalid messages resulting in a Variant Key Mismatch error: + + ``` + match {$one} + when 1 2 {Too many} + when * {Otherwise} + ``` + + ``` + match {$one} {$two} + when 1 2 {Two keys} + when * {Too few} + when * * {Otherwise} + ``` + - **Missing Fallback Variant errors** occur when the message does not include a Variant with only catch-all keys. + Example invalid messages resulting in a Missing Fallback Variant error: + + ``` + match {$one} + when 1 {Value is one} + when 2 {Value is two} + ``` + + ``` + match {$one} {$two} + when 1 * {First is one} + when * 1 {Second is one} + ``` + - **Resolution errors** occur when the runtime value of a part of a message cannot be determined. - **Unresolved Variable errors** occur when a variable reference cannot be resolved. + For example, attempting to format either of the following messages + must result in an Unresolved Variable error if done within a context that + does not provide for the variable reference `$var` to be successfully resolved: + + ``` + {The value is {$var}.} + ``` + + ``` + match {$var} + when 1 {The value is one.} + when * {The value is not one.} + ``` + - **Selection errors** occur when message selection fails. - **Selector errors** are failures in the matching of a key to a specific selector. + For example, attempting to format either of the following messages + may result in a Selector error if done within a context that + uses a `:plural` selector function which requires its input to be numeric: + + ``` + match {(horse) :plural} + when 1 {The value is one.} + when * {The value is not one.} + ``` + + ``` + let $sel = {(horse) :plural} + match {$sel} + when 1 {The value is one.} + when * {The value is not one.} + ``` + - **Formatting errors** occur during the formatting of a resolved value, for example when encountering a value with an unsupported type or an internally inconsistent set of options. + For example, attempting to format any of the following messages + may result in a Formatting error if done within a context that + + 1. provides for the variable reference `$user` to resolve to + an object `{ name: 'Kat', id: 1234 }`, and + 2. uses a `:get` formatting function which requires its argument to be an object and + an option `field` to be provided with a string value, + + ``` + {Hello, {(horse) :get field=name}!} + ``` + + ``` + {Hello, {$user :get}!} + ``` + + ``` + let $id = {$user :get field=id} + {Hello, {$id :get field=name}!} + ``` + Syntax and Data Model errors must be emitted as soon as possible. During selection, an Expression handler must only emit Resolution and Selection errors. @@ -77,15 +174,31 @@ Between the brackets, the following contents are used: - Expression with Literal Operand: U+0028 LEFT PARENTHESIS `(` followed by the value of the Literal, and then by U+0029 RIGHT PARENTHESIS `)` + + Examples: `{(horse)}`, `{(42)}` + - Expression with Variable Operand: U+0024 DOLLAR SIGN `$` followed by the Variable Name of the Operand + + Example: `{$user}` + - Expression with no Operand: U+003A COLON `:` followed by the Expression Name + + Example: `{:platform}` + - Markup start: U+002B PLUS SIGN `+` followed by the MarkupStart Name + + Example: `{+tag}` + - Markup end: U+002D HYPHEN-MINUS `-` followed by the MarkupEnd Name + + Example: `{-tag}` + - Otherwise: The U+FFFD REPLACEMENT CHARACTER `�` character -For example, the formatted string representation of the expression `{$foo :bar}` -would be `{$foo}` if the variable could not be resolved. + Example: `{�}` + +Option names and values are not included in the fallback string representations. The formatted string representation of a message with a Syntax or Data Model error is the concatenation of U+007B LEFT CURLY BRACKET `{`, From 4d8c753864a7098c801d9d39e073a9675fa5b62b Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Tue, 24 Jan 2023 09:42:51 +0200 Subject: [PATCH 06/11] Add Unknown Function + Syntax/Data Model error priority + separate Fallback String Representations section --- spec/formatting.md | 44 ++++++++++++++++++++++++++++++++------------ 1 file changed, 32 insertions(+), 12 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index a96097897f..83e644307e 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -98,6 +98,23 @@ These are divided into the following categories: when * {The value is not one.} ``` + - **Unknown Function errors** occur when an Expression includes + a reference to a function which cannot be resolved. + + For example, attempting to format either of the following messages + must result in an Unknown Function error if done within a context that + does not provide for the function `:func` to be successfully resolved: + + ``` + {The value is {(horse) :func}.} + ``` + + ``` + match {(horse) :func} + when 1 {The value is one.} + when * {The value is not one.} + ``` + - **Selection errors** occur when message selection fails. - **Selector errors** are failures in the matching of a key to a specific selector. @@ -152,6 +169,10 @@ During formatting, an Expression handler must only emit Resolution and Formattin In all cases, when encountering an error, a message formatter must provide some representation of the message. An informative error or errors must also be separately provided. +When a message contains more than one error, +or contains some error which leads to further errors, +an implementation which does not emit all of the errors +should prioritise Syntax and Data Model errors over others. When an error occurs in the resolution of an Expression or Markup Option, the Expression or Markup in question is processed as if the option were not defined. @@ -160,10 +181,17 @@ though an error must still be emitted. When an error occurs within a Selector, the selector must not match any VariantKey other than the catch-all `*` -and a Selector error is emitted. -When selection fails to match any Variant, -an empty string is used as the formatted string representation of the message -and a Missing Fallback error is emitted. +and a Resolution or Selector error is emitted. + +## Fallback String Representations + +The formatted string representation of a message with a Syntax or Data Model error +is the concatenation of U+007B LEFT CURLY BRACKET `{`, +a fallback string, +and U+007D RIGHT CURLY BRACKET `}`. +If a fallback string is not defined, +the U+FFFD REPLACEMENT CHARACTER `�` character is used, +resulting in the string `{�}`. When an error occurs in a Placeholder that is being formatted, the fallback string representation of the Placeholder @@ -199,11 +227,3 @@ Between the brackets, the following contents are used: Example: `{�}` Option names and values are not included in the fallback string representations. - -The formatted string representation of a message with a Syntax or Data Model error -is the concatenation of U+007B LEFT CURLY BRACKET `{`, -a fallback string, -and U+007D RIGHT CURLY BRACKET `}`. -If a fallback string is not defined, -the U+FFFD REPLACEMENT CHARACTER `�` character is used, -resulting in the string `{�}`. From 2e1f943c22eb91c51821b961ffebdb4b294f3238 Mon Sep 17 00:00:00 2001 From: Addison Phillips Date: Tue, 24 Jan 2023 11:35:19 -0800 Subject: [PATCH 07/11] Update spec/formatting.md Co-authored-by: Eemeli Aro --- spec/formatting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index 83e644307e..7bff316483 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -58,7 +58,7 @@ These are divided into the following categories: ``` match {$one} {$two} when 1 2 {Two keys} - when * {Too few} + when * {Missing a key} when * * {Otherwise} ``` From 38e33ad168e3a60ce841be5d23096a59e29abf73 Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Tue, 24 Jan 2023 23:19:50 +0200 Subject: [PATCH 08/11] Specify Declaration vs Placeholder Expression for fallback string --- spec/formatting.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/spec/formatting.md b/spec/formatting.md index 7bff316483..1e1258ab7f 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -227,3 +227,27 @@ Between the brackets, the following contents are used: Example: `{�}` Option names and values are not included in the fallback string representations. + +When an error occurs in an Expression with a Variable Operand +and the Variable refers to a local variable Declaration, +the fallback string is formatted based on the Expression of the Declaration, +rather than the Expression of the Placeholder. + +For example, attempting to format either of the following messages within a context that +does not provide for the function `:func` to be successfully resolved: + +``` +let $var = {(horse) :func} +{The value is {$var}.} +``` + +``` +let $var = {(horse)} +{The value is {$var :func}.} +``` + +would result in both cases with this formatted string representation: + +``` +The value is {(horse)}. +``` From 8e12260e101f2de346ae7b6cfca3d625f78d7289 Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Mon, 30 Jan 2023 10:26:14 +0200 Subject: [PATCH 09/11] Update spec/formatting.md --- spec/formatting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/formatting.md b/spec/formatting.md index 1e1258ab7f..2fd5d86c07 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -42,7 +42,7 @@ These are divided into the following categories: ``` - **Data Model errors** occur when a message is invalid due to - violating one of the following semantic requirements on its structure: + violating one of the semantic requirements on its structure: - **Variant Key Mismatch errors** occur when the number of keys on a Variant does not equal the number of Selectors. From 22866fd4cba72f8afb75944cfecd6e7325c2025b Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Mon, 30 Jan 2023 11:31:08 +0200 Subject: [PATCH 10/11] Add paragraph on validation & field=$field example --- spec/formatting.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index 2fd5d86c07..abc5518af0 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -21,6 +21,12 @@ refer to a local variable that's defined after it in the message. ## Error Handling +Errors in messages and their formatting may occur and be detected +at multiple different stages of their processing. +Where available, +the use of validation tools is recommended, +as early detection of errors makes their correction easier. + During the formatting of a message, various errors may be encountered. These are divided into the following categories: @@ -144,8 +150,10 @@ These are divided into the following categories: may result in a Formatting error if done within a context that 1. provides for the variable reference `$user` to resolve to - an object `{ name: 'Kat', id: 1234 }`, and - 2. uses a `:get` formatting function which requires its argument to be an object and + an object `{ name: 'Kat', id: 1234 }`, + 2. provides for the variable reference `$field` to resolve to + a string `'address'`, and + 3. uses a `:get` formatting function which requires its argument to be an object and an option `field` to be provided with a string value, ``` @@ -161,6 +169,10 @@ These are divided into the following categories: {Hello, {$id :get field=name}!} ``` + ``` + {Your {$field} is {$id :get field=$field}} + ``` + Syntax and Data Model errors must be emitted as soon as possible. During selection, an Expression handler must only emit Resolution and Selection errors. From d46856fe3d30bdd3ff7757a23d5f3fbeaf7b5442 Mon Sep 17 00:00:00 2001 From: Eemeli Aro Date: Tue, 31 Jan 2023 00:32:14 +0200 Subject: [PATCH 11/11] Apply suggestions from code review --- spec/formatting.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/formatting.md b/spec/formatting.md index abc5518af0..a0fcf9a90e 100644 --- a/spec/formatting.md +++ b/spec/formatting.md @@ -40,7 +40,7 @@ These are divided into the following categories: ``` ``` - {Unknown {?placeholder}} + {Unknown {#placeholder#}} ``` ``` @@ -126,7 +126,7 @@ These are divided into the following categories: - **Selector errors** are failures in the matching of a key to a specific selector. For example, attempting to format either of the following messages - may result in a Selector error if done within a context that + might result in a Selector error if done within a context that uses a `:plural` selector function which requires its input to be numeric: ``` @@ -147,7 +147,7 @@ These are divided into the following categories: or an internally inconsistent set of options. For example, attempting to format any of the following messages - may result in a Formatting error if done within a context that + might result in a Formatting error if done within a context that 1. provides for the variable reference `$user` to resolve to an object `{ name: 'Kat', id: 1234 }`,