Reserve sigals or syntax for future expansion

[aphillips]

Is your feature request related to a problem? Please describe.
We want to make the syntax future proof.

Describe the solution you'd like
I would like to reserve a number of punctuation marks from name and nmtoken and define a set of sigals that are permitted to introduce a placeholder (even though these are not used for anything now). For example:

expression = (literal / variable) [ RSP annotation] / annotation
annotation = (function / reserved) *(RSP option)
function = ":" name
reserved = ("!" / "@" / "#" / "%" / "^" / "&" / "=" / "?") name ; reserved for future standarization

Describe why your solution should shape the standard
In our call of 2023-02-27 we discussed future expansion of the syntax. The best way to do this is to reserve potential syntax now, such that parsers and messages in production are not invalidated by future standardization.

Additional context or examples
N/A

[alerque]

Won't any existing parsers be invalidated anyway? They will immediately choke on any new content that they don't have handling for. The only thing this saves is messages, not parsers, by making sure legacy messages are easier to use with future parsers.

[aphillips]

The only thing this saves is messages, not parsers, by making sure legacy messages are easier to use with future parsers.

It's the other way around. Future messages would be compatible with legacy processors. The "legacy" parsers won't understand what the sigals mean and could produce resolution errors or perhaps formatting errors when they see the reserved sigals--rather than syntax errors (as they would now). However, this also allows for processors outside the runtime which don't care as much about meaning (such as translation tools) to say "this is a valid placeholder that I need to protect, although I don't know what it does" vs. "this is broken".

Implementers also would know what we would (and thus would not) do in future additions to the standard. We can require implementations to handle unknown reserved markers in a specific way other than just "syntax error".

We could also expose one or more of these sigals for private use, although what that would mean vs. just using a function I'm not sure offhand.

[eemeli]

I like this. I think we'd want to also reserve ~, *, and /.

This will also need a specified error and a fallback definition.

[macchiati]

Might review the characters from \p{P}\p{S}&[\x{0}-\x{ff}] for reserving outside of literal text.

…

On Wed, Mar 1, 2023 at 12:15 PM Eemeli Aro ***@***.***> wrote: I like this. I think we'd want to also reserve ~, *, and /. This will also need a specified error and a fallback definition. — Reply to this email directly, view it on GitHub <#360 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMGOXRHNNZLRKMZGC73WZ6U5TANCNFSM6AAAAAAVMKKNLM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[aphillips]

To @macchiati's point, it's probably reasonable to just reserve everything (unused ASCII punctuation plus the range U+00A1-U+00BF) that can't be used to start names anyway. Reserving the characters doesn't cost us anything.

@eemeli I did omit * because we are using it in our syntax already and because it looks like a wildcard (or block comment). ~ is a good callout. I omitted / but probably should not have.

Omitting a character from the list is basically a promise never to give the character meaning as a name introducer in the future. The likelihood that we'll use up very many of these seems small, but we should preserve a fair amount of choice for future needs. On the other hand, I omitted several from my proposal on purpose: dot, comma, apostrophe, double-quotes, and the remaining enclosing punctuation [/], (/). I also omitted +/- pending our discussion of markup. Lowline (_) is already part of name.

[aphillips]

merged #374 today