(Design) Document the open questions around standalone markup #542

stasm · 2023-11-29T13:06:28Z

Right now, the doc recognizes that standalone elements are still an open question, but the documented proposal goes one step further: it suggests to use the registry to encode the standalone aspect.

In this PR, I list all previously suggested alternatives on an equal level, since no decision has been made about them yet.

eemeli

Overall it's fine, though see inline comments. On the standalone alternatives, it would be good to point out somewhere that the second and third are valid no matter what we decide about the syntax.

exploration/open-close-expressions.md

eemeli · 2023-11-29T13:54:35Z

exploration/open-close-expressions.md

@@ -178,7 +183,7 @@ This is {#strong}bold{/strong} and this is {#img alt=|an image|}.
 Markup names are _namespaced_ by their use of the pound sign `#` and the forward slash `/` sigils.
 They are distinct from `$variables`, `:functions`, and `|literals|`.

-This allows for placeholders like `{#b}`, `{#img}`, and `{#a title=|Link tooltip|}`.
+This allows for placeholders like `{#b}`, and `{#a title=|Link tooltip|}`.


The proposal does allow for {#img}. It just doesn't differentiate the formatted parts of e.g. {#b} and {#img} from each other except by the name.

Suggested change

This allows for placeholders like `{#b}`, and `{#a title=|Link tooltip|}`.

This allows for placeholders like `{#b}`, `{#img}`, and `{#a title=|Link tooltip|}`.

I would still prefer to have this change reversed. The currently proposed design does factually allow for a placeholder like {#img}, even if it does not strictly communicate in the syntax that this markup element does not expect a closing element {/img}.

eemeli · 2023-11-29T13:57:40Z

exploration/open-close-expressions.md

@@ -195,13 +200,40 @@ Markup is not valid in _declarations_ or _selectors_.

 * Introduces two new sigils, the pound sign `#` and the forward slash `/`.



Something like this should still be included as a con of the currently-proposed solution.

Suggested change

* As in HTML, differentiating "open" and "standalone" placeholders requires

additional information not encoded in the bare syntax.

I would not say "elements" (use placeholder instead). HTML has elements, but we don't.

A different way of saying this is:

Suggested change

There is no difference between an "open" and a "standalone" placeholder in terms of syntax.

It is up to the implementation as to whether a given placeholder is treated

as an opening item or an isolated item in the output.

Updated my suggestions above with s/elements/placeholders/.

I disagree with this suggestion, at least until we discuss the open questions related to standalone elements. The whole point of this PR is to emphasize that the current proposal is incomplete without our answering these questions.

Note that the "hash and slash" proposal did include a way to express standalone markup, and so did the "plus and minus" proposal. I'm concerned that suddenly we're discussing a different proposal in which standalone markup has been removed.

How about we add the following con instead of the ones you both proposed?

Suggested change

- Doesn't answer how to handle standalone markup; see below.

@stasm I don't think that's quite correct. The proposal does say how to handle standalone markup, but not in a syntactically distinct way.

I think there is a case to be made for separate standalone syntax, as it allows relatively naive implementations to be created that don't have a list of standalone vs. spanning markup.

Thus I think your "con" would be better phrased as:

Suggested change

- Standalone placeholders are not syntactically distinct from opening placeholders;

implementations require out-of-band information to distinguish standalone

from opening placeholders.

I hope I don't seem contrary but £ is called a pound sign. # is called an octothorpe.
Coming from a country that used to have pounds as a unit of currency.

@kenguest In the specification it would be U+0023 NUMBER SIGN # because that is its Unicode name. Octothorpe and pound sign are both alternate names 😉

The proposal does say how to handle standalone markup, but not in a syntactically distinct way.

But it’s not the same proposal that was called “hash and slash” which was added in #517.

I object to how #535 removed the standalone syntax from that proposal, quoting consensus which was not officially discussed.

I remain somewhat confused by the desire to remove this documented "con" of the currently proposed design. This PR and our discussions around this issue have made it clear that there's still desire to discuss standalone markup, but the proposed design itself is not changed here. It still does not differentiate in syntax standalone and open placeholders. Why can't we mention that? Either with the original text, or that proposed by @aphillips or myself earlier in this thread.

On a somewhat meta level, in reply to @stasm from #542 (comment):

The whole point of this PR is to emphasize that the current proposal is incomplete without our answering these questions.

This approach makes this seem like this PR is attempting re-re-define how we do design and how we work with design documents. Thus far, we've been able to iterate on these docs and their proposed solutions while keeping them internally coherent. Witness for instance the changes to this document and its proposed design. Why can't we continue doing so?

We clearly intend to discuss standalone markup further; I don't see how leaving this out helps that. Or are you suggesting that "standalone" and "open" using the same syntax is not a disadvantage of the proposed design?

I remain somewhat confused by the desire to remove this documented "con" of the currently proposed design. This PR and our discussions around this issue have made it clear that there's still desire to discuss standalone markup, but the proposed design itself is not changed here. It still does not differentiate in syntax standalone and open placeholders.

It used to, however, right? And then #535 changed it, and I’m trying to object how it did it. See my comment in that PR.

I appreciate the attempt to limit the scope of the proposal to just open and close, but I also note that by doing so, we effectively make a decision about standalone.

I’d prefer it if the doc included {#foo/}, as it used to previously. But I also acknowledge that we will discuss this further on Monday.

Since I’m away this week, I’ll defer to you and @aphillips. And since I’m typing this on my phone, would you mind taking over so that we’re able to merge this PR soon? Thanks!

Co-authored-by: Eemeli Aro <[email protected]>

aphillips

Some suggestions

aphillips · 2023-11-29T15:11:52Z

exploration/open-close-expressions.md

+
+* Introduce additional syntax inspired by XML: `{#foo/}`.
+
+  This approach encodes the _standalone_ aspect in the data model, and doesn't rely on external sources of truth,


Not just the data model: it's in the syntax too!

This is mentioned in the same sentence, which continues onto the next line: at the expense of introducing new syntax, which is admittedly a bit clunky.

aphillips · 2023-11-29T15:17:14Z

exploration/open-close-expressions.md

+  (e.g. the structure of the source to which the message is applied).
+
+  This approach offers a cleaner syntax at the expense of relying on external sources of truth for telling open and standalone elements apart.
+  Without the access to these sources of truth (e.g. a custom registry), standalone markup will be interpreted as open elements.


Without access to some implementation, all forms of markup are just so much noise.

I'm slightly sad that we didn't show using namespaces in the design document, since I suspect that {#html:span id=foo}I am spanned{/html:span} is friendlier to tools, given {#ttml:span id=foo}I am timed text{/ttml:span} exists too. Namespacing suggests how implementations would solve the problem of "external knowledge". There's no way that any of this works without some form of external information anyway.

Yes, but you can get a lot of mileage without an implementation, just working with the data model, especially in tooling.

I agree that html:span would be much friendlier to tools, and probably also to translators. Note that I do mention namespacing in the following bullet, the one about re-using regular functions for standalone markup (:html:img).

I think namespaces are important for markup regimes to work. They won't be built-in to the default registry, so there needs to be a way to plug them in and for tools and implementations to know what they mean.

aphillips · 2023-11-29T15:22:08Z

exploration/open-close-expressions.md

+  In this approach there's also no way to enforce the lack of operands to standalone markup at the syntax level.
+  Instead, access to a custom registry is required to know that `:img` does not accept any operands.


I would say this differently.

Suggested change

In this approach there's also no way to enforce the lack of operands to standalone markup at the syntax level.

Instead, access to a custom registry is required to know that `:img` does not accept any operands.

In this approach, there's no way to distinguish standalone markup for regular functions.

This also means that there is no way to enforce a lack of operands for standalone

(as we do for open/close).

The open/close placeholders are default ignorable in certain formatting operations

because they can be distinguished from normal placeholders.

With `:img`, the placeholder would appear in all formatting operations, possibly as the

replacement "logo" when unsupported.

In this approach, there's no way to distinguish standalone markup for regular functions.

There is, though: the registry. This is why it's important to call out that there's no way at the syntax level (aka the data model level).

The open/close placeholders are default ignorable in certain formatting operations
because they can be distinguished from normal placeholders.

The ignoring is performed at runtime, which means that it can be implemented by functions rather than the implementation. For example, {:html:img} could be a function call which formats to an empty string, rather than be skipped by the implementation during formatting. So we can still distinguish standalone markup from regular function, but only by calling them.

So I think the real con here is that {:html:img} needs to be called at runtime for the implementation to determine that it's a standalone markup placeholder.

How about this:

Suggested change

In this approach there's also no way to enforce the lack of operands to standalone markup at the syntax level.

Instead, access to a custom registry is required to know that `:img` does not accept any operands.

In this approach there's also no way to enforce the lack of operands to standalone markup at the syntax level, as we do for open/close.

Instead, access to a custom registry is required to know that `:img` does not accept any operands.

Similarily, the only way to know at runtime that an expression produces markup would be to call the expression's function.

It's not possible to deduce this fact from the data model prior to the formatting operation.

I'd call out that nothing would or should prevent developers from writing the function :html:img (or any other element), should they so desire. So what we're really debating is the addition of syntactically distinct spannable placeholders.

... but using Unicode code points and names instead of being super informal. This is strictly an editorial change.

stasm · 2023-12-06T14:48:43Z

@aphillips @eemeli Are you satisfied with my answers to your comments? Can we merge this to better explain the controversy around the standalone markup before to the next meeting?

eemeli

I still have two open questions here; see inline threads.

eemeli · 2023-12-07T12:54:55Z

exploration/open-close-expressions.md

@@ -178,7 +183,7 @@ This is {#strong}bold{/strong} and this is {#img alt=|an image|}.
 Markup names are _namespaced_ by their use of the pound sign `#` and the forward slash `/` sigils.
 They are distinct from `$variables`, `:functions`, and `|literals|`.

-This allows for placeholders like `{#b}`, `{#img}`, and `{#a title=|Link tooltip|}`.
+This allows for placeholders like `{#b}`, and `{#a title=|Link tooltip|}`.


I would still prefer to have this change reversed. The currently proposed design does factually allow for a placeholder like {#img}, even if it does not strictly communicate in the syntax that this markup element does not expect a closing element {/img}.

eemeli · 2023-12-07T14:07:01Z

exploration/open-close-expressions.md

@@ -195,13 +200,40 @@ Markup is not valid in _declarations_ or _selectors_.

 * Introduces two new sigils, the pound sign `#` and the forward slash `/`.



I remain somewhat confused by the desire to remove this documented "con" of the currently proposed design. This PR and our discussions around this issue have made it clear that there's still desire to discuss standalone markup, but the proposed design itself is not changed here. It still does not differentiate in syntax standalone and open placeholders. Why can't we mention that? Either with the original text, or that proposed by @aphillips or myself earlier in this thread.

On a somewhat meta level, in reply to @stasm from #542 (comment):

The whole point of this PR is to emphasize that the current proposal is incomplete without our answering these questions.

This approach makes this seem like this PR is attempting re-re-define how we do design and how we work with design documents. Thus far, we've been able to iterate on these docs and their proposed solutions while keeping them internally coherent. Witness for instance the changes to this document and its proposed design. Why can't we continue doing so?

We clearly intend to discuss standalone markup further; I don't see how leaving this out helps that. Or are you suggesting that "standalone" and "open" using the same syntax is not a disadvantage of the proposed design?

mihnita · 2023-12-09T00:20:50Z

I really think that we need a standalone concept.

Traditionally HTML got away with it because it was "permissive" by design.
And because every browser hard-coded somewhere the "knowledge" that img, and br, and hr and so on are in fact standalone.

And once you have a tree, then a node without kids is "standalone"

But this does not work for unknown tags. Which MF2 allows.
Even HTML allows custom tags now.
So what does This is <foo> means? Is foo standalone or not?

Does it matter?
Yes, it does.

For example This is <foo lang=ar> something or This is <foo dir=rtl> something ?
Is "something" Arabic, or RTL?
We don't know, because we don't know anything about <foo>
So we don't know how to post-process the text after <foo>, for example.

We also don't know how to leverage.
If we translated a week ago text1 text2 text3, and now we get for translation text1 text2 text3, we know it is safe to take the old translations and algorithmically change the ... to ...
Because we replace open-close with open-close.

But is unsafe to replace open with standalone, or standalone with open.

We know that with translation the sloppier we are the less we can trust that leveraging and validation work properly.

And we also know that we want to be able to use mf2 markup with more than html.
So we should not prevent certain use cases "by design", because we emulated html.

eemeli

@stasm I merged from main to resolve a conflict, and have re-added below the two changes I've been asking for in previous threads. Applying them would let us merge this PR, and then follow this with a next one that proposes some actual change to the proposed design.

As is, this PR effectively blocks concrete standalone syntax proposals from being presented, and that seems contrary to what you're looking for?

exploration/open-close-placeholders.md

stasm · 2023-12-13T20:24:25Z

Thanks, @eemeli.

As I've expressed previously, I feel that something was lost in our communication or perhaps in translation — I don't really understand why the currently proposed solution (after your suggestions) doesn't include the standalone syntax. It was part of both the plus/minus alternative and the hash/slash alternative before #535.

However, it seems that discussing this further won't resolve this, and I'm more interested in seeing this PR land and then following up with updates to the design doc.

Co-authored-by: Eemeli Aro <[email protected]>

aphillips · 2023-12-13T23:07:55Z

From on- and offline conversation, merging these changes. Don't feel that we can't iterate the design doc from here 😉

Document the open questions around standalone markup

81de317

stasm mentioned this pull request Nov 29, 2023

Update open-close design doc to accept the hash & slash alternative #535

Merged

eemeli requested changes Nov 29, 2023

View reviewed changes

Update exploration/open-close-expressions.md

d9feb62

Co-authored-by: Eemeli Aro <[email protected]>

aphillips reviewed Nov 29, 2023

View reviewed changes

aphillips added syntax Issues related with syntax or ABNF design Design document or issues related to design labels Dec 1, 2023

aphillips changed the title ~~Document the open questions around standalone markup~~ (Design) Document the open questions around standalone markup Dec 3, 2023

stasm self-assigned this Dec 3, 2023

Address comment about Unicode character references

f8bf04d

... but using Unicode code points and names instead of being super informal. This is strictly an editorial change.

stasm mentioned this pull request Dec 6, 2023

[Discussion] {{Spannables}} #537

Closed

eemeli requested changes Dec 7, 2023

View reviewed changes

Merge branch 'main' into standalone-open-questions

246f23d

eemeli requested changes Dec 13, 2023

View reviewed changes

exploration/open-close-placeholders.md Outdated Show resolved Hide resolved

exploration/open-close-placeholders.md Show resolved Hide resolved

Apply suggestions from code review by @eemeli

0e8d167

Co-authored-by: Eemeli Aro <[email protected]>

eemeli approved these changes Dec 13, 2023

View reviewed changes

aphillips merged commit 2a3b88e into main Dec 13, 2023

aphillips deleted the standalone-open-questions branch December 13, 2023 23:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(Design) Document the open questions around standalone markup #542

(Design) Document the open questions around standalone markup #542

stasm commented Nov 29, 2023 •

edited

Loading

eemeli left a comment

eemeli Nov 29, 2023

eemeli Dec 7, 2023

eemeli Nov 29, 2023 •

edited

Loading

aphillips Nov 29, 2023

eemeli Nov 29, 2023

stasm Dec 3, 2023

aphillips Dec 3, 2023

kenguest Dec 3, 2023

aphillips Dec 4, 2023

stasm Dec 6, 2023 •

edited

Loading

eemeli Dec 7, 2023

stasm Dec 7, 2023

aphillips left a comment

aphillips Nov 29, 2023

stasm Dec 3, 2023

aphillips Nov 29, 2023

stasm Dec 3, 2023

stasm Dec 3, 2023

aphillips Dec 3, 2023

aphillips Nov 29, 2023

stasm Dec 3, 2023

aphillips Dec 3, 2023

stasm commented Dec 6, 2023

eemeli left a comment

eemeli Dec 7, 2023

eemeli Dec 7, 2023

mihnita commented Dec 9, 2023

eemeli left a comment

stasm commented Dec 13, 2023

aphillips commented Dec 13, 2023

	This allows for placeholders like `{#b}`, and `{#a title=\|Link tooltip\|}`.
	This allows for placeholders like `{#b}`, `{#img}`, and `{#a title=\|Link tooltip\|}`.

		@@ -195,13 +200,40 @@ Markup is not valid in _declarations_ or _selectors_.

		* Introduces two new sigils, the pound sign `#` and the forward slash `/`.



	* As in HTML, differentiating "open" and "standalone" placeholders requires
	additional information not encoded in the bare syntax.

+There is no difference between an "open" and a "standalone" placeholder in terms of syntax.
+It is up to the implementation as to whether a given placeholder is treated
+as an opening item or an isolated item in the output.


	- Doesn't answer how to handle standalone markup; see below.

+- Standalone placeholders are not syntactically distinct from opening placeholders;
+  implementations require out-of-band information to distinguish standalone
+  from opening placeholders.


		* Introduce additional syntax inspired by XML: `{#foo/}`.

		This approach encodes the _standalone_ aspect in the data model, and doesn't rely on external sources of truth,

		In this approach there's also no way to enforce the lack of operands to standalone markup at the syntax level.
		Instead, access to a custom registry is required to know that `:img` does not accept any operands.

-  In this approach there's also no way to enforce the lack of operands to standalone markup at the syntax level.
-  Instead, access to a custom registry is required to know that `:img` does not accept any operands.
+In this approach, there's no way to distinguish standalone markup for regular functions.
+This also means that there is no way to enforce a lack of operands for standalone
+(as we do for open/close).
+The open/close placeholders are default ignorable in certain formatting operations
+because they can be distinguished from normal placeholders.
+With `:img`, the placeholder would appear in all formatting operations, possibly as the
+replacement "logo" when unsupported.

(Design) Document the open questions around standalone markup #542

(Design) Document the open questions around standalone markup #542

Conversation

stasm commented Nov 29, 2023 • edited Loading

eemeli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eemeli Nov 29, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stasm Dec 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aphillips left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stasm commented Dec 6, 2023

eemeli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mihnita commented Dec 9, 2023

eemeli left a comment

Choose a reason for hiding this comment

stasm commented Dec 13, 2023

aphillips commented Dec 13, 2023

stasm commented Nov 29, 2023 •

edited

Loading

eemeli Nov 29, 2023 •

edited

Loading

stasm Dec 6, 2023 •

edited

Loading