[BUG] comma-operator is irrelevant when parsing template-argument-list

[seanbaxter]

https://github.com/hsutter/cppfront/wiki/Design-note%3A-Unambiguous-parsing#a-first-match-wins-there-are-no-comma-expressions-and-a-relational-comparison-in-a-template-argument-must-be-parenthesized

In Cpp2 my intent is to make this a non-problem (famous last words!) by addressing this in two chunks:

• Cpp2 has no comma operator, so that removes the possibility of b,c being a comma-expression.
• Cpp2 requires a relational comparison expression in a template-argument-list to be parenthesized. (*)

In C++, b,c in a template-argument-list cannot be a comma-expression because the grammar sets the template-argument production to conditional-expression, which is higher precedence than expression, which is where the comma is parsed. The notes concerning comma-expression are irrelevant.

More generally, I don't see how context insensitivity is achieved. What is changed, other than requiring parens around comparisons in the template-argument-list?

f(temp_or_object<a, b>(c));

Is this disallowed Cpp2 syntax? It's definitely ambiguous.

[seanbaxter]

Ah there's another problem in your parsing document.

if (auto e = expression(false)) {   // disallow unparenthesized relational comparisons in template args
    term.arg = std::move(e);
}
else if (auto i = id_expression()) {
    term.arg = std::move(i);
}

The id_expression case is unreachable, because id-expression is a terminal of expression. This code doesn't do anything to disambiguate.

[filipsajdak]

Any example of code that will suffer from that implementation?

[seanbaxter]

f(temp_or_object<a, b>(c)); suffers from it. The grammar isn't context-free.

[gregmarr]

Since it took me a while to spot the ambiguity here (at least partially because I was reading temp_or_object as temporary instead of template), just wanted to confirm that these are the possible interpretations and what parens would be required to disambiguate them:

f((object_name < a), (b > (c)));
f((template_name<a, b>(c)));

[filipsajdak]

Ah, now I get it.

[filipsajdak]

I am thinking about simplifying this. Maybe it would be good to introduce a rule that operators need to be separated with whitespace and if you want to write a template you need to put < directly after the id-expression.

a < b; // correct form of calling < operator
a< b; // error
a<b;   // error

temp<int> i1; // ok
temp<
  int
> i2; // ok
temp <int> i3; // error

I thought about it as I was confused reading your example and my second thought was that we shall support syntax that can be easily interpreted by humans and by compilers. So, maybe we can limit the syntax to only acceptable forms and make it clear what we are looking at.

[seanbaxter]

That may be a valid engineering solution. I don't care one way or another. But making C++ white space sensitive would be received as a provocation. You'd set off a never-ending amount of bickering.

[filipsajdak]

Well, it is all about having fewer things to learn, and in most of the coding guidelines we have such rules already:

• https://codingart.readthedocs.io/en/latest/c/Formatting.html#c-format-usage-of-spaces
• https://google.github.io/styleguide/cppguide.html#Horizontal_Whitespace
• https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#nl15-use-spaces-sparingly (not exactly this - but most of the code written on CppCoreGuidelines is following the rule of having space before and after binary operators; described well in the first link),

For sure that is something that will increase the readability of the code and will make writing tools easier as there will be fewer ambiguity cases.

That will save us from the code like:

return a*pa**ppa**; // 8

which would be forbidden in favor of

return a * pa* * ppa**; // 8

and I would love to avoid that kind of ambiguity and puzzle-like code.

[gregmarr]

Previously C++ had some whitespace sensitivity requirements, namely that std::vector<std::vector<long>> had to be written as std::vector<std::vector<long> > for it to parse properly. That was eventually removed. So on the one hand, there is precedence for whitespace sensitivity. On the other hand, there is also precedence for a desire to remove whitespace sensitivity.

Personally I prefer having whitespace around binary operators, and not having whitespace between a unary operator and its operand. I also prefer not having whitespace between template and < and between my_funcname and (, but I know others prefer having the space there, so I'm not sure that's a method that people can get behind unless there is literally no other option than changing the template bracket type to be something that is already only used in a balanced form.

[seanbaxter]

Please don't give prefix, infix and postfix operators whitespace-sensitive semantics.

Look at the SPECS project from the 90s, which is a context-free reskin of C++. Start there and move it forward to include modern features. Will have to reimagine the declarator syntax, cast-expression, etc. Don't expect it to resemble C++. Can't just wave your hands over C++ and declare it to be context-free, which is what cppfront does currently.

I don't think a context-free reskin of C++ is very useful. It presents a big training barrier with almost no benefit. OTOH, nailing down the semantics of the parameter passing directives would be super useful. Designing memory safety types (ref and mutref) would be super useful.

[gregmarr]

Please don't give prefix, infix and postfix operators whitespace-sensitive semantics.

I think I'm leaning that way as well.

It presents a big training barrier with almost no benefit.

I disagree on the "almost no benefit" here based on how difficult parsing is here. Now, I'm not generally a tool writer, though in a past life I was primary author on some language parsers, so I feel the pain of tool writers having to create context-sensitive parsers. If this lowers the burdens for tool writers and also makes it easier for people to understand, then it's a plus. I have learned multiple programming languages on my own over the last 40 years, I've been writing C++ professionally for 25+ years, have been following standards processes and language development, attending C++ conferences for more than a decade, and it still took me a relatively long time to see the ambiguity in f(temp_or_object<a, b>(c));. I can't imagine now what it's like for new coders having to learn it.

Having said that, I do appreciate the discussion and alternate points of view, so keep it coming.

[seanbaxter]

If this lowers the burdens for tool writers

It doesn't really matter for compiler writers. C++ is hard to translate for different reasons. I'm not saying no value, but compared to parameter directives or memory safety...?

[jcanizales]

Will have to reimagine the declarator syntax, cast-expression, etc. Don't expect it to resemble C++

I mean that's already the case for cppfront. Declarations use a colon and the type goes after the identifier, the recommended way to cast is to use as (and it's not clear old-style parenthesis cast should be allowed, given how the current C++ guidelines suggest to avoid it). If you look at the example files, it only resembles C++ in the places where that resemblance doesn't compromise the other goals of the project. And that makes sense: if resemblance had priority, why try a new syntax at all.

It doesn't really matter for compiler writers.

Per his comments on the design of the language, that's not Herb's experience (and he should know a thing or two about it 🙂 ), but more importantly not all tool writers are writing compilers. Some are writing refactoring tools. Some are writing intellisense-like plugins. Some are writing simple syntax highlighters. Some are writing static analyzers. None of them have to translate C++, but still are impeded by its syntax.

[jcanizales]

I do agree though that being context-free won't be the main driver for adoption: better tools are IMO just a side benefit. Care has to be taken to not end up more verbose than C++, for example (except in cases where the benefit is very clearly obvious to the user). I think SPECS failed in that respect. Swift and Kotlin had that easy: Java and Objective-C were super cumbersome to start with, and just the ability to call existing libraries with a modern syntax was a relief. (The latter, btw, I think is crucial for adoption too: using C++ libraries within cppfront should be seamless, should feel and look like cppfront, and should feel like an improvement).