Meta issue tracking implementation of generic function type syntax in Dart 1.5

[leafpetersen]

This is a tracking bug for implementing support for generic function type syntax (generic typedef) in Dart 1.5.

Proposal: https://gist.github.com/eernstg/ffc7bd281974e9018f10f0cb6cfee4aa

The corresponding individual issues will be listed here:

• Language team decision on syntax (private issue): dart-lang/dart-lang-evolution#88
• Analyzer implementation issue: Implement generic function type syntax in the analyzer #27969
• VM implementation issue: Implement generic function type syntax in the VM. #27966
• Dart2js implementation issue: Implement generic function type syntax in dart2js #27967
• DDC implementation issue: Implement generic function type syntax in DDC #27971
• dartfmt issue: Implement support for generic function types dart_style#563
• dartdoc issue: Implement generic function type syntax in dartdoc dartdoc#1321
• Make sure changelog has an entry for this

• Final specification issue: Write specification for generic function type syntax #27970

Description

Dart 2 supports generic methods and generic tear-offs (or anonymous functions). This means that a function can request a function that is generic:

// Takes a function `f` that is generic on T.
void foo(A<T> constructorForA<T>()) {
  A<int> x = constructorForA<int>();
  A<String> x = constructorForA<String>();
}

It is straightforward to add the syntax to the existing syntax for function types for parameters (as was done in the example). However, we don't have an easy way to do the same for fields or local variables:

// Does *not* do what we want it to do:
typedef A<T> ConstructorForAFun<T>();

A f;  // A function that returns an `A<dynamic>`.

We already used the most consistent place for the generic method argument as a template argument to the typedef itself. If we could go back in time, we could change it as follows:

typedef<T> List<T> TemplateTypedef();
TemplateTypedef<int> f;  // A function that returns a List<int>.
TemplateTypedef f;  // A function that returns a List<dynamic>.

typedef List<T> GenericTypedef<T>();
GenericTypedef f;  // A function that is generic.
List<int> f<int>();
List<String> f<String>();

Given that this would be a breaking change we have considered alternatives and are now exploring new typedef syntax altogether. The idea is to provide syntax that can also be used for locals and fields at the same time.

For example:

typedef F = (int) -> void;  // Function from (int) to void.
typedef F<T> = () -> List<T>;  // Template Typedef.
typedef F = <T>(T) -> List<T>;  // Generic function from T to List<T>.

This way we would solve 3 issues at the same time:

• users often forget the parameter name and expect typedef void F(int) to be a function that takes an int.
• users can write the function type of fields and locals without needing to write a typedef.
• of course: we can write the type of a generic function.

While investigating different syntax we found that the following points need consideration:

• nullability: the syntax must be nullable without too much hassle:

    (int)->int?;  // A function that is nullable, or that returns a nullable integer?
   Problem disappears with <-
   int <- (int)? ; vs int? <- (int) 

• readability
• interactions with union types (if we ever want them).

[a-siva]

@crelier

[floitschG]

Status update:

I have updated the issue description for more context.
We are still mainly exploring the "how does it look and feel" parts of the syntax (while still paying attention to how the parser should work).

The following CLs show explorations:

https://codereview.chromium.org/2439573003/
https://codereview.chromium.org/2466393002/

[crelier]

lgtm with a slight preference for the <- variant.

[leafpetersen]

status: Prototyping several options, @floitschG is driving.

[floitschG]

Latest proposal:

typedef F = void Function(int);

// also works for field or local types:
final int Function(String, bool) myFun = ...;

• works nicely with the current Function type.
• very similar to the existing parameter types. Hopefully this also means that the parser can reuse parts of the parameter code.
• longer than <- or ->.

Here is a CL that experiments with it: https://codereview.chromium.org/2482923002/

Note: we would still allow Function(int x). However, Function(int) should be a function that takes int (and not a function that takes a dynamic argument of name int).
Eventually we would like to change the old syntax to do the same, but that requires some migration effort.

[crelier]

I do not like this latest proposal. The name Function is promoted to some kind of keyword used for parsing. Also, Function is not declared to be generic, therefore, using it to declare a non-generic typedef for a generic function will look like an error.

[floitschG]

It's not a perfect choice, but it has its advantages.

Dart already uses Function as general type for functions. It is true that Function was a real type, and the proposed function-type syntax shifts it into a keyword-like position, but there is a strong connection between the two. It's easy to see and understand that Function (without return and parameter types) is just like a reduced function-type syntax. It's not the same (as you correctly state), but it's very close to it.

It's true that the Function type/class isn't generic, but it also has no way to declare its return and parameter types. Clearly, the function-type syntax does more than just the Function type. It really is an anonymous subtype of Function. As such, it doesn't feel wrong to add generic functionality to it.

Personally, I'm not that much concerned about the fact, that Function now becomes a keyword for function types. On the contrary, I would expect this to improve the readability, since the new syntax is so close to the existing parameter types, and the meaning of the stand-alone Function fits the new interpretation.

  Function f;  // a function.
  Function(int x) f;  // a function that takes an int.
  double Function(int) f;  // a function that takes an int and returns a double.

I'm more concerned about the fact that it is longer, and that we intend to interpret one-identifier parameters differently in the new syntax than in parameter types in the old syntax: Function(int) is a function that takes an int and not a function that takes one dynamic argument of name int. However, I believe that the benefits outweigh the cost here. In fact, we would advocate to change the meaning of one-identifier types for old-syntax parameter types, too. This would be a breaking change, though, and would require a careful migration effort.

Just for completeness sake, here are some examples of how the syntax would deal with the different generic arguments (to the typedef or for the function).

typedef F = void Function(int);  // Function from (int) to void.
typedef F<T> = List<T> Function();  // Template Typedef.
typedef F = List<T> Function<T>(T);  // Generic function from T to List<T>.

[crelier]

I agree that having a special marker for function types has advantages and is easier to read. I only find the already defined "Function" a questionable choice for this marker, especially since the formal argument syntax is modified and not what you would expect after an identifier (i.e. regular function signature syntax). That's the reason I'd prefer a symbol rather than an identifier (or keyword), as in the previous proposals.

I agree that "<-" of the previous proposal is a bit hard to read when generic.
T <- <T>(T)
Maybe reusing the just retired # symbol would work better.
T #<T>(T)

As a side note, you will also need to specify the syntax for optional arguments (no default values allowed), both positional and named, something like:

void Function(int, [int, bool]);  // optional positional
void Function(int, {int i, bool b});  // optional named

[floitschG]

# is still used for symbols (which was one of the reasons we didn't like it for tear-offs. For example, it wasn't possible to tear off globals).

Yes. We will have to specify the syntax for the optional arguments. (This is the reason that Function<R, P1, P2> isn't even a real option). Your proposal looks right (and I don't think there are any corner-cases that are hiding).

I/We will think about using something else than Function, but I doubt that we will find something better. The obvious candidates, -> and <-, have already been considered, and most other choices are probably inferior.

Personally, I also tend to avoid using symbols for uncommon functionality. Function types are not extremely rare, but they are not common enough to warrant their own symbol.

[munificent]

I/We will think about using something else than Function, but I doubt that we will find something better

Would it be possible to add a new contextual keyword like function or fun or something? I'm guessing that would be a breaking change, but maybe it's not actually ambiguous in the grammar?

Personally, I also tend to avoid using symbols for uncommon functionality.

+1. Punctuation is almost always the wrong choice for new syntax unless it represents something almost all users already know, like using + for addition.

[lrhn]

The backwards-compatible advantage in using Function is nice, but not essential. If you can't omit the parameters, so that int Function x is any function returning an int, then the syntaxes will be completely disjoint. If you can omit the parameters, then I think we should definitely use Function as the keyword - then you can write Function Function Function x; and have something thrice callable :)

We can make Function no longer be a class. Nobody should implement Function anyway, so it's not a loss. The only complication is Function.apply which needs a namespace, but if we remove call and make function objects completely distinct from class instances, then they can have apply as an instance method.

[eernstg]

About omitted parameter lists in the <return-type> Function<formal-parameter-list> syntax: That would allow developers to specify a return type and leave the parameter list unspecified (that is: anything goes), which is exactly what was requested in, say, #26420.

We might decide to introduce syntax for variadic functions at some point, in which case this will become a shorthand for "the most general argument list", but even at that point it seems likely that such a shorthand would be convenient.

[eernstg]

In the informal spec (which will be made public when it has matured a bit) I've proposed a rename of this feature to 'Generalized type aliases' (because a limitation of this mechanism to handle generic function types only is artificial, and presumably counter productive).

[a-siva]

Could we add some tests cases for this so that the implementations can be verified.

[mit-mit]

Marking informal spec complete for implementation purposes based on #27970 (comment)

[mit-mit]

Moving to 1.22 for implementation.

[mit-mit]

Punting to 1.23 per email discussion

[floitschG]

We reverted the syntax for 1.22 and add a deprecation warning for class Function instead.
1.23 will then feature the actual syntax.

[leafpetersen]

Is this still on track for 1.23?

[bwilkerson]

I can't speak for anyone else, but I am actively working on the analyzer support.

[leafpetersen]

Any updates?

[crelier]

I am not sure who you are asking, but the corresponding VM issue has been closed on January 27.

[mit-mit]

@leafpetersen see detailed status in the two unchecked items in the top checklist

[jcollins-g]

For dartdoc, it turns out there's no need to do anything directly for the SDK release and the work that does need to be done isn't blocking any longer. I've dropped the 1.23 label from dart-lang/dartdoc#1321 accordingly, and there are more details there as to why.

[jcollins-g]

I had misunderstood a key piece of the feature, I recommend blocking this feature on dartdoc now. Dartdoc in the SDK won't crash without the upcoming fix, but it will display the typedef with the generic parameter missing.

[munificent]

dartfmt support is in flight.

[munificent]

dartfmt is fixed and in the repo.

[mit-mit]

@leafpetersen looks like this one is still missing in changelog.md?

[leafpetersen]

I left it out, since I wasn't 100% sure that it was landing, but it looks like maybe the dartdoc issue is resolved? If so, I can go ahead and write something up.

[leafpetersen]

There seems to still be quite a bit of missing functionality still in the analyzer implementation, see discussion at the end here: #27969 .

[mit-mit]

Can we go ahead and update changelog.md now?

[dgrove]

Any updates on this?

[dgrove]

@leafpetersen @floitschG can we please finish this for 1.24?