Consider pushing implicit conversions down

[stereotype441]

The Dart language currently supports the following kinds of implicit conversions:

• Conversion from an integer literal to an equivalent double literal
• Conversion from dynamic to some other type via an implicit cast
• Conversion from an interface type containing a .call method to a function type, by implicitly tearing off .call

The first conversion only happens on integer literals, and is based on the context type. The other two conversions are currently only performed at sites where the spec demands assignability, and is based on the type being assigned to. The language team is considering changing the behavior so that all implicit conversions are based on context type, and happen at the innermost expression where they're applicable, regardless of assignability constraints. (We would retain the restriction that int-to-double conversion only happens to literals).

Here are some examples to illustrate the user-observable consequences of this change (changes are in bold). (Assume F is a function type, f is an expression of type F , fq is an expression of type F?, C is a callable class with a foo method of type void Function() and a call method of type F, c is an expression of type C, cq is an expression of type C?, and d is an expression of type dynamic. Also assume there's a declaration of Object o = f; in scope, with declared type Object and promoted type F):

• (1) F x = b ? c : f; will be interpreted as F x = b ? c.call : f;, which will work. Today it's a compile time error because b ? c : f has static type Object, which is not assignable to F.
• (2) F x = b ? c : d; will be interpreted as F x = b ? c.call : (d as F), which will work, unless b is false and d's runtime type is not F. Today this code is accepted, because b ? c : d has static type dynamic, but it fails at runtime if b is true, because there is no implicit tearoff of call.
• (3) F x = b ? "x" : d; won't change. Today it's interpreted as F x = (b ? "x" : d) as F;. This will change to F x = b ? ("x" as F) : (d as F);, which is indistinguishable.
• (4) o = c; will be interpreted as o = c.call;, retaining the promotion on o. Today this code is accepted, but there is no implicit tearoff of call, so it demotes o to Object, which is probably not what the user wants.
• (5) o = d; will be interpreted as o = (d as F);, which retains the promotion on o, but fails at runtime if d's runtime type is not F. Today this code is interpreted as o = (d as Object);, which demotes o.
• (6) F x = cq ?? f; won't change. This code is rejected because we don't imply implicit .call tearoffs to nullable expressions, so cq ?? f has type Object, which is not assignable to F.
• (7) var x = cq ?? f; wont' change. This code is accepted and interpreted as Object x = cq ?? f; (no downcasts or implicit tearoffs).
• (8) var x = cq ?? d; will be interpreted as C? x = cq ?? (d as C?), which has a more specific type, but will fail at runtime if cq is null and d's runtime type is not C?. Today this code is interpreted as dynamic x = cq ?? d;, which never fails but has a less specific type.
• (9) var x = fq ?? d; will be interpreted as F? x = fq ?? (d as F?), which has a more specific type, but will fail at runtime if fq is null and d's runtime type is not F?. Today this code is interpreted as dynamic x = fq ?? d;, which never fails but has a less specific type.
• (10) F x = fq ?? c; will be interpreted as F x = fq ?? c.call;, which will work. Today it's a compile time error because fq ?? c has static type Object, which is not assignable to F.
• (11) var x = fq ?? c will be interpreted as F x = fq ?? c.call;, which is more useful. Today it's accepted and interpreted as Object x = fq ?? c, which doesn't fail, but also leaves x in a state where it's hard for the user to do anything useful with it, because at runtime it might either be an instance of C or a function.
• (12) F x = c..foo(); will be a compile-time error because it's interpreted as F x = c.call..foo();, and c.call has type F, which has no method foo. Today it's interpreted as F x = (c..foo()).call;, which works.
• (13) F x = (c); won't change. Today it's interpreted as F x = (c).call;. This will change to F x = (c.call);, which is indistinguishable.

(Thanks @lrhn for working through most of these examples).

A rough summary of this might be:

• For the conditional operator (cases (1) through (3)) it's a clear win: code that leads to compile-time and runtime errors today will "just work".
• For assignments to local variables (cases (4) and (5)) it's probably a win; I suspect that retaining the promotion is more likely to be what the user wants in most circumstances. But it's not as clear cut, since the downside is a tighter downcast for (5), which might lead to a runtime failure if it's not what the user intended.
• For the null-aware operator (cases (6) through (11)) it's also not 100% clear whether it's an improvement. Cases (10) and (11) seem like clear wins, but cases (8) and (9) could lead to runtime failures. Personally I think it's still a worthwhile tradeoff, because even though (8) and (9) could lead to runtime failures, they also lead to the variable x having a more specific type, which should help reduce dynamism (and catch errors at compile time) in the code that follows.
• For cascades (case (12)) it's a clear loss of functionality, although IMHO it's probably an acceptable loss, since I think doing this kind of thing is rare, and there's an easy workaround (the user can just explicitly write F x = (c..foo()).call;).

[lrhn]

I think number 3 is slightly incorrect.

• (3) F x = b ? "x" : d; becomes F x = b ? "x" : (d as F); which is a static error because UP(String, F) is Object which isn't assignable to F. There is no implicit downcast for "x".

That's progress! It used to be a dynamic error. With this change, a dynamic in one branch does not infect the other branches of the conditional expression.

[Jetz72]

I presume enabling strict casts for dynamics in the analyzer settings will avoid the potential runtime errors introduced by (5), (8), and (9)? I don't like 5 in particular - getting a runtime error for trying to store a value with a type in a variable you explicitly declared with a matching type. Can something like that happen currently?

Also out of curiosity, was there a reason why a callable class doesn't act as an actual implementation of the respective Function type? Like c can behave as F in many cases by implicitly becoming c.call, but c is F is false and c as F doesn't work.

[srawlins]

I presume enabling strict casts for dynamics in the analyzer settings will avoid the potential runtime errors introduced by (5), (8), and (9)?

Yes, I believe all three.

[lrhn]

@Jetz72 Yes, there is a good reason a callable class cannot implement a function type directly. We tried during the Dart 2.0 type system design, and it just broke soundness too much. I don't remember the details (I'm sure some still do), but it was related to function types being contravariant in some of its subtypes, and generics being covariant, and together those could make two different types mutual subtypes and collapse the type hierarchy.

Doing implicit .call tear-off was the compromise that made existing code keep working.

[eernstg]

One reason to stop supporting the fully general approach where a class implementing a method named call is a subtype of the signature of that method is recursion:

class C {
  void call(C c) {}
}

If C should be a full-fledged function type then it would be void Function(C), but that's the same thing as void Function(void Function(void Function(...)))). We did not want to deal with recursive types like that. Perhaps there's a nice model out there relying on lots of heavy maths, but it wasn't obvious that all the work needed to make this thing well-defined would also be worth the trouble in practice.

The other main reason was (as far as I know) that it is possible to optimize primitive function objects. For instance, if a function is a different kind of entity than a regular object then we can get access to the code of the function using a special lookup protocol (for instance, the address of the code could be stored at offset 12 in the heap entity layout). If we constrain ourselves to use exactly the same representation as we do for regular objects then (presumably) the invocation of a function object would be the same thing as an invocation of the call method of a regular object. That step would involve a normal object-oriented dispatch, and that's likely to involve more than a single memory read, in order to find the address to jump to.

[stereotype441]

In dart-lang/sdk#49106 (comment), lrhn@ points out an example of a situation where it would be undesirable to switch to using context type for implicit downcasts:

class C {
  int operator [](A key) => ...;
  void operator []=(B key, int value) { ... }
}
test(C c, dynamic d) {
  c[d] += 1;
}

This is currently desugared by the CFE to:

let final C #t1 = c in let final dynamic #t2 = d in #t1[#t2 as B] = #t1[#t2 as A] + 1;

which shows that the downcast type is currently not determined by the context but instead by the assignability requirements implied by compound assignment desugaring.

If we make the change that currently seems to be favored in dart-lang/sdk#49106 (change the context type to be the GLB of the two key types), then in the case where A and B have multiple possible maximal lower bounds, we may end up choosing a GLB which is not satisfied by the actual value, yet where the actual value does satisfy both key types. So if we don't want to break this case, we should not use the GLB for downcasts.