Conditional instance members and constructors

[eernstg]

In response to #2308, this is a proposal to add support for conditional instance members of classes and mixins, and conditional constructors of classes. The intuitive meaning of such members and constructors is that they are accessible when the type variables of the receiver are known to satisfy certain constraints, and they are inaccessible otherwise.

For example, using this feature, a sort method with no arguments can be available on a List<T> when T <: Comparable, but when we can not prove T <: Comparable, it is a compile-time error to invoke the method.

What we currently have is a sort method that will throw at run time if it runs sort(), and some elements aren't Comparable. This proposal would improve on the static type safety of such methods.

Similarly, a constructor may be valid when certain constraints on the class type variables are satisfied, and it would then be helpful if the constructor is only accessible when those constraints are actually satisfied. For example, List<E>([int? length]) has been deprecated, but it could actually be allowed in cases where T is nullable.

Introduction and motivation

Please take a look at the examples in #2308 to see that these situations do occur in practice. In this context we will consider a minimal example which is specifically designed to show what the mechanism can do.

The class A describes objects that contain an object of type X extends num. This means that the contained object could be an int, but we cannot know for sure that it is an int unless we have a receiver of type A<int>. In other words, it is safe to execute isEven on an instance of A<int>, but it is unsafe to do it on an instance of A<T> for any T which isn't a subtype of int.

class A<X extends num> {
  final X x;
  A(this.x);
  bool get isEven => (x as int).isEven;
}

void main() {
  A<int> a1 = A(1);
  print(a1.isEven); // 'false'.
  var a2 = A('Hello'); // Inferred type `A<String>`.
  print(a2.isEven);
}

Obviously, A.isEven may incur a dynamic type error, because it contains a type cast. However, assume that we want to have the isEven method on A anyway, because it is very convenient to be able to call it directly (rather than remembering to do (a.x as int).isEven every time). If we know statically that the type argument to the A is int then the method is guaranteed to avoid the dynamic type error, but we can't express that relationship.

In this case we're just writing a forwarding method, but the mechanism is applicable much more broadly:

A use case arises whenever there is a container-like object whose behavior includes performing some operations on the contained objects (e.g., List.sort), and some type parameters describe the contained objects in some way (e.g., a List<T> contains elements of type T), and we don't want to constrain the class/mixin type parameters so much that the contained objects are always guaranteed to have the required types (e.g., we do not want to constrain List such that no list can ever contain an object unless it is Comparable).

Another case arises when there is an object, instance of a class class C<X1..Xk> .., which has a reference to some other object(s), and the type of those other objects is described by some of the type parameters X1 .. Xk. For instance, a graph with nodes of type Node<X> referring to other nodes of type Node<X>.

In any case, here's the proposal put forward in this issue for how to make it statically type safe:

class A<X extends num> {
  final X x;
  A(this.x);

  // We introduce the notion of instance members conditioned on
  // a list of type parameter constraints.
  if <X extends int>
  bool get isEven => x.isEven;
}

void main() {
  A<int> a1 = A(1);
  a1.isEven; // OK.
  var a2 = A('Hello');
  a2.isEven; // Compile-time error, `String <: int` does not hold.
  A<Object> a3 = a1;
  a3.isEven; // Compile-time error, `Object <: int` does not hold.
  (a1 as dynamic).isEven; // OK.
  (a2 as dynamic).isEven; // OK at compile time, throws at run time.
}

The ability to make an instance member or constructor conditional by adding a prefix similar to if <X1 extends T1 .. Xk extends Tk> to the declaration is the main element of this feature. However, the feature also includes the ability to control the requiredness of optional parameters of a conditional method. In particular, if <[Null extends X]> void foo([X x]) .. specifies that foo takes an optional parameter in the situation where the receiver has a statically known type argument X such that Null extends X, but that parameter is not optional when this property cannot be guaranteed statically, and similarly for optional named parameters with {} around some or all of the constraints.

The feature proposed here, conditional instance members and constructors, amounts to specifying some constraints on the type parameters of the enclosing class, reporting a compile-time error whenever such members are invoked or torn off unless the constraints are statically known to be satisfied by the receiver, and being able to use those constraints to improve the static analysis of the given method body.

Proposal

Syntax

The Dart grammar is adjusted as follows:

classMemberDefinition ::= // Modified rule.
  classMemberCondition? unconditionalClassMemberDefinition

unconditionalClassMemberDefinition ::= // New rule, equal to classMemberDefinition rule before change.
  methodSignature functionBody |
  declaration ';'

classMemberCondition ::= // New rule.
  'if' '<' classMemberConstraints (',' ('[' classMemberConstraints ']' | '{' classMemberConstraints '}'))? '>'

classMemberConstraints ::= // New rule.
  classMemberConstraint (',' classMemberConstraint)*

classMemberConstraint ::= // New rule.
  type 'extends' type

Static analysis

Consider an instance member declaration or constructor declaration of the form if memberConds D in a class or mixin C.

Note that a compile-time error will occur if any type that occurs in memberConds is not well-bounded, or it contains an identifier which is not in scope, or it contains an identifier which resolves to a declaration of something that isn't a type, just like all other occurrences of a type in a Dart program.

If memberConds is of the form memberCond1 .. memberCondK, [memberCondK+1 .. memberCondN], it is a compile-time error unless D declares a method or constructor that accepts one or more optional positional parameters.

If memberConds is of the form memberCond1 .. memberCondK, {memberCondK+1 .. memberCondN}, it is a compile-time error unless D declares a method or constructor that accepts one or more optional named parameters.

If D declares an instance member name that overrides a member signature s in a direct superinterface of the enclosing class C, a compile-time error occurs unless s has a member condition sMemberCond such that satisfaction of sMemberCond implies satisfaction of the member condition memberConds.

(We have simple cases like Iterable<E> which is a superinterface of List<E>. If a conditional member of List overrides a conditional member of Iterable, and it has a single constraint E extends T for some specific type T, the same condition can be used in List. In some cases where the type parameters are passed to superinterfaces in non-trivial types, e.g., class B<X> extends A<List<X>, X Function()> .. it is possible that the implication cannot be proven even in some cases where it does hold. IN that case B just cannot override a conditional member by a conditional member, but it could declare an unconditional one and check the types as needed. So this is an area where some further studies are needed.)

Consider a member access e.m (that is, a getter invocation like e.myGetter, a method invocation like e.myMethod(42), etc.) and assume that the name of the member is n. Assume that the member signature named n in the interface of the static type of e has a condition if memberConds.

In the case where memberConds is of the form < memberCond1 .. memberCondK ...>, a compile-time error occurs, unless it is statically known that each of the subtype relationships in memberCond1 .. memberCondK are satisfied when substituting the statically known actual type arguments of the receiver for the type parameters of the statically known receiver class.

In the case where memberConds is of the form <... [memberCond1 .. memberCondK]>, an invocation that omits one or more of the optional positional arguments is a compile-time error unless memberCond1 .. memberCondK are satisfied, in the same way as specified above. When all optional positional parameters are provided in the invocation, memberCond1 .. memberCondK are ignored. When a default value is specified for an optional positional parameter, each of memberCond1 .. memberCondK can be assumed when checking the type of the default value.

In the case where memberConds is of the form <... {memberCond1 .. memberCondK}>, an invocation that omits one or more of the optional named arguments is a compile-time error unless memberCond1 .. memberCondK are satisfied, in the same way as specified above. When all optional named parameters are provided in the invocation, memberCond1 .. memberCondK are ignored. When a default value is specified for an optional named parameter, each of memberCond1 .. memberCondK can be assumed when checking the type of the default value.

The same rules apply when we consider an instance creation expression (like C<...>() or C<...>.named(), where C is a class or a type alias that denotes a class).

When the static analysis of the body of D is performed (if any, D could also be abstract), each of the member constraints that are not inside [] or {} can be assumed.

In particular, if a constraint of the form X extends T exists where X is a type variable declared by the enclosing class C, X is treated as if it had been declared by the class with the bound T. If a constraint of the form Null extends X exists where X is a type variable declared by the enclosing class C, X is treated as a nullable type (this is not sound unless we have sound declaration-site variance and X is declared to be contravariant or invariant, but it will be checked dynamically when X is a dynamically checked covariant type variable, that is, the ones we have today).

Dynamic semantics

Instance member invocations based on statically known types have the same semantics with conditional instance members as they have without the condition. Similarly, instance creation expressions have the same semantics with conditions as they would have without conditions.

(The use of conditions will prevent certain statically checked invocations or instance creations from happening at compile time, but it will not change the behavior of the invocations/instance creations which are not flagged as errors.)

Dynamic instance member invocations will check dynamically that the constraints in <> are satisfied. Furthermore, the constraints in [] will be checked dynamically if one or more optional positional arguments have been omitted, and the constraints in {} will be checked dynamically if one or more optional named arguments have been omitted.

Examples

The example given in the introduction will work with the feature:

class A<X extends num> {
  final X x;
  A(this.x);

  if <X extends int>
  bool get isEven => x.isEven;
}

void main() {
  A<int> a1 = A(1);
  a1.isEven; // OK.
  var a2 = A('Hello');
  a2.isEven; // Compile-time error, `String <: int` does not hold.
  A<Object> a3 = a1;
  a3.isEven; // Compile-time error, `Object <: int` does not hold.
  (a1 as dynamic).isEven; // OK.
  (a2 as dynamic).isEven; // OK at compile time, throws at run time.
}

We can allow specific default values in specific cases, and we can (of course) also make constructors conditional:

class C<X> {
  X x;

  if <{String extends X}>
  C({this.x = 'A string!'})
}

void main() {
  C<Object>(); C<String>(); C(); // OK, using the string default.
  C<int>(); // Compile-time error.
  C<int>(x: 12); // OK.
}

Conditional constructors may be particularly useful, because they are sound without adding any sound variance mechanisms to the language (because they are always invoked statically, so there is no variance).

The core library examples will work as follows:

class List<E> ... {
  ...
  if <Null extends E>
  external factory List<E>([int? length]);
  ...
  if <[E extends Comparable]>
  sort([int compare(E a, E b) = Comparable.compare]) {
    ... // Note: we cannot assume `E extends Comparable` here, that will
    ... // be true for some invocations, and not for others. But we do know
    ... // that `compare` is non-null and has type `int Function(E, E)`.
  }
}

class Completer<T> ... { // With sound declaration-site variance, we'd use `inout T`.
  ...
  if <[Null extends T]>
  complete([FutureOr<T> value]) {...} // The default value of `value` is `null`.
}

class B {}

int compareB(B b1, B b2) {...}

void main() {
  List<int> xs = ...;
  xs.sort(); // OK.
  List<B> ys = ...;
  ys.sort(); // Compile-time error.
  ys.sort(compareB); // OK.

  Completer<int?>().complete(); // OK.
  Completer<int>().complete(); // Compile-time error.
  (Completer<int>() as Completer<int?>).complete(); // OK statically, throws at run time.
  (Completer<int>() as dynamic).complete(); // OK statically, throws at run time.
}

The isEven example illustrates the straightforward situation where we want to have an instance method that runs safely when some extra bound is known to be satisfied on a particular type argument. This allows the method to use the enhanced interface of objects of type X, and it has the same soundness properties as we would have if the type parameter had been declared with the extra bound (but in that case the class as a whole could not be used with any actual type arguments that don't satisfy this extra bound).

The sort example illustrates that we can make a positional parameter conditionally optional, such that we know it will only be omitted in situations where some extra constraints are known to hold. This allows us to handle the default value of the parameter under the assumption that the extra constraints are satisfied. In particular, we may then use a default value which is not type correct when those extra constraints are not known to hold.

The complete example shows that we can express that the parameter is optional (with default value null) when T is nullable, which also makes FutureOr<T> nullable, so it's not an error to leave the default value implicit. However, when the constraint is not statically known (module covariance unsoundness) to be satisfied, the parameter is mandatory.

NB: If we can change Completer to have an invariant type parameter (inout T) then we can omit the dynamic check that value can actually have the value null without violating soundness, because that is then statically guaranteed to be the case. This means that this feature will work in synergy with sound variance (just like lots of other features of Dart ;-).

Discussion

The treatment of overriding in the case where the type variables of a class are used in non-trivial ways in superinterfaces needs to be clarified. We would certainly need to consider cases like the following:

class A<X> {
  if <X extends Iterable<Object>>
  void foo(X x) {...}
}

class B<Y> extends A<List<Y>> {
  if <Y extends Object> // OK!
  void foo(List<Y> y) {...}
}

If it turns out to be non-tractable to prove the required implications then we would presumably have a number of supported situations (the known easy ones), and then it would simply be a compile-time error to try to use conditional instance members with an overriding relation in cases that we can't prove. In any case, an overriding member can soundly be non-conditional, and that would be sufficient to allow the member to exist. It might then have to contain dynamic type checks (just like it would contain dynamic type checks today, where we can't have conditional members at all).

[jakemac53]

It is worth noting that many use cases of this feature (but not all) could be implemented via extensions, for instance the original example:

class A<X> {
  final X x;
  A(this.x);
}

extension IsEven on A<int> {
  bool get isEven => x.isEven;
}

void main() {
  A<int> a1 = A(1);
  print(a1.isEven); // 'false'.
  var a2 = A('Hello'); // Inferred type `A<String>`.
  print(a2.isEven); // compile time error, `isEven` does not exist
}

I think the main thing this approach doesn't support is the ability to override these methods? And you can't have conditional fields either.

[eernstg]

the main thing this approach doesn't support is the ability to override these methods

That's definitely a good approximation. However, there is also the ability to have optional parameters with a default value which is only type correct when some additional constraints are satisfied (those parameters will simply be required otherwise).

class C<X> ... {
  final X x;
  if <[String extends X]>
  C([this.x = 'Default x!']);
}

void main() {
  C<Object>(); // OK, `String <: Object`, can use the default.
  C<int>(1); // OK, but the actual argument cannot be omitted.
}

Another thing we can't do with extensions is to introduce constructors of the target class (I forgot to mention constructors in the first version of the proposal in several different locations in the text, that's been fixed now).

The ability to invoke conditional methods dynamically (and have the type variable constraints checked at run time) is another thing that extensions won't support. Note that the constraints are expressed and checked in terms of the actual value of each type variable, not in terms of the type arguments in the statically known receiver type.

Yet another thing is lower bounds (like if <Null extends E> set length(int newLength) ...). In the case where we have dynamically checked covariance (which is true for all type variables today, and until we get sound declaration-site or use-site variance), we can't statically guarantee that Null extends E, but that will be fully statically checked as soon as we are able to make the relevant type variables invariant or contravariant. Also, there is no subtype subsumption during instance creation expression evaluation, which means that there are no variance related type checks, and hence no danger that they will fail.

[jakemac53]

Another thing we can't do with extensions is to introduce constructors of the target class (I forgot to mention constructors in the first version of the proposal in several different locations in the text, that's been fixed now).

The constructor case is interesting... would it be totally crazy to also add an else if <...> construct, to allow invoking a different constructor but with the same signature, based on the generic type? Essentially that gives you overloading based on type arguments? Not sure that is actually a good idea, just a thought haha.

class C<X> ... {
  final X x;
  if <[String extends X]>
  C([this.x = 'Default x!']);
  else if <[num extends X]>
  C([this.x = 1.0]);
}

[eernstg]

Haven't thought about that! 😃

I'm usually somewhat worried about static overloading (e.g., because it breaks tear-offs), but it is certainly interesting to consider.

[jakemac53]

I'm usually somewhat worried about static overloading (e.g., because it breaks tear-offs), but it is certainly interesting to consider.

I do think in general this is a fair bit easier to reason about than regular overloads - the selection process for methods is straightforward since you just select the first one that matches, you could desugar as well to something that can select the right function at runtime. If you want to be able to statically choose the right function when a tearoff is invoked, that would involve probably making it a part of the static function type though.

In general tear-offs probably need to be talked about for this proposal anyways, but maybe you end up with a tearoff that has a generic with a bound that matches the condition? So C<T> Function<T extends String>(T) for the example above would be the type of the tearoff C.new?

[eernstg]

just select the first one that matches

I think we have to consider static overloading to be a more complex beast than that.

I'd expect static overloading to rely on a notion of specificity, which would be a partial order on member signatures. For any given invocation e.f() which is subject to static overloading, we'd find the set of applicable implementations f1 .. fk, compute the ordering according to their specificity, check that exactly one fj is at least as specific as every one of f1 .. fk, and then proceed to compile e.f() as e.fj().

Note that specificity gets gnarly quite easily. For instance, is void foo([Object?]) more specific than void foo([FutureOr<dynamic>]) because Object? is "more of a top type" than FutureOr<dynamic>? The two function types void Function([Object?]) and void Function([FutureOr<dynamic>]) are mutual subtypes of each other, so subtyping won't help us. How about void foo(Object?) (that is, where the parameter is not optional)? The function type void Function([Object?]) is a subtype of void Function(Object?), but it is not obvious to me that the former implementation is the correct choice in the case where we invoke e.foo(1), and we can choose between those two signatures. I don't think it's good for code maintainability to rely on distinctions like that in order to make the choice between executing one or the other among two completely separate pieces of code.

Similarly for void foo(int, [String]) vs. void foo(int, [List<Boolean>]) when the invocation is e.foo(1), or void foo(String, {int x}) vs. void foo(String, [int x]), etc.etc.etc.

We do have rules to answer some questions that are similar to these specificity questions when we are determining the interface of a class (for example, using the function MORETOP), but in that case the choice does not determine a selection of one piece of code over another, it just determines the interface of the class, and the concrete declarations in the superclass chain get to decide exactly which function body we will execute, no matter which static type the receiver has at the call site.

If we're trying to redo static overloading at run time then we'd need to know the static type of the receiver at the given call site, because otherwise we'd get a different set of applicable callees. Next, we'd have to keep track of the scoping environment (such that we can find exactly the set f1 .. fk mentioned above, and avoid including declarations that aren't imported into the library that contains the call site). This is something that we've otherwise compiled away, by selecting the meaning of each name application according to the scope rules at compile-time, and then forgetting all about those names which weren't looked up. An even more ominous issue is that we might decide that no most-specific signature exists, which could give rise to a dynamic error, similar to noSuchMethod, except that it's "noMostSpecificMethod".

My conclusion is that static overloading is inherently a compile-time mechanism, and the work which is needed in order to replicate the compile-time decision faithfully at run time is so costly that we'd never want to do it, and it's not even a useful way to think about it.

The proposal here seems to enable some notion of static overloading, but I'd prefer to look at it from a different angle: When an instance member is conditional, it is accessible if and only if the associated constraints on the type variables of the enclosing class are satisfied. This means that the implementation which is torn off in case of a tear-off operation is well defined (even if done dynamically), it just happens to be an error (compile-time if typed, run-time if done dynamically) to tear off a conditional member in the case where the constraints are not satisfied.

Constructors are different: When a constructor is conditional, the condition is on the actual type arguments which are passed to the constructor in the instance creation expression. This means that we can obtain exactly the same guarantees regarding upper bounds by declaring those upper bounds as bounds of the type variables of a torn-off constructor which is a generic function:

class C<X> {
  final X x;

  C(this.x);

  if <X extends int>
  C.even(X x): this.x = x.isEven ? x : x + 1;
}

void main() {
  print(C.even.runtimeType); // `C<X> Function<X extends int>(X x)`.
}

However, we can't express anything other than an upper bound in the function type, which means that we may have to restrict the tear-off to a safe set of cases:

class A<X> {
  if <[String extends X]>
  A([X x = 'A string default']);
}

void main() {
  print(A.new.runtimeType); // `A<X> Function<X>(X x)`: `x` is not optional.
}

[jakemac53]

Yeah I definitely agree that true static overloading gets gnarly fast. Moving the selection process more directly into an if/else type structure greatly simplifies it though (all the hard questions are instead handled by the user, which also gives them greater control and flexibility). We no longer have to define specificity rules etc. I do agree tear-offs are hard though, although I guess you could enforce that the user has to provide the generic type arguments up front in order to take a tearoff of one of these functions.

[eernstg]

Right. I do think, however, that there is no direct match with this proposal, because the constraints are on the type parameters of the enclosing class, not on the type parameters of the function itself. Let's try to emulate a simple case:

class C {
  void m(int i) { print(i.isEven); }
  void m(String s) { print(s.length); }
}

We can't use if <"anything"> as proposed, because the class doesn't declare any type parameters. We could try to introduce an extra type parameter, in order to have something to constrain:

class C<X> {
  if <X extends int>
  void m(X x) { print(x.isEven); }
  else if <X extends String>
  void m(X x) { print(x.length); }
}

That would work, but we can't call both of them on the same receiver: We can't create a C<T> for any T other than Never, such that T <: String and T <: int. So we'd basically have to create a C<Never> whenever we need a C, and that means that it would allow us to call any of the two methods, but then we couldn't call m because it insists on getting an actual argument of type Never, even in the case where the statically known type of the C instance is C<int> or any other C<T> for some T.

I don't know if it could work, but it isn't obvious to me how to do it...

[jakemac53]

Right. I do think, however, that there is no direct match with this proposal, because the constraints are on the type parameters of the enclosing class, not on the type parameters of the function itself.

Right, I think you could extrapolate this same proposal to work for type parameters of the function as well, but it does get weird (can the type parameters also be different for each version??).

We can't create a C<T> for any T other than Never

I am not sure if I understand exactly what you are getting at here - I think that actually this just ends up enforcing that only a C<int> and C<String> can see this method m? You could support a regular else clause as a fallback as well, making some function exist for all versions. I agree that you can't make a function which takes a C<T>, and invokes m without first doing a type check to promote it to a C<int> or C<String>.

but then we couldn't call m because it insists on getting an actual argument of type Never,

Can you explain this a bit more? I agree that a C<anything but int or String> will not have a callable m method but isn't that exactly what we want? A C<int> etc should be callable.

[munificent]

Another thing we can't do with extensions is to introduce constructors of the target class (I forgot to mention constructors in the first version of the proposal in several different locations in the text, that's been fixed now).

We could approach this problem from the other direction, then, and add support for constructors (and other static methods) in extensions (#723). I think that's a smaller, simpler feature, and would cover many of the use cases here. You lose the ability to override but given how complex that seems to be, that could arguably be a good thing.

In general, I like the relatively simple model that instance methods dispatch on class and extension members dispatch on type. If you have a member that you want to only be callable on certain types (i.e. generics instantiated with certain type arguments), that sounds a lot like an extension method call to me.

[eernstg]

We could approach this problem from the other direction, then, and add support for constructors (and other static methods) in extensions (#723).

That is true, but it's worth noting that these mechanisms have different expressive power. (As an aside, I think static extensions are a promising feature, but there's a lot of work to do before we have that. For instance, I think we'd need to introduce the notion of 'the static type of a class' in order to make a static extension applicable to more than a single class, we may have to declare that type using static implements clauses, perhaps static extends, etc, so perhaps this is a shortcut which is longer than adopting the proposal in this issue. ;-)

Here's an example where the proposed static extensions feature is used to add a constructor (taken from here):

static extension X on List<int> {
  factory X.n(int count) => List<int>.filled(count, 0);
}

void main() {
  ... List<int>.n(5) ...
}

Now consider introducing a constructor which is applicable when two type arguments are related in some non-trivial way. With a conditional constructor we can do things like this:

class Pair<X, Y> {
  final X x;
  final Y y;
  Pair(this.x, this.y);
  if <List<X> extends Y>
  Pair.withList(this.x) : y = [x];
}

The declaration of a static extension (the set of variants that I've seen, at least) seems to rely on something which is similar to the on-type of a (regular, non-static) extension declaration. That is, the static extension is applicable when the class matches the class specification (in the example above that's a denotation of a specific class, List) and the type arguments satisfy the bounds (in the example, the actual type argument is int, so presumably that static extension is only applicable when the actual type argument is exactly int).

I don't see how the static extension mechanism would be able to express a constraint like List<X> extends Y without generalizing that proposal in ways that are essentially the same thing as including this proposal as a part of the static extensions proposal.

With conditional constructors, as proposed in this issue, the specification given in <> is a series of subtype constraints, which is considerably more powerful than the upper bounds in regular formal type parameter declarations. The actual type arguments are selected using completely normal type inference at the call site, and then we just plug the result into the given if <...> constraint list, checking that all the required subtype relationships are satisfied.