Fasta strong mode inference confused same class extending Base<C> and mixing class extending Base<B>

[mraleph]

The code below is extracted from Flutter codebase with classes renamed for brevity.

abstract class B {
}

class C extends B {
  bool c;
}

abstract class Base<T extends B> {
  T get f => null;
}

abstract class M extends Base<B> {
}

class X extends Base<C> with M {
  bool problem() => f.c;
}

Here class X is extending Base<C> and also mixing class M which extends Base<B> where C extends B. Base<T> has a getter f returning T. In class X which extends Base<C> therefore it is expected that f returns C. However Fasta in strong mode reports:

flu.dart:19:19: Error: The getter 'c' isn't defined for the class '#lib1::B'.
Try correcting the name to the name of an existing getter, or defining a getter or field named 'c'.
   bool get c => f.c;
                   ^

This means it thinks f returns B and not C. Furthermore Fasta also generates throw of a compile error in X.problem body.

    method problem() → core::bool
      return (let final dynamic #t1 = this.{flu::Base::f} in let dynamic _ = null in const core2::_ConstantExpressionError::•().{core2::_ConstantExpressionError::_throw}(new core2::_CompileTimeError::•("file:///usr/local/google/home/vegorov/src/dart/sdk/t/flu.dart:19:23: Error: The getter 'c' isn't defined for the class '#lib1::B'.\nTry correcting the name to the name of an existing getter, or defining a getter or field named 'c'.\n  bool problem() => f.c;\n                      ^"))) as{TypeError} core2::bool;

@lrhn can you take a look at this code and say if it is expected to compile and run correctly according to the current thinking about mixins in Dart 2.

If it is we will need this to be fixed in Fasta.

I am marking this as P0 because this completely blocks us from running Flutter in strong mode with no workaround available.

/cc @lrhn @leafpetersen @stereotype441 @kmillikin @a-siva

[mraleph]

@stereotype441 if there is something quick we could patch into the Fasta inference to make this case work that would also be appreciated.

[stereotype441]

Uh oh. It's a fundamental limitation of both kernel and fasta, as currently implemented, that no class can implement any other class multiple times with different type arguments. IIRC, this limitation was inherited from dart2js, and at some point in the last few months I believe @leafpetersen and @lrhn discussed the idea of making the limitation explicit in the Dart 2.0 spec. @leafpetersen can you confirm?

Fully relaxing this limitation would be nontrivial, because it underlies the design of the kernel ClassHierarchy API, which is used pervasively. Specificially, ClassHierarchy.getClassAsInstanceOf and ClassHierarchy.getTypeAsInstanceOf are intended to return a single result. If we allowed code like you quote above, it's not clear whether ClassHierarchy.getClassAsInstanceOf(X, Base) should return Base<B> or Base<C>.

Unfortunately, I can't think of something quick we could patch into Fasta inference to make this work. The best workaround I can think of is as follows: partially relax the limitation, changing the rule to say "a class can implement another class multiple times with different type arguments, provided that one of the implemented types is a subtype of all the others". Then we could modify ClassHierarchy.getClassAsInstanceOf and ClassHierarchy.getTypeAsInstanceOf to return the implemented type that is a subtype of all the others.

So in the example above, ClassHierarchy.getClassAsInstanceOf(X, Base) would return Base<C> since Base<C> is a subtype of Base<B>. This would fix the @mraleph saw because the front end type inferrer uses ClassHierarchy.getTypeAsInstanceOf to figure out the appropriate substitution to do when inferring the type of f, so it would infer a type of C, and then the getter c would be found in class C.

@kmillikin the would all have to be made to ClassHierarchy, which is not really code I'm terribly familiar with. What do you think? The chief complication as I see it would be that currently ClassHierarchy doesn't contain a reference to TypeEnvironment, so ClassHierarchy can't do the necessary subtype checks to figure out which implementation is a subtype of the others (if any).

I'm happy to discuss this in more detail over VC if that would be helpful.

[stereotype441]

Here's an idea for a quicker (but more fragile) workaround: change the forEach loop in ClosedWorldClassHierarchy._recordSuperTypes from this:

      superInfo.genericSuperTypes?.forEach((Class key, Supertype type) {
        subInfo.genericSuperTypes[key] = substitution.substituteSupertype(type);
      });

to this:

      superInfo.genericSuperTypes?.forEach((Class key, Supertype type) {
        if (!subInfo.genericSuperTypes.contains(key)) {
          subInfo.genericSuperTypes[key] = substitution.substituteSupertype(type);
        }
      });

When determining which interfaces a given class implements, the supertype is visited first, then mixins, then classes in the "implements" clause. This will ensure that the interface seen first dominates, so in the example above, the extends Base<C> will dominate.

You'll also need to make the same change to IncrementalClassHierarchy._recordSuperTypes.

[lrhn]

I can confirm that we sanctioned the "no implementing two different instantiations of the same interface" rule for Dart 2. So no, it's not supposed to compile in Dart 2.

This is one of the reasons we can even add the "extractTypeParameter" function - otherwise its behavior would be underdefined, and it makes the "flatten" spec function well-defined too. It's a good thing.

If being this restrictive turns out to be a problem in practice, we might be able to loosen that in the future, by allowing multiple implementations of the same interface, as long as one of them is the most specific one, and that is the one that is used whenever it matters.
Since dart2js never allowed it, most general purpose code should not have a problem.

[mraleph]

If being this restrictive turns out to be a problem in practice, we might be able to loosen that in the future, by allowing multiple implementations of the same interface, as long as one of them is the most specific one, and that is the one that is used whenever it matters.

Could we relax it now rather then in the future? It seems absolutely reasonable that application should be able to satisfy mixin requirement with a more specific class. It already works when generics are not involved, so it's not clear why it should not work when generics are involved, especially when these generics are fully instantiated like here.

This seems to be a pretty common pattern in Flutter. To give you a real example:

abstract class State<T extends StatefulWidget> extends Diagnosticable {
  T get widget;
}

abstract class AutomaticKeepAliveClientMixin extends State<StatefulWidget> {
}

class _DismissibleState extends State<Dismissible> with TickerProviderStateMixin, AutomaticKeepAliveClientMixin {
   bool get _directionIsXAxis {
    // here widget is expected to be Dismissable
     return widget.direction == DismissDirection.horizontal
         || widget.direction == DismissDirection.endToStart
         || widget.direction == DismissDirection.startToEnd;
   }
}

As a workaround I guess I could rewrite this code as:

class _DismissibleState extends State<StatefulWidget> with TickerProviderStateMixin, AutomaticKeepAliveClientMixin {
  @override
  Dismissible get widget => super.widget; 

   bool get _directionIsXAxis {
     return widget.direction == DismissDirection.horizontal
         || widget.direction == DismissDirection.endToStart
         || widget.direction == DismissDirection.startToEnd;
   }
}

but that also affects few other methods in State<T> that reference T, which would mean you need to override them as well to tighten T.

[leafpetersen]

I try not to understand mixins, (with a fair bit of success I think it's fair to say), but while I had the same initial thought (that this produced something with two different interfaces at the same type), is that really the case (or rather, should it be)? My understanding of

abstract class M extends Base<B> {
}

when thought of as a mixin would be that Base<B> is a constraint on the superclass, not an interface directly provided by M. I don't remember if Lasse's proposal for Dart 2.0 mixins allow them to be used as interfaces or not? If not, I think there's not problem in Dart 2.0: you just say that X has only the direct interface Base<C> which is a subtype of Base<B>.

If they are usable as interfaces, then perhaps there's a problem, I'm not sure. It doesn't seem like you need to keep both around on the instance, but you do presumably need to know that M <: Base<B>. Or maybe not? Maybe you just say that the interface induced by a super mixin is only itself, not including the super interfaces?

Ahh mixins.

[lrhn]

What you are seeing is a consequence of using class syntax to introduce mixins (something it's not very well suited for, and the reason we want to introduce dedicated mixin declarations).

abstract class M extends Base<B> {}

This is a class declaration. It might be intended as a mixin, but here it is just a class declaration. The class M implements Base<B> because the interface of the superclass is always a superinterface of the class.

When you do a mixin application of the mixin derived from M, that mixin application also implements the interface of M - and therefore also implements Base<B>.

I don't remember if we ever decided whether the new mixin syntax would introduce an interface. I'd be willing to say no, because then it wouldn't automatically implement the super-constraints.
Making the mixin a nominative type, but not including its supertypes as superinterfaces, will be odd. It's like saying class X extends private Y ... so that X inherits Y but doesn't implement it. Sure, it's possible, and C++ does it, but it'll feel weird. It'll probably work for mixins because the mixin application will always inherit a suitable super-interface from the superclass, but you would't be able to do: M mixinApplication = ... ; Base<B> b = mixinAppliation;.

Now, I'm not sure mixins is the only problem here.

If we have family polymorphism, like we do in Flutter in some cases, then it's likely we'll see code like:

abstract class Widget<S extends WidgetState> { ... }
class TextWidget implements Widget<TextWidgetState> { ... }
class UnderlinedTextWidget extends TextWidget implements Widget<UnderlinedTextWidgetState> { ... }

(I don't know Flutter well enough to say that it happens, but I wouldn't be surprised).
This has exactly the same issue, without any mixins. You cannot specialize a class implementing SomeWidget<SomeWidgetState> to SpecializedWidget<SpecializedWidgetState>, you have to have a common generic superclass instead.

The question is whether we want to support it, or require it to be rewritten into:

abstract class Widget<S extends WidgetState> { ... }
class TextWidgetBase implements Widget<T extends TextWidgetState> { ... }
class TextWidget extends TextWidgetBase<TextWidgetState> { }
class UnderlinedTextWidget extends TextWidgetBase<UnderlinedTextWidgetState> { ... }

(which is an option).

[leafpetersen]

My preferred state would be that mixins have their own syntax, and are thought of as functions from classes to classes rather than classes in an of themselves, and hence have no interface of their own (or alternatively, have a higher order interface : Base -> Result).

Obviously, we're not there yet, and we need to get flutter up and running, so I don't see any good alternatives other than allowing multiple interfaces.

It seems like the proposal to allow classes to implement the same interface multiple times so long as there was one most precise version of the interface would be reasonable: it seems like at least for runtime subtyping purposes implementations could only record the most precise interface. For static checking the implementations would need to record all of them. Does this seem right? Can dart2js still use its existing approach and be compliant with this, or are there corner cases where it needs to know all of the super-interfaces?

This is subject to the restriction we discussed, that all concrete method implementations must satsify all of the super-interfaces, not just the most specific one.

[lrhn]

It seems like the proposal to allow classes to implement the same interface multiple times so long as there was one most precise version of the interface would be reasonable: it seems like at least for runtime subtyping purposes implementations could only record the most precise interface. For static checking the implementations would need to record all of them. Does this seem right?

I don't see why static checking would be different, as long as it is just subtype checks.

Can dart2js still use its existing approach and be compliant with this, or are there corner cases where it needs to know all of the super-interfaces?

The one corner-case I can think if is covariant parameters. We have so far said that an overriding covariant parameter must be a sub- or super-type of every super-interface parameter that it overrides.
It then matters whether add(num) is an actual super-interface or just a super-type.
I'd love to change that instead, and just require covariant overrides to have to be a super- or sub-type of the immediate super-interface method parameters.
(Or, in other words, I'd like to get to the point where it doesn't matter if something is a super-interface or "just" a super-type—interface declarations would just be one way to introduce a super-type).

The other corner-case is someone wanting to implement interfaces that implement List<Object> and List<dynamic> respectively (and add List<void> for even more "fun"), since either is more specific than the other. We need to pick one of them as the actual static List-type for the class. There is no way to manually disambiguate, so making it an error is likely problematic.
We have similar issues in other places, so maybe we need to find a way to order our infinitely many top-types, void, dynamic, Object, FutureOr<Object>, etc., for preference when we need to choose between them (and in a way that generalizes to choosing between types like Map<Object, void> vs Map<dynamic, Object>). One option is picking the one from the syntactically first interface in the class declaration.

Apart from that, @johnniwinther WDYT?