[Records] Should pointwise implicit downcasts be allowed in records?

[leafpetersen]

Consider the following example:

  dynamic d = "hello";
  (int, int) r = (d, 4);

There are two possible inferred types for the record literal.

• We might choose to infer this as (dynamic, int)
  • In this case this code is a static error, since neither type is dynamic, and the RHS type is not a subtype of the LHS type.
• We might choose to infer this as (int, int)
  • In this case this code is a runtime error, since there must be an implicit cast added to cast d to int before constructing the record.

Which do we intend?

In general, I'm not keen on adding more implicit downcasts. I'd rather the user make the cast explicit here.

We do push casts inwards for list literals however:

  dynamic d = "hello";
  List<int> l = [d];

fails at runtime, not statically (as it must, since we prefer the context type for nominal types, unlike the structural types).

cc @munificent @eernstg @lrhn @jakemac53 @natebosch @stereotype441 @kallentu @chloestefantsova @johnniwinther @scheglov @srawlins

[lrhn]

So the question is whether the context type is propagated into the record, and what that means.

The context type must be propagated into a record, otherwise (double, double) point = (0, 1); won't work, and I think it must work. Records are just parallel operations that otherwise work just as the individual operations would.

For (int, int) r = (d, 4); that means that the context type of d is int and the static type of the d variable is dynamic.

We should downcast d at the same place we would in int _ = d;, which ... I don't actually know where is.

We are not consistent between decimal-literal-to-double promotion and implicit downcast:

extension <T> on T {
  T log() {
     print("$this:$runtimeType @ $T"); 
     return this;
  }
}

void main() {
  double d = 1..log(); // Prints "1.0:double @ double", so static type of `1` is double.
  var d = "x" as dynamic;
  // int x = d..log();  // Does not work because static type of `d` is `dynamic`.
  int x = d..toString().log(); // prints "x:String @ String" before failing the cast, so it's at least outside of the *cascade*.
}

I don't remember what we decided about where to insert the downcasts.
For double-decimal-literals, it's easy, because the conversion can only happen in one place.
For casts, we can add them at any point in the flow up to where they're assigned to the variable providing the context type.

int x = ((d)..something());

could be ant of

int x = ((d as int)..something());
int x = ((d)..something() as int);
int x = ((d)..something()) as int;

It's not the first one.
Another example:

  double y = args.isEmpty ? 1 : 2.5;  // Works.
  var d = "a" as dynamic;
  int x = (args.isEmpty ? d : 4)..log();  // Throws because static type is `dynamic` for the ..log call.

We cast outside the conditional expression.

So, for consistency, I guess

(int, int) r = (d, 4);

should give the RHS the static type (dynamic, int) and then do an implicit downcast to (int, int) at the assignment.

That's also what happens when the RHS is not a literal.

var d = (random ? 'hello" : 2) as dynamic;
(dynamic, int) p1 = (d, 4);
(int, int) p2 = p1;

The (dynamic, int) type is assignable to (int, int) because each field is assignable.
The effect is an implicit downcast like:

(int, int) p2 = p1 as (int, int);

(where an optimizing compiler can omit checking the second field, and only check the ones typed as dynamic).

Same thing should happen for:

var (int x, int y) = p1;

The RHS is implicitly downcast to the LHS's required type before destructuring. If ordering matters, which it shouldn't here, but might for something like;

dynamic d = "a" as dynamic;
Foo foo = Foo(); // has x getter with side effects
var (Foo(x: var x), int y) = (foo, d);

Here the RHS is assignable to the LHS type of (Foo, int), but does not have that type because d is dynamic.
We should downcast the RHS pair (Foo, dynamic) to (Foo, int) before starting on the destructuring.

[eernstg]

I saw the feature spec updates before I saw this issue, and commented here.

I agree that there should be an implicit cast (and it's a separate effort, using --strict-something or a lints to avoid having expressions of type dynamic in the first place), but I think it's inconsistent to start having implicit casts from composite types containing dynamic to other composite types (e.g., casting from (dynamic, double) to (int, double)).

I'd prefer if we could specify this mechanism as a transformation that adds a cast to an expression of type dynamic, based on the propagated context type schema.

(So type inference is just one of several transformations, transforming f(42) into f<int>(42) and such; transforming an integer literal into a double-typed integer literal is another one, and adding a cast from dynamic is a third one; adding .call is a fourth & fifth one which may become just a fourth one. ;-)

So I'm suggesting that we transform e to e as T when e has static type dynamic, also when e is a field expression in a record literal, where T is the context type obtained from the context type schema and the result of inference on other parts of the enclosing expression.

The most compelling reason to do this is probably the potential introduction of bottom-up inference on other expressions, namely instance creations where some type arguments use statically checked variance:

class C<out X, inout Y> {
  final X x;
  Y y;
  C(this.x, this.y);
}

void main() {
  var d = 1 as dynamic;
  C<int, int> c = C(d, d); // Error or implicit casts?
}

[lrhn]

Erik's question is whether we define "assignability" for records so that a record type is assignable to another record type iff they have the same shape and the field types are pointwise assignable, or just by subtyping.

I think we should define assignability like that, for a couple of reasons.

• It allows users to think of record operations by their point-wise behavior. If you can do a = c; b = d; you can also do (a, b) = (c, d);, and you can understand the latter in terms of the former.

  I want all of the following to work:

  • (double, double) r = (1, 1);
  • (Function, int) r = (callableObject, 1);
  • (int Function(int), int) r = (id, int); (where id is T id<T>(T value) => value;)
  • (int, int) r = (1, 1 as dynamic);
    because it's so much easier to explain.

• Also because they all work for arguments in argument lists. Records look like argument lists. I'd hate it if
  foo(1, callableObject, 1 as dynamic) was valid but indirectFoo(args:(1, callabalObject, 1 as dynamic)) is invalid,
  where the args parameter has the same types as the parameters of foo.

• It makes it less important where we do the implicit downcasts, something which is currently ... complicated. For (int, int) r = (1, d);, having (1, dynamic) being assignable to (int, int) means that the assignment is definitely valid, all we can discuss is where the downcast happens. Is it = (1, d as int) or = (1, d) as (int, int). If assignability betwen records is purely by subtyping, the former implicit downcast, caused by an injected context type reaching the d, would make the assignment valid, but the latter downcast would not happen implicitly. I can do int x = d, but I can't do var (int x,) = (d,);. But I can do var (double x,) = (1,); because some coercions do work.

I have a strong opinion about assignability of records, which implies that an implicit downcast will be made.
For a non-record literal it's a given, it happens just before the assignment;

var p1 = (1, 2 as dynamic);
(int, int) p2 = p1;  // implicit `as (int, int)`.

For a literal, it matters mostly when combined with conditional expressions or cascades.

(int, int) r = someTest ? (1, 2 as dynamic) : (2 as dynamic, 1);
(int, int) q = (1, 2 as dynamic)..$1.arglebargle(); // allowed statically.

What we currently do is to delay the cast as long as possible (for double literals, that's no delay at all, ditto for the instantiated tear-off, because originally we couldn't instantiate after tear-off, for call and downcast, it's later).
Whatever we do, we should do it consistently for records too.
(we can change it, independently of what we do with records, as log as we stay consistent).

[chloestefantsova]

Like Erik, I commented on that topic elsewhere. I copy my comment below.

Maybe another minor argument in defence of casting from dynamic is the potential spreading of records into argument lists. In the following example the spreading will not work, forcing the manual spreading. The manual spreading may appear especially surprising in this case because it will work without the explicit downcast from dynamic, making the reader questioning why the spreading was done manually in the first place.

foo(Funcion(int, String) f, (dynamic, String) a) {
  // This will not work.
  f(...a); // '...a' is a hypothetical syntax for spreading arguments.

  // This will work.
  f(a.$0, a.$1); // The downcast from 'dynamic' will be inserted here.
}

[leafpetersen]

Ok, I think this discussion broadened a bit, so let me try to summarize the issues raised.

First, as I should have pointed out in my initial comment, this is quite connected to the issues raised here by @stereotype441 .

With that in mind, there are three possible directions that have been raised above (only two of which I had intended to introduce). To make it concrete, let's discuss WRT the following example:

  dynamic d = "hello";
  (dynamic, dynamic) dr = (d, d);
  (int, int) r = (d, 4);  // Example 1 
  r = dr; // Example 2

There are three possible interpretations we could take:

• We could use the specification I have proposed, in which casts are done at the point of assignment (the way they are currently generally done for other types, see the issue I linked above).
  • In this model, the inferred type of (d, 4) would be (dynamic, int) and there would be a static error on the line labelled "Example 1" because that type is not assignable to (int, int)
  • In this model, "Example 2" remains a static error.
• We could define inference for records specifically to follow the proposed direction from Consider pushing implicit conversions down #2129 , and push the casts in as far as possible into the structure of the term (I had originally specified inference this way in my draft, which led me to file this issue).
  - In this model, the inferred type of (d, 4) would be (int, int) and there would be a runtime cast inserted by the compiler making this the equivalent of (d as int, 4).
  - In this model "Example 2" becomes a static error.
• We could expand our definition of assignability to treat record types specially. So we would say that a value of record type R is assignable to a location of record type Q if each of the fields of R is pointwise assignable to the corresponding field of Q.
  • In this model, the inferred type of (d, 4) would be (dynamic, int), and there would be a runtime cast inserted by the compiler on the assignment: (d, 4) as (int, int).
  • In this model, "Example 2" also is accepted, with the same runtime cast inserted.

I had not proposed to go the third route, and am somewhat opposed to it - it's highly inconsistent with what we do with all other types. For example, we do not allow List<int> x = <dynamic>[3]. Nor do we special case function types in such a way.

Addressing some specific comments:

@chloestefantsova

Maybe another minor argument in defence of casting from dynamic is the potential spreading of records into argument lists. In the following example the spreading will not work, forcing the manual spreading.

I don't think this follows. Note that we do not make List<dynamic> assignable to List<int>, but we do define the static semantics of spreading to allow <int>[...<dynamic>[3]]. I don't immediately see why we can't/wouldn't just do the same for record spreading.

@lrhn

Erik's question is whether we define "assignability" for records so that a record type is assignable to another record type iff they have the same shape and the field types are pointwise assignable, or just by subtyping.

I think we should define assignability like that, for a couple of reasons.

None of your examples require the pointwise assignability (my option 3 above). They are all covered by option 2, I believe. That said:

@lrhn

• It allows users to think of record operations by their point-wise behavior. If you can do a = c; b = d; you can also do (a, b) = (c, d);, and you can understand the latter in terms of the former.

It is somewhat compelling that this would work uniformly if we did choose to specify option 3. The fact that the runtime type of a record is computed from its values means that casting any value r of record type to a record type Q will work iff casting every field of r to the corresponding type in Q would also work (assuming they have the same shape).

I'm not sure I find this compelling enough to want to add more implicit downcasts the the language though.

[eernstg]

I usually argue in favor of option 2, #2129, exactly because it is a simple and consistent approach. The argument against this approach is usually that it does not work very well for cascades:

class Callable {
  void call(int i) {}
  void foo() {}
}

void main() {
  // Standard example: Works today, breaks if taken to mean `(Callable().call)..foo()`
  void Function(int) f = Callable()..foo();

  // Variant that arises for records and cast-from-dynamic.
  var d = 1 as dynamic;
  (num, num) r = (d, 3.14)..$0.foo(); // Allowed with outer cast, fails with `(d as num, 3.14)`.
}

We surely do want to propagate the context type to cascade receivers (such that C() can receive useful actual type arguments in C<int> c = C()..foo();), and we discussed making it an error to have .call insertion on a cascade receiver (or indeed removing .call insertion entirely). The main problem is that a cascade receiver in a sense has more than one context: In the example above we may wish to change (d, 3.14) to (d as num, 3.14) in order to fit the needs of the enclosing variable declaration, but this change may interfere with the needs of the cascaded member accesses (like $0.foo()).

However, I don't see any additional inconveniencies caused by pushing casts into record literals, so I'll continue to support option 2 as the most consistent approach. That shouldn't prevent any of the solutions to the cascade problem that we might come up with.

On the other hand I'm worried about option 3 (that is, lifting assignability such that not just dynamic is assignable to T for all types T, but also (dynamic, S) is assignable to (T, S) for all T, etc., recursively), in particular because we're very unlikely to introduce a similar kind of lifting for any other kind of type.

void f(int i) {}

void main() {
  void Function(dynamic) g = f; // Will _not_ desugar as `... g = (dynamic d) => f(d);`
}

[lrhn]

I don't think comparing records to other kinds of types, mainly collections, is the right choice. They are not like other types.

Records are defined entirely by their content. A triple is not a special object that exists in its own right, it's just three objects in a trenchcoat. The record "object" has no identity, no behavior other than projecting values out of it, and no type other than the combination of the types of the field values. It's reasonable to consider a record object as a boxing of the collection of individual values, rather than records being inherently unboxable objects.

A list is an object with its own behavior, which can include mutability, and its own runtime type. It happens to be containing other objects, but it's not defined entirely by its elements. Because of our reified generics, a List<dynamic> is a different object from a List<int>, even if they contain the same integer elements.
We don't do implicit downcast from List<dynamic> to List<int> because the value might just be a List<dynamic>. And we can't actually check the elements using a single cast.

For a value of type (int, dynamic), we know, with absolute certainty, that the second field is not an object with runtime type dynamic. The runtime type of an (int, dynamic) will be a proper subtype, and a cast to (int, int) will ensure that the one value with static type dynamic has the expected type. It's immutable and entirely covariant.

Another place where it's obvious that a record is just adjacent individual values is in the treatment of Never.
An expression with static type (int, Never) is never going to complete normally. A List<Never> is just an empty list.
Explaining that to users should be simple: A record is just a way to do simple, normal object things in "parallel".

The more we lean into that explanation, the easier things are to explain and understand.

So I believe we should do number 3: Define assignability to include records of the same shape, where the corresponding fields are assignable.
We should do the casts in the same places we do today, which is late (just before the assignment).
That's fine, because any place works if you're just doing assignment.

[leafpetersen]

We don't do implicit downcast from List<dynamic> to List<int> because the value might just be a List<dynamic>. And we can't actually check the elements using a single cast.

For a value of type (int, dynamic), we know, with absolute certainty, that the second field is not an object with runtime type dynamic.

I don't really understand the argument being made here. It's true that the list value might be a List<dynamic>. It's also true that it might not. In other words, the cast might work, it might not. Similarly, for the record type, while the second field is certainly not an object with runtime type dynamic, it could be an object with runtime type Object, or Null or String or anything else, which doesn't seem meaningfully different to me. As with the List case, the cast might work, or it might not.

So what's the principle here? We have a bunch of implicit casts we could do based on the principle "it might work". The principle we've taken as of 2.12 is that we allow a cast from dynamic to anything, but not from composite types that contain dynamic (e.g. FutureOr<dynamic>, void Function(dynamic), etc). Any of these "could work". So why not allow them?

[lrhn]

Ok, I'll be honest: I'd be fine with allowing an implicit downcast from List<dynamic> to List<whatever>. We didn't, and as I remember it there were technical reasons for it too, so it's not a completely arbitrary choice. I'm OK with that, but I'd been fine with allowing the downcast too (if we could make the technical issues go away).

The principle here is that records are not like other things, and I want record assignments and operations to behave like they are single-object operations performed in parallel.
Records don't really exist. They have no identity. They have no behavior. It's just a temporary gathering up of independent values, and when you start using them, it should be like using the independent values. I want to push that view throughout the entire record specification, because I believe it's the best model for explaining and understanding what a record is.

If there is a significant difference in how any of these six record assignments work:

dynamic d = 0 as dynamic;
var p = (1, d);

/*1*/ (int, int) r1 = (1, d);
/*2*/ (int, int) r2 = p;
/*3*/ var (int x1, int y1) = (1, d);
/*4*/ var (int x2, int y2) = p;
/*5*/ int x3 = (1, d).$0,  y3 = (1,d).$1;
/*6*/ int x4 = p.$0,       y4 = p.$1;

then I'll have a hard time explaining why.

Numbers 5 and 6 will definitely work.
Numbers 1-4 will all work if we let record assignability be point-wise assignability. If not, ...
Number 1 will work if we do deep implicit downcast, otherwise not.
Number 2 will not work.
Number 3 and 4 - I don't actually know if they get caught in the type checks of pattern declarations.
(If var [int x, int y] = <dynamic>[1, 2]; works, this should too. I don't know if it does.)

One of these solutions is easy to predict and understand, the one where it doesn't matter in which order you do things.
That's not an accident, it's (as I see it) because it's the most natural behavior, the one where we don't try to introduce complications that do not need to be there.

[leafpetersen]

then I'll have a hard time explaining why.

Fair enough. But there's no getting around this in general. For int to double conversions, 2 and 4 will never work. Similarly for .call tearoffs, and for any future direct syntactic conversions we specify. So making this work for implicit casts, while perhaps more internally consistent, makes those conversions less consistent across the language.

In general, this tends to make me lean in the other direction: for downcasts from dynamic (at least), this feels like a good argument for not pushing the context type into things (or at least into records). In that case, 3-6 work, and 1, 2 don't, and it's easy to explain: if you're assigning something of type dynamic to something of a different type, it just works - otherwise it's an error if it's not a subtype.

At a higher level, I think the consistency argument is reasonable, but I think it's not sufficient: doing the wrong thing consistently is still doing the wrong thing. I'd like to see a positive argument here: why is it generally helpful to users to allow these assignments? In general, my experience with implicit assignments from dynamic is that the make it harder to read the code - you can't trust that what you're seeing is what you're getting. I'm reluctant to expand the set of situations in which we do this.

Note too that we do not do this for other composite types (e.g. FutureOr and Function types). So it's not clear to me that this is even a net win in terms of consistency even ignoring the other kinds of conversions.

[eernstg]

@leafpetersen wrote:

this feels like a good argument for not pushing the context type into things (or at least into records).

Perhaps we should make a distinction between propagating the context type schema into various constructs, and performing code transformations based on differences between the context type schema and the type of the inferred subexpression?

class C<X> {
  void setup(String s) {...}
}

void main() {
  C<int> c = C()..setup('');
}

Surely we don't want C() to be a compile-time error because it doesn't get the context type C<int>.

I tend to prefer the proposals where a mismatch between the context type schema and the inferred expression type triggers a code transformation initially, but with each propagation of a context type schema into a subexpression, the particular shape of the enclosing expression will or will not propagate the ability to perform the code transformation. When specific code transformations are not enabled, the result of a typing discrepancy is usually a compile-time error.

For instance, we could disable the ability to perform .call insertion for the transition from e..cascadeSection to e, but the same transition would preserve the ability to add missing actual type arguments, and the ability to turn an integer literal into 'an integer literal of type double', etc.

From that perspective, it's an open choice whether or not we'd propagate the ability to cast from dynamic into a record literal: We have some code transformations that are enabled in similar situations, and other code transformations that aren't enabled.

I would tend to enable cast from dynamic in record literals, because of the very direct connection between each field of the record literal and the context type schema for that field.

However, we don't have any other code transformations that are similar to the one that transforms a record-typed expression e into let v = e in (v.$0 as int, v.$1), and hence I don't think we should "cast from dynamic" inside any expression e which is not syntactically a record literal. In other words, /*2*/ and /*4*/ in this comment are completely different from the others.

[lrhn]

If we define record assignability as point-wise assignability, it's true that we get into a problem with .call.

It would make (int, CallableClass) assignable to (int, Function), and require an implicit .call tear-off in the assignment (int, Function) p = q; where q has type (int, Function).

We can do that. It means every record assignment can be an implicit destructuring and rebuilding.
It's as if (int, int) r2 = p; is implicitly (int, int) r2 = (p.$0, p.$1); and we apply coercions on every field expression.
That would also apply to .call. It wouldn't apply to "double literals" because those were never runtime coercions to begin with.

So, we can make the number 2 case work. And I'd be in favor of doing that.

Also, number 4, var (int x2, int y2) = p;, is actually format that I would expect to work with .call tear-off. It's a destructuring followed by assignment, so:

(int, CallableClass) p = ...;
var (int x, Function f) = p; 

I'd actually expect the second line to be the same as int x = p.$0; Function f = p.$1;, which is again int x = p.$0; Function f = p.$1.call; because p.$1 is a CallableClass expression assigned to Function.