Late parameters, late-init-query operator, parameter element

[lrhn]

(Because I'm sure I've written this before, but it's probably in a comment of another issue, here is a stand-alone version.)

The copyWith problem is that it's impossible to distinguish an omitted argument from an explicit passing of the default value.
That's generally considered a good thing, but it makes forwarding arguments to another function awkward.

Dart once had a "was argument passed" operator, ?x which was true if x had an argument passed, and false if it got its default value implicitly. It was intended to make it easier to forward arguments to other functions, but since those other functions could now also behave differently depending on whether an argument was passed or not, it actually made it harder. It was quickly removed again.

I propose that we add two things, both a way to see if an argument was passed, and a way to easily pass or not pass an argument in constant code size.

Proposal

Allow optional parameters to be declared late. A late parameter need not be initialized, so it can be optional and not have a default value. (It's not allowed to have a default value, because if it has one, it doesn't need to be late.)

Add a prefix operator ?? (strawman syntax) which applies to a late local variable, and which evaluates to a bool value that is true if the variable is initialized, and false if it's not. (This value affects "definite assignment" analysis, so it's possible to write if (??x) { return x; } else { x = 42; } no matter what the assignment analysis said about the late variable x before.)
(See also #1028. Maybe ?? should only apply to late parameters, but it can be extended to any other late variable.)

Finally, allow argument elements, which are arguments which can evaluate to no value.

<argumentList> ::= '(' <arguments> ')'
<arguments> ::= (<argument>? ',')* <argument>?
<argument> ::= (<identifier> ':')? <argumentElement>
<argumentElement> ::= 
      'if' '(' <expression> '>') <argumentElement> ('else' <argumentElement>)?
      'switch' '(' <expression> ')' '{' <elementSwitchCases> '}'
    | '?' <expression>
    | <expression>
<elementSwitchCases> ::= (<elementSwitchCase> ',')* (<elementSwitchDefault> | <elementSwitchCase>) ','?
<elementSwitchCase> ::= 'case' <pattern> ('when' <expression>) ':' <argumentElement>
<elementSwitchDefault> ::= 'default' ':' <argumentElement>

Anywhere we currently allow an <expression> as an argument, we now use <argument>, which allows omitting a value for that argument.

• The if can have no else branch.
• The switch needs not be exhaustive (unless it's an always-exhaust type, as usual).
• The ? <expression> is a non-value if the expression is null.
• The <argument> itself can be omitted (otherwise ?null would be the shortest "no value" syntax).

(The element-switch and null-aware element ?e should both be added as collection elements as well.)

That allows a call of foo(, if (false) 0, switch (0) { case 1: 0 }, ?null, 42) to call foo with the first four arguments being omitted.
It's the same as foo(,,,, 42). It's an error if a parameter is not optional, and its argument element can evaluate to no value (more precisely: an <argumentElement>? may evaluate to no value if it's empty, if it's an if element that has no else branch, or where the argument element of either branch may evaluate to no value It's empty, if it's a switch element which is not exhaustive, or where any case element may evaluate to no value, or if it's an ?-element).

It's new that non-trailing optional positional arguments can be omitted, and later parameters can be passed.

There is no good way to avoid that property while allowing conditionally passing an argument, not without completely disallowing that for positional arguments. Then we'd disallow late positional parameters too. That's an annoying asymmetry with named parameters, so it's better to just allow non-trailing positional parameters to be omitted. The majority of functions can't tell the difference (yet, without the ?? feature), they'll just get their default values.
(A few functions may deliberately use private sentinel default values to check whether an argument was passed. That's not generally possible, which is why it doesn't solve the callWith problem, but if the parameter type is Object? anyway, any sentinel value can be used. Calling those functions incorrectly may give you a different result than you expected (well, what did you expect?). So don't do that. The functions will probably be rewritten to use ??parameter instead of identical(parameter, _sentinel) soon enough.)

Semantics

The semantics are straight-forward.

• A late parameter with no argument value passed is uninitialized after "binding actuals to formals", behaving just like any other uninitialized local late variable. If an argument was passed, it's initialized to that value. The variable starts as potentially assigned, which means it's not a static error to read or write it, but may be a runtime error. Reading it will throw, if unassigned, writing will succeed if unassigned or non-final. Static type is the declared type. (For a covariant parameter, the parameter's local variable's type is the declared type, even if the function parameter's runtime type is Object?. Same as always.)
• The ??parameterVariable has static type bool, the operator checks whether the late parameter is initialized or not. If used in a conditional position, the operator promotes the variable to definitle assigned on the true branch and definitely unassigned on the false branch. It's a compile-time error to use it on anything other than a late variable name.
• Argument elements are evaluated as would be expected. If they end up with no value (if with false value and no else branch, non-exhaustive switch with no matching case, ?e with a null value, or just an omitted <argument>), then the argument list has no value at that position (or at that name, if named, an omitted <argument> is always counted as a positional argument). This is new, argument lists now need a way to represent "no value" at any position or name, in a way that can be acted on when binding actuals to formals. See first item.

Consequences

It's now possible to write a copyWith:

class Point 
  final int x, int y;
  final Color? color;
  Point(this.x, this.y, {this.color});
  Point copyWith({late int x, late int y, late Color? color}) =>
    Point(??x ? x : this.x, ??y ? y : this.y, color: ??color ? color : this.color);
}

(I'm quite certain that the first request we'll get is to allow ?? as infix, treating an unassigned late variable the same as a null-valued nullable non-late variable. Then copyWith would be:

  Point copyWith({late int x, late int y, late Color? color}) =>
      Point(x ?? this.x, y ?? this.y, color: color ?? this.color);

That's ambiguous when a variable is both nullable and late, since reading the variable is allowed (it only might throw).
More likely the ?? will be saved for special cases, and you can still use null where it's not a valid value:

  Point copyWith({int? x, int? y, late Color? color}) =>
      Point(x ?? this.x, y ?? this.y, color: ??color ? color : this.color);

Still, expect this request.)

It's also now possible to forward any known argument list, without knowing the default values:

Result foo(int x, {late Banana banana, late SecretSauce sauce}) {
  log("foo(${["$x", if (??b) "banana: $banana", if (??sauce) "sauce: $sauce"].join(", ")})");
  return super.foo(x, banana: if (??banana) banana, sauce: if (??sauce) sauce);
}

or from noSuchMethod without an exponential blowup in the number of optional named parameters:

  noSuchMethod(i) {
    if (i.memberName ==  #foo) {
      return target.foo(i.positionalArguments[0] as int,
           banana: if (i.namedArguments case {#banana: Banana banana}) banana,
           sauce: if (i.namedArguments case {#sauce: SecretSauce sauce}) sauce,
      );
   } else {
     return super.noSuchMethod(i);
   }
 }

The forwarding is more signficant than just copyWith. If copyWith was the only problem to solve, allowing non-constant default values is enough:

  Point copyWith({int x = this.x, int y = this.y, Color? color = this.color}) =>
      Point(x, y, color: color);

(Which we should just do. #140! But that does allow seeing whether an argument was passed using side-effects through external variables, so we might still want argument elements to counteract that ability.)

[jakemac53]

I propose that we add two things, both a way to see if an argument was passed, and a way to easily pass or not pass an argument in constant code size.

Definitely agreed that we need both, and I really like that the latter one also helps with default values.

I don't think it fully fixes the default value issue because often it is actually an override, and you aren't just calling super or do need access to the default value even if you are calling super. But, it still helps with some cases.

[jakemac53]

This proposal to me is a nice solution that avoid adding something like undefined, but essentially gives you that power in a limited way using an existing concept (late variables), and without introducing an actual instance of some weird undefined object.

[abitofevrything]

Why not allow late parameters to have an initializer (same syntax as default values) that is evaluated on every invocation if the parameter is not explicitly passed?

The copyWith could then further be reduced to this (omitting some fields as I'm typing this on my phone):

class Point {
  final int x;

  Point(this.x);

  Point copyWith({late int x = this.x}) => Point(x);
}

Late variables are evaluated lazily, so the presence of default value won't prohibit the use of ?? a - unless the variable was accessed before the test:

f({late x=1, late y=2}) {
  print(x); // x gets initialized here
  print(?? x); // true - variable is already initialized
  print(?? y); // false - variable is not initialized yet: asking the question doesn't count as an "access"
}

The example of "print(x)" shows that the confusion between "passed" and "initialized" might be common. For that reason, maybe it's better to adopt a narrow definition: '?? a` means "parameter was passed" regardless of whether it is initialized or not at the moment. Not sure.

[abitofevrything]

I think the ?? prefix operator could be generalised to all late variables - not just parameters - so having it use a parameter-specific term like "passed" might not work great in the future.

I still wouldn't want it to become common practise to have late final variables that have their initialisation checked with ?? though as that goes against the spirit of late final. Better have a nullable field.

[abitofevrything]

I'm less enthusiastic about the "adding more places a variable can be uninitialised" part of this proposal though. My opinion is that uninitialised variables are dangerous, with better alternatives available.

Not having to pass a value for a parameter is fine, but it should only be allowed for parameters that can already be omitted (optional positional and non-required named parameters). For the named parameters, it's completely useless as you can just not pass the named parameter.

There should probably also be a lint encouraging people to use named parameters instead of optional positional ones if they do find themselves having to omit a value to "skip" a positional parameter. The foo(,,) syntax likely looks like a syntax error to most people.

[mateusfccp]

I'm less enthusiastic about the "adding more places a variable can be uninitialised" part of this proposal though.

Maybe you just want #140, then?

[jakemac53]

I think the ?? prefix operator could be generalised to all late variables - not just parameters - so having it use a parameter-specific term like "passed" might not work great in the future.

+1 it is annoying today that you can't check the initialized status of late variables, and it means you can't use them in places you might like to. I think extending this proposal to all late variables helps solve multiple problems, which is always a good sign for any proposal.

[mmcdon20]

If we have both late parameters and non-constant default values, then I think you should allow late parameters to have default values also.

This could allow you to for example assign a late parameter to the result of an expensive function call that may or may not get executed due to lazy evaluation.

---

EDIT: Correct me if I am wrong, but I assume a late parameter would allow you to short circuit an argument, for example:

bool and(bool a, late bool b) => a && b;
bool or(bool a, late bool b) => a || b;

bool truePrint(String s) {
  print(s);
  return true;
}

void main() {
  and(true, truePrint('A')); // side effect, prints A
  and(false, truePrint('B')); // no side effect, second argument never evaluated
  or(true, truePrint('C')); // no side effect, second argument never evaluated
  or(false, truePrint('D')); // side effect, prints D
}

[Mike278]

A workaround I use for "the copyWith problem" is just to generate separate with$NAME methods:

void main() {
  final x = Foo('some', 1);
  final y = x.withBar('y');
  final z = x.withBaz(null).withBar('another');
  print('$x\n$y\n$z');
}

class Foo {
  final String bar;
  final int? baz;
  Foo(this.bar, this.baz);

  Foo withBar(String newBar) => Foo(newBar, baz);
  Foo withBaz(int? newBaz) => Foo(bar, newBaz);

  @override
  String toString() => 'Foo(baz=$bar, baz=$baz)';
}

The intermediate objects obviously aren't ideal performance-wise but IMO in practice it's pretty uncommon that you're updating a large number of fields at once, and IIUC Dart's garbage collector is optimized for large numbers of short-lived objects. If it started to become a problem I can imagine a few optimizations for chaining multiple withs together.

[mateusfccp]

As an alternative to the ??argument syntax, we could have something like in Common Lisp, where a new name is introduced:

(defun foo (a b &optional (c 3 c-supplied-p))
  (list a b c c-supplied-p))

In Dart, we could reuse some token to indicate this (strawman using +):

class Point 
  Point(this.x, this.y, {this.color});

  final int x, int y;
  final Color? color;

  Point copyWith({
    late int x +hasX,
    late int y +hasY,
    late Color? color +hasColor,
  }) {
    return Point(
      hasX ? x : this.x,
      hasY ? y : this.y,
      color: hasColor ? color : this.color,
    );
}

[Cat-sushi]

I see some relation with #877.

[Cat-sushi]

@abitofevrything

Why not allow late parameters to have an initializer (same syntax as default values) that is evaluated on every invocation if the parameter is not explicitly passed?

I think late doesn't necessarily mean lazy like top level variables.
So, the syntax you proposed is confusing for me.