How should non-nullability handle fields that are initialized after object creation?

[leafpetersen]

A pattern that comes up with some frequency is to have fields in an object which is not initialized in the constructor (or at least not in the initializer list), but which should never be observed to be null once some further initialization is done. How should Dart with non-nullable types handle this? Some options that have been used in other languages:

• Kotlin style lateinit: mark a field as "will be initialized before use". The programmer writes the code as if it is non-nullable, and the compiler checks uses that it can't prove are safe.
• Swift style implicitly unwrapped types. Swift allows you to mark a type as nullable, but allowed to be accessed as non-nullable.
• Pre-post conditions on methods (allow methods to specify in some way that they initialize certain fields, and other methods to specify that they require certain fields to be initialized).

cc @lrhn @eernstg @munificent

[srawlins]

I think there is a request to support a "late initialize" for finals as well, including final locals, though I can't find it. e.g.

void f(bool x) {
  final int i;
  if (x) {
    i = 7;
  } else {
    i = 42;
  }
  // ...
}

This could be a very similar request; could maybe be solved with one solution.

Broadly, declaring locals with type foo; of course cannot be elevated to be non-nullable without really rearranging the code. If a solution for fields will be inapplicable to locals, it would be great to have a similar thread about locals.

[leafpetersen]

I think there is a request to support a "late initialize" for finals as well, including final locals, though I can't find it. e.g.

For locals, I think that we will likely have some form of definite assignment analysis to allow you to write code like your example. It's an interesting question as to whether this should be allowed for final variables as well: it seems to me that it would be nice, but on the other hand single assignment definite analysis is a bit trickier.

[kasperpeulen]

Kotlin style lateinit: mark a field as "will be initialized before use". The programmer writes the code as if it is non-nullable, and the compiler checks uses that it can't prove are safe.

Kotlin also allows to initialize final non-nullable fields in the constructor, even without marking them as lateinit. Dart doesn't support this either, which always gives me a good bit of frustration as this often means you need to write a factory constructor if you want final fields. Kotlin supports this, and it works the same for final locals as in @srawlins example.

fun main(args: Array<String>) {
    print(MyClass(true).i);
    f(false);
}

class MyClass(x: Boolean) {
    val i: Int;
    init {
      if (x) {
       	i = 7;
      } else {
      	i = 42;
      }
    }
}

fun f(x: Boolean) {
    val i: Int;
    if (x) {
        i = 7;
    } else {
    	i = 42;
    }
    println(i);
}

I think supporting this style is more elegant then lateinit, and should handle most cases.
Dart now implicitly assigns null in both cases.
I think it would be great if Dart stops implicitly assigning null to variables.
In most cases this is not what you want.

Note that the value of i is not given the value undefined as in javascript. It gives an error if you try to read the variable:

[munificent]

I think we can split this into a few problems:

Analyzable

These are storage locations (variables, fields, top-level variables) that aren't initialized at their declaration but where we can reasonably statically determine that it is assigned before use.

For local variables, definite assignment analysis works for most common cases:

void f(bool x) {
  var i;
  if (x) {
    i = 7;
  } else {
    i = 42;
  }
  print(i);
}

Whether you can make a local like this as final or not is orthogonal, I think.

For instance fields, Dart does have a solution, factory constructors and constructor initialization lists. @kasperpeulen, I think you're right to point out that it's not a very usable solution, but it's there. Given that, I think we should move discussion of improving this to a separate issue, or just not worry about it for now.

The good thing about analyzable locations is that we can make them soundly non-nullable without needing the performance cost of any runtime checking to validate that it isn't null.

The bad part is that we can't reasonably do this for all storage locations. The analysis just isn't feasible, or would require adding too much complexity to the language (i.e. some kind of complex typestate mechanism that tracks which fields for a given instance have been known to be initialized) to be worth it.

A variable in this category:

• Has a non-nullable getter.
• Has a non-nullable setter.
• Never needs to promote. It's always non-nullable.
• Needs no runtime checking ever.

"Latched non-null"

These are storage locations where we can't statically prove that one will be initialized to a non-null value before its read. But we can statically prove that it will only ever be assigned a non-null value.

This means that if you ever read it and get a non-null result, you know it will be non-null forever after. That implies that we could safely promote it:

class Box {
  // Not real syntax.
  @latched int value;
}

main() {
  var box = Box();
  print(box.value); // "null".
  box.value = null; // Static error. Setter is non-nullable.
  box.value = 3; // OK.

  if (box.value != null) {
    box.value.isEven; // OK. Can promote.
  }
}

We don't even need to do additional runtime checking inside the promoted body. For this to work, we do need more control over overriding to ensure that there couldn't be some malicious subclass of Box that overrides the value getter to return null after the setter has been called.

So this basically means a "latched variable":

• Has a nullable getter.
• Has a non-nullable setter.
• Can soundly promote after checking.
• Needs to store a bit indicating whether or not it is null so that you can do null checks. Does not need runtime checking after promotion.

"Non-null by policy"

Then I think we may also run into cases where a variable may get assigned null, but where the code has some surrounding policy that ensures its never used when it is null and where the overhead of manually null-asserting on every use would be frustrating.

These are addressed by Swift's "Implicitly Unwrapped Optionals". For example:

class Border {
  int _thickness;
  Color _color;

  Border(this._thickness, [this._color]) {
    if (_thickness > 0 && _color == null) {
      throw ArgumentError("Must have color if border has non-zero thickness.");
    }
  }

  void draw() {
    if (_thickness > 0) {
      paintBox(_thickness, _color.red, _color.green, _color.blue);
    }
  }
}

There's a policy here, and the class enforces it, but the type system can't see it.
We could not have language support to make cases like this easier. I think that's probably the best plan to start with.

But, if stuff like this comes up often, we could add another kind of implicitly checked variable that:

• Has a nullable getter.
• Has a nullable setter.
• Cannot soundly promote after checking.
• Needs to be checked at runtime before using or assigning to something non-nullable.

It's basically syntax sugar to not require ! every time you use it, but it has all of the unsafety and performance cost of a nullable type.

[leafpetersen]

Kotlin also allows to initialize final non-nullable fields in the constructor, even without marking them as lateinit.

Note that Kotlin constructor initialization is also not null-sound though - you can see and call methods on non-nullable variables in their uninitialized state.

[kasperpeulen]

@munificent

So this basically means a "latched variable":

• Has a nullable getter.

I don't understand why you would want this. Kotlin gives a runtime error, if you read or use it before it is initilized. UninitializedPropertyAccessException That seems better than to allow it to have a nullable getter.

I think your model is that it assings null first, and then this type is later promoted. But I have the feeling that is not what happens. In Kotlin, you don't implicitly assign null if you assign nothing, even not if the type is nullable:

I think a better anology is that Kotlin implicitly assings something like an JS undefined value to the variable. And when you try to read undefined, it will be a compile time error and with lateinit it will raise an UninitializedPropertyAccessException.

@leafpetersen

Note that Kotlin constructor initialization is also not null-sound though - you can see and call methods on non-nullable variables in their uninitialized state.

That is not true. It is a compile time error if you see and call methods on non-nullable variables in their uninitialized state. With lateinit it is a runtime exception.

[leafpetersen]

@kasperpeulen

That is not true. It is a compile time error if you see and call methods on non-nullable variables in their uninitialized state. With lateinit it is a runtime exception.

This seems to disagree with you, and the following Kotlin program compiles, runs and prints out the value of an uninitialized non-nullable field in the Kotlin playground.

fun main(args: Array<String>) {
    print("${TestClass().v}\n")
}

class AA() {
    override fun toString() : String { return "AA" }
}
class TestClass() {
    val v: AA;
    init {
      setup();
      v = AA();
    }
    fun setup() {
        print("$v + \n");
    }
}

[kasperpeulen]

@leafpetersen
Ah, that’s right. So I guess if the compiler fails to recognize that it is not initialized, then it does compile and it indeed is null.

I think I was mistaken in my analogy as you have to normally explicitly set it to null. A better anology would then be Dart’s ‘void’, as the compiler tries to make sure you don’t use a void variable but at runtime it has no special value, if you “escape it”.

However, I would rather have a runtime exception like with lateinit, if the compiler fails to recognize.

[srawlins]

LMK if locals should be a separate discussion, but I had a thought about how this will affect test code:

void main() {
  Completer<bool> c1;
  List<int> l1;
  // ...
  setUp(() {
    c1 = ...;
    l1 = [1, 2, 3];
  });
  // tests use and modify c1, l1, whose values and state are always set/reset between tests.
  test('list remove removes', () {
    l1.remove(1);
    expect(l1, not(contains(1));
  });
}

(E.g. this typical test in angular)

This is a super common pattern in all testing, and it would be great if users didn't have to make them all nullable (both in the short term for the migration pain, and in the long term, for the benefits of NNBD.

But this code is not analyzably non-nullable (from Bob's analyzable above); it could be encoded as latched non-nullable.

Non-null by policy, @munificent, would just mean that these are declared as Completer<bool>? and List<int>?, right? In order to support the latched concept, we'd need something like latched Completer<bool>? If that's the case, then there will either be a huge migration to question marks or latched annotations.

[leafpetersen]

This is a super common pattern in all testing, and it would be great if users didn't have to make them all nullable (both in the short term for the migration pain, and in the long term, for the benefits of NNBD.

Well, the problem is that without some syntax, I don't know how to tell the difference between that code (for which you would want runtime checking) and other cases (for which you want the compiler to tell you that you've forgotten to assign a local). Unless you're suggesting that all locals be latched by default? This seems a bit loose, but maybe.

I guess another option would be to say that

• it's a warning if definite assignment fails
• a local that isn't definitely assigned becomes latched by default

Then you could suppress definite assignment warnings for test files.

[srawlins]

Unless you're suggesting that all locals be latched by default? This seems a bit loose, but maybe.

No, I'm not suggesting that, but it sure sounds like I am 🙁. I think I just painted myself into a corner and realized there will probably be a migration for tests with uninitialized variables declared in main.

If we say that an NN local declared without an assigned value is legal (because it is latched), that might be confusing... sort of changes the whole idea of NN. And I imagine back-ends want to weigh in on the whole latching concept anyhow. I'm not casting a vote in any direction here, just noting that this will probably require that every test file go through a migration for uninitialized locals.

[munificent]

I don't understand why you would want this. Kotlin gives a runtime error, if you read or use it before it is initilized. UninitializedPropertyAccessException That seems better than to allow it to have a nullable getter.

There are two different use cases:

1. A field that may not be initialized until later. Users may access the field before it's initialized and see that it's null. In fact, seeing that it's null is how they tell if it's initialized. But, it can only be assigned a non-null value. Once it has, you know for certain that it will never be null again, which makes it safe to type promote on it.

2. A field that may not be initialized until later. It's a programmatic bug to try to access it before it has been initialized. You don't even want users to be able to write code that determines at runtime whether or not it has been. It should just throw an exception.

The "latched non-null" I describe is for the former, but I think you have in mind the latter. For the latter, something more like the "non-null by policy" is a better fit.

[lrhn]

The descriptions here sound very much like Dart's lazily initialized static fields. If you read it before you write it, it executes some code to get the initial value.

We could define lazily initialized instance fields, or even local variables, in almost the same way: If you read before writing, some code is run. That code can either initialize the variable lazily or (if the initializer expression is omitted) just throw an UninitializedError. If you write first, it's fine and no initialization will happen. That allows you late-init and lazy-init in the same language feature, with late-init just being a throwing lazy-init initializer.

That approach would be using an existing language feature in a more general way, rather than inventing a new one (which is great, because we don't want too many similar features). The only issue is that static fields are lazy by default, but instance fields or local variables are not, so we'd need more syntax to opt-in to it. Some options:

• Make lazy a built-in identifier and write lazy int x;. Or lateinit or some other word.
• Make non-nullable variables with no initializer late-init. That leaves no way to get lazy initialization, though.
• Invent a new assignment operator: int? x =#= initializer and use that for for lazy initialization (but it's annoying that it's different from static fields).
• Prefix the variable somehow: int *x = ... or int *x;. Works both with and without initializer, but not intuitive (Not for any character in !@#%^&*\ - some even look like they mean something else).

We actually do need to modify what we do to static fields too, because if the initialization fails (throws, perhaps due to cyclic references), then it currently stores null in the field. We should probably not do that any more, since we can't always store null. I'd say that failure during initialization should keep the field uninitialized, so the next access also fails.