Using `if (foo?.bar == somethingNotNull)` should promote `foo`

[matanlurey]

Another surprise (to me):

void main() {
  String? foo = 'Foo';
  if (foo?.toUpperCase() == 'FOO') {
    // Error: foo might be null.
    print(foo.toUpperCase();
  }
}

[lrhn]

The only value comparisons which promote are != null and == null, and only on the variable itself.
So foo != null promotes foo, but foo == 'bar' does not, and foo?.something != null doesn't either.

Here you are deriving that because foo?.toUpperCase() is equal to a non-null value, then it must be a non-null value, and therefore foo must be non-null too. That's not expected to happen.

So, working as intended.

[matanlurey]

Why isn't that expected to happen? The above is a contrived case, but patterns like this are common enough:

if (foo?.isDelicious() == true) {
  print(foo.describeDeliciousness());
}

[rakudrama]

I find it slightly surprising that this does not work. Promotion of a local would happen if you manually desugared

if (foo?.something != null) ...

to

if (foo != null && foo.something != null) ...

so why should you have to do that yourself?

Not promoting foo == 'bar' makes some sense since without looking at the definition, you don't know what operator== does. We perform constant-folding on strings, so why not incorporate more properties of the primitive types into promotion?

With foo?.something == 'bar', along one path of possibilities you have null as an operand to ==.

[lrhn]

I'm not surprised we don't promote because I have memorized which operations can promote, and this isn't one of them.
There are lots of operations that imply something being non-null, and we don't recognize all of them, only the most direct ones.

If we consider foo?.bar ... equivalent to foo != null ? foo.bar ... : null for local variables (and equivalent to let x = foo in x != null ? x.bar ... : null for not-local-variables), then we could do promotion from the implicit, equivalent foo == null.
We haven't specified it, so we don't.

Generally, I really, really don't want to reason about code from syntactic equivalences. It gives surprising behavior (like ++x?.foo short-circuiting the ++ because it's equivalent to to x?.foo += 1, but we don't short-circuit other prefix operators. I think this was a bad result of using equivalences as basis for reasoning).
So, if we can generalize the rules so that localVar?.selectors != null promotes localVar without resorting to equivalence to something containing foo != null, then I'd be fine with doing so, but I don't think it's viable to try adding it to the initial release of null safety.

We also don't promote foo to non-null for foo == "bar" at all. We could, but we haven't specified it, so we don't.

https://nullsafety.dartpad.dev/5a783a27daed945ead642728a80229c6

[matanlurey]

Respectfully whether you personally want to reason about it isn't really the point of this issue - at least me and Stephen (likely more) consider this valid.

I suspect this pattern is very common across both external and Google packages and unless we intentionally want users to hand desugar to the longer-form code we should consider improving this.

I'm happy to wait for other responses, thanks!

[natebosch]

My intuition is that this feature would make users less confused than they would be without it. I'd be inclined to push on it, especially if we think we can do it relatively quickly. If we think it will take too long we should consider whether it will be breaking to release it later. @leafpetersen

[matanlurey]

Totally! I could also see at minimum adding some information to our migration guides (/cc @munificent) that highlight "these are cases you'd think would promote, but don't, here are some tracking issues for future versions of Dart". Otherwise I think users are going to think they did something wrong.

[lrhn]

Maybe I'm just not sure what the point of this issue is.

The described behavior is correct according to the specification, so all tools are working as intended and required. That makes it neither a bug, nor a tool enhancement issue, except perhaps to give better messages when encountering the pattern. (Read: I can't triage the issue because I don't know what action is requested.)

If we want to change the specified behavior, we should move the issue to the language repository.

[leafpetersen]

cc @stereotype441 Am I correct that this is one of the cases that would be handled if we explicitly track the "nonNull" sets?

[stereotype441]

cc @stereotype441 Am I correct that this is one of the cases that would be handled if we explicitly track the "nonNull" sets?

@leafpetersen Yes, I believe it would, and I would actually really enjoy the mental exercise of making it work.

But it would be quite a pandora's box to open, because the formulation we had with null() and nonNull() had a logical reasoning error that we never got around to patching: null(E) and nonNull(E) tracked the answers to the questions "what would be known about the state of all variables, at the time evaluation of E completes, if E evaluated to a null / non-null value?" But then, when analyzing an expression like E1 == E2, we were trying to synthesize information from null(E1)/nonNull(E1) and null(E2)/nonNull(E2), glossing over the fact that those models represented the state of the program at different times. Which meant that flow analysis would have produced unsound results if E2 had had any side effects on local variables.

If I were going to tackle this idea I would play with conservativeJoin to see if I could use it to cover up any soundness holes that arose from side effects of E2. I'd probably give myself about a 60% chance of success with that approach.

As a backup plan, I might consider using null()/nonNull(), but only trying to promote on equality checks of the form E1 == null, E1 != null, null == E2, and null != E2. We'd be able to handle those checks soundly because evaluating the literal null has no side effects. I'm pretty sure that would be sufficient to address this particular issue, but it would still fail to promote some expressions that theoretically could be promoted (e.g. a?.b == c + d, which in principle could promote a to non-nullable).

[munificent]

I'm not sure what the general behavior you're proposing is. There are plenty of other cases where a ?. in a subexpression of an if condition does not imply that when the overall condition is true then the receiver must have been non-null:

void main() {
  String? foo = 'Foo';
  if (foo?.toUpperCase() == null) { ... }
  if (foo?.toUpperCase() is Object?) { ... }
  if (foo?.toUpperCase().isEmpty ?? true) { ... }
  if (foo?.toUpperCase() == foo?.toUpperCase()) { ... }
  if ([foo?.toUpperCase()].isNotEmpty) { ... }
  if (Future.value(foo?.toUpperCase()) is Future) { ... }
}

What are the rules you have in mind for when ?. causes promotion on the receiver and when it doesn't?

[leafpetersen]

What are the rules you have in mind for when ?. causes promotion on the receiver and when it doesn't?

The general principle that I think @matanlurey and @rakudrama are going from is that:

• if you have a e1 == e2
• and e1 being non-null necessarily implies that some variable x is non-null
• and e2 is non-null
  - then you know that if the equality evaluates to true, then x is non-null.

One way to see this is:

• The combination of e2 not being null and e1 == e2 true implies e1 is non-null
• If e1 is non-null, then by hypothesis, we know that x is non-null

The specific instance of

• and e1 being non-null necessarily implies that some variable x is non-null
  that they are going from is that e1 is foo?.method():
• foo?.method() being non-null implies that foo is non-null.

That said, I'm really quite surprised that this is considered something that "obviously should work". You guys are better at reasoning about nullability than I am... I had to think about that one for a bit. :) That said, I do see the reasonableness of the foo?.isDelicious pattern that @matanlurey describes above.

I'm skeptical about pushing on this right now, we really need to get this feature out the door. If we do tackle this (now or later), I think it would likely be on a somewhat ad hoc basis. Presumably we want to notice that one of the arguments of the equality was "obviously" a non-nullable literal, and use that to gather some promotion information while traversing the other argument.

[stereotype441]

I'm not sure what the general behavior you're proposing is. There are plenty of other cases where a ?. in a subexpression of an if condition does not imply that when the overall condition is true then the receiver must have been non-null:

void main() {
  String? foo = 'Foo';
  if (foo?.toUpperCase() == null) { ... }
  if (foo?.toUpperCase() is Object?) { ... }
  if (foo?.toUpperCase().isEmpty ?? true) { ... }
  if (foo?.toUpperCase() == foo?.toUpperCase()) { ... }
  if ([foo?.toUpperCase()].isNotEmpty) { ... }
  if (Future.value(foo?.toUpperCase()) is Future) { ... }
}

What are the rules you have in mind for when ?. causes promotion on the receiver and when it doesn't?

Not sure who you're asking, but speaking for myself, whatever we do my number 1 requirement is that it has to be sound, and in all your examples there, it wouldn't be sound to promote because the if test would pass if foo is null.

The thing Leaf and I were talking about with null() and nonNull() is essentially what he proposed for flow analysis before I ripped it out to make it match the current implementation. It would involve reverting b901765 and then doing some yet-to-be-determined thing to fix the soundness issue I mentioned in my previous comment. The basic idea is that for each expression, flow analysis would track two additional pieces of information: (a) what do we know about the types of local variables if this expression evaluates to a non-null value?, and (b) what do we know about the types of local variables if this expression evaluates to a null value? That may sound like a lot but it's actually a pretty trivial extension of how today we track what we know about the types of local variables if expressions evalute to true/false. And then we would propagate this information through the code using similar rules to the rest of what flow analysis does.

So, for the example of if (foo?.bar == somethingNotNull), flow analysis would trivially know that if foo evaluates to a non-null value, that means that foo must have been a String. Propagating that through ?.bar, it would conclude that if foo?.bar evaluates to a non-null value, that means that foo must have been a String. Propagaing that through == somethingNotNull, it would conclude that if foo?.bar == somethingNotNull evaluates to true, that means that foo must have been a String. Therefore, foo gets promoted to String in the body of the if. But for your other examples, the chain of reasoning would break down in one way or another and we wouldn't do any promotion.