while x is not None not recognised by Optional type

[dlehmenk]

Bug Report

I have a list with type List[Optional[X]] and a loop over it:

from typing import List, Optional

a: List[Optional[str]] = [None, None, None]

for node in ['a','b','c']:
     o = len(node)

     while a[o] is not None:
         somestr: str = a[o]

(yes, this does absolutely nothing, for a more complete example see mypy_bug.py.txt. But the bug is the same.)

Mypy shows the following error: tmp.py:9: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str"). But a[o] can never be None, because of the loop, and thus has to be str, if this code is executed.

To Reproduce

Check the code posted above with mypy.

Expected Behavior

No error should be shown, since the types are correct (IMHO)

Your Environment

• Mypy version used: 0.761
• Mypy command-line flags:
• Mypy configuration options from mypy.ini (and other config files):
• Python version used: Python 3.8.5
• Operating system and version: Ubuntu 20.04.2

[kbrose]

Another example of this behavior. I was caught off guard because the variable was inferred correctly as not None when used in a simple way, but not when used inside a lambda:

from __future__ import annotations


def func(x: int | None):
    while x is not None:
        reveal_type(x)  # Revealed type is "builtins.int"
        x * 5  # passes typechecking
        (lambda y: x * y)(5)  # fails typechecking: Unsupported operand types for * ("None" and "int")

(python 3.9.1, mypy 0.910)

[hauntsaninja]

@kbrose your issue is different, it's https://mypy.readthedocs.io/en/stable/common_issues.html#narrowing-and-inner-functions (more discussion in #2608 )
@karoshi42 while you are correct, I don't think mypy's going to be smart enough to figure it out any time soon. Your issue is a slightly more complicated duplicate of #7339.

[erictraut]

@karoshi42, subscript expressions with dynamic indices are difficult for any static type checker to narrow because the analyzer needs to prove that the index expression remains the same between the point where it's tested and where it's accessed again. Most static type checkers in most languages therefore support narrowing only for subscript expressions that use constant indices. That's true of mypy and pyright, for example.

The workaround I recommend is to use a temporary variable and an assignment (walrus) operator. Not only does this eliminate the type error, but it also makes your loop run faster.

    while (x := a[o]) is not None:
        somestr = x