Use information from @overload to better model narrowing in implementation

[ngnpope]

Bug Report

There are potentially two issues that I have encountered:

• Consider a case where we use @overload and, for example, two arguments have narrower types than the signature of the non-@overload-decorated definition. If one of those is a Literal type and we branch on a comparison to the value of that literal type, the other argument isn't narrowed to the type(s) defined in the matching overloaded function(s).
  • This can be worked around by using isinstance(), but it seems redundant.
  • It seems, according to Python's documentation that the current behaviour is sort of incorrect as it states:

    The @overload-decorated definitions are for the benefit of the type checker only, since they will be overwritten by the non-@overload-decorated definition, while the latter is used at runtime but should be ignored by a type checker.

  • But until either the literal comparison occurs or isinstance() is checked on the other argument, it's not possible to narrow, but it should be after.
• The other issue is that, once we have called isinstance() to narrow the type of a value, a subsequent nested function definition keeps the non-narrowed type when referring to the value.

See the reproducer below which should help clarify the above issues.

To Reproduce

See playground example.

from typing import Literal, overload

@overload
def f(name: Literal["integer"], value: int) -> int: ...

@overload
def f(name: Literal["string"], value: str) -> str: ...

def f(name: Literal["integer", "string"], value: int | str) -> int | str:
    if name == "integer":
        reveal_type(value)
        assert isinstance(value, int)
        reveal_type(value)

        def do_integer() -> int:
            reveal_type(value)  # XXX: Expected `builtins.int`
            return value        # XXX: Expected no error.

        reveal_type(value)

        return do_integer()

    if name == "string":
        reveal_type(value)
        assert isinstance(value, str)
        reveal_type(value)

        def do_string() -> str:
            reveal_type(value)  # XXX: Expected `builtins.str`
            return value        # XXX: Expected no error.

        reveal_type(value)

        return do_string()

Expected Behavior

Something like this mocked up output if the the type of value were narrowed based on the literal of name:

❯ mypy --strict bug.py 
bug.py:11: note: Revealed type is "builtins.int"
bug.py:13: note: Revealed type is "builtins.int"
bug.py:16: note: Revealed type is "builtins.int"
bug.py:19: note: Revealed type is "builtins.int"
bug.py:24: note: Revealed type is "builtins.str"
bug.py:26: note: Revealed type is "builtins.str"
bug.py:29: note: Revealed type is "builtins.str"
bug.py:32: note: Revealed type is "builtins.str"
Success: no issues found in 1 source file

Something like this mocked up output if only the type inside the nested function respected the narrowing performed by the isinstance() calls:

❯ mypy --strict bug.py 
bug.py:11: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:13: note: Revealed type is "builtins.int"
bug.py:16: note: Revealed type is "builtins.int"
bug.py:19: note: Revealed type is "builtins.int"
bug.py:24: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:26: note: Revealed type is "builtins.str"
bug.py:29: note: Revealed type is "builtins.str"
bug.py:32: note: Revealed type is "builtins.str"
Success: no issues found in 1 source file

Actual Behavior

❯ mypy --strict bug.py 
bug.py:11: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:13: note: Revealed type is "builtins.int"
bug.py:16: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:17: error: Incompatible return value type (got "Union[int, str]", expected "int")  [return-value]
bug.py:19: note: Revealed type is "builtins.int"
bug.py:24: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:26: note: Revealed type is "builtins.str"
bug.py:29: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:30: error: Incompatible return value type (got "Union[int, str]", expected "str")  [return-value]
bug.py:32: note: Revealed type is "builtins.str"
Found 2 errors in 1 file (checked 1 source file)

Your Environment

• Mypy version used: 1.0.0
• Mypy command-line flags: --strict
• Mypy configuration options from mypy.ini (and other config files): N/A
• Python version used: 3.10.9

[loopj]

The latter issue sounds like a dupe of #2608

[hauntsaninja]

Using the type information from the overloads inside the definition of the overloaded function is quite complicated. In general, mypy is rarely able to keep track of relationships of types of multiple variables. I'm not awaare of a type checker with this feature.

The Python documentation is maybe poorly worded, the intention there is probably that the types on the overload implementation are not visible to callers.

[ngnpope]

(Was doing a drive-by review of some of my open issues and there has been some improvement here...)

The latter issue sounds like a dupe of #2608

Agreed. The nested functions after the isinstance() calls are properly narrowed now which was fixed in v1.4.0.

Using the type information from the overloads inside the definition of the overloaded function is quite complicated.

I see.

Actual Behavior

Playground Example

❯ mypy --strict bug.py 
bug.py:11: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:13: note: Revealed type is "builtins.int"
bug.py:16: note: Revealed type is "builtins.int"
bug.py:19: note: Revealed type is "builtins.int"
bug.py:24: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:26: note: Revealed type is "builtins.str"
bug.py:29: note: Revealed type is "builtins.str"
bug.py:32: note: Revealed type is "builtins.str"
Success: no issues found in 1 source file

Your Environment

• Mypy version used: 1.7.1
• Mypy command-line flags: --strict
• Mypy configuration options from mypy.ini (and other config files): N/A
• Python version used: 3.11.5