False positive on _ (underscore) as variable name with disallow-any-expr

[lkct]

Bug Report

mypy gives a false positive with variable _ (underscore) under certain circumstances (see repro).

To Reproduce

https://mypy-play.net/?mypy=latest&python=3.8&flags=disallow-any-expr&gist=bca874771ec5eba30815133c9bbd4ab7

from typing import Callable, Tuple, TypeVar

T = TypeVar("T")

def wrapper(fn: Callable[[], T]) -> Tuple[T, float]:
    return fn(), 1

def wrapper2(fn: Callable[[], T]) -> T:
    return fn()

def func() -> int:
    x = 1
    return x

def a() -> None:
    _, c = wrapper(func)  # <- 2 errors here
    b, c = wrapper(func)
    _ = wrapper2(func)

_, c = wrapper(func)

Interestingly only one line causes the error:

• statement inside a function
• variable named _
• wrapper returns a tuple

Actual Behavior

main.py:16: error: Expression type contains "Any" (has type "Tuple[Any, float]")  [misc]
main.py:16: error: Expression has type "Any"  [misc]
Found 2 errors in 1 file (checked 1 source file)

Your Environment

(environment of the playground)

• Mypy version used: 1.3.0
• Mypy command-line flags: --disallow-any-expr
• Python version used: 3.8 (same for other versions)

[AVUKU-PRAGATHESWARI]

The code provided contains two errors when calling the wrapper function. I will explain each error and provide a corrected version of the code.

Error: Missing argument in wrapper function call
The line _, c = wrapper(func) is missing the argument fn in the wrapper function call. The wrapper function expects a callable function as an argument, but it is not provided in this case.

Correction:
To fix this error, you need to pass the func function as an argument to the wrapper function. Here's the corrected code:

Error: Incorrect number of values to unpack
The line b, c = wrapper(func) tries to unpack two values from the return value of wrapper, but the wrapper function actually returns a single value wrapped in a tuple.

Correction:
To fix this error, you can modify the wrapper function to return a tuple containing the result and a default value for the float. Alternatively, you can modify the line to only assign the first value of the tuple. Here's the corrected code using the second approach:

`from typing import Callable, Tuple, TypeVar

T = TypeVar("T")

def wrapper(fn: Callable[[], T]) -> Tuple[T, float]:
return fn(), 1.0

def wrapper2(fn: Callable[[], T]) -> T:
return fn()

def func() -> int:
x = 1
return x

def a() -> None:
_, c = wrapper(func)
b = wrapper(func)
_ = wrapper2(func)

_, c = #`wrapper(func)``

[AVUKU-PRAGATHESWARI]

from typing import Callable, Tuple, TypeVar

T = TypeVar("T")

def wrapper(fn: Callable[[], T]) -> Tuple[T, float]:
return fn(), 1.0

def wrapper2(fn: Callable[[], T]) -> T:
return fn()

def func() -> int:
x = 1
return x

def a() -> None:
_, c = wrapper(func)
b = wrapper(func)
_ = wrapper2(func)

_, c = wrapper(func)

I hope this code will rectify the error..

[lkct]

@AVUKU-PRAGATHESWARI Sorry, I don't think your answer is correct.

(And, no offence, but I doubt if I'm talking to human.)

[AVUKU-PRAGATHESWARI]

I am a human😊...But its fine

[A5rocks]

Here's a more minimal reproduction!:

# flags: --disallow-any-expr
from typing import Tuple, TypeVar

T = TypeVar("T")

def f(x: T) -> Tuple[T, float]: ...

def a() -> None:
    _ = f(42)

It only shows a single error, though. ... Oh, it's cause this:

mypy/mypy/semanal.py

Lines 3737 to 3741 in 2ede35f

	if self.is_func_scope():
	if unmangle(name) == "_":
	# Special case for assignment to local named '_': always infer 'Any'.
	typ = AnyType(TypeOfAny.special_form)
	self.store_declared_types(lvalue, typ)

Uhhhhhhhhhhh, I'll think about this.

[A5rocks]

For what it's worth, widening the special case that allows wrapper2 to work does not work, unfortunately. I tried that and a bunch of type errors across mypy's codebase popped up :(

[A5rocks]

@ilevkivskyi you initially added this special case in 626ff689e6df171e1aec438b905efdf999bdc09e

Do you know of how it could maybe generalize? Or if this is just some issue that takes more effort than warranted. I see you added a comment right above about some sort of solution, but I'm not exactly sure of what this entails or whether maybe there's a better way nowadays:

            #   * We need to update all the inference "infrastructure", so that all
            #     variables in an expression are inferred at the same time.
            #     (And this is hard, also we need to be careful with lambdas that require
            #     two passes.)

Specifically, mypy is seeing a ret_type of T'1 for wrapper2, and a ret_type of tuple[T'1, float] for wrapper. erased_ctx is Any for both. This means wrapper2 lucks into having a special case meaning it doesn't get modified, but for wrapper, T'1 gets constrained to Any and that gets inferred through.

[A5rocks]

@ilevkivskyi again in case you haven't seen this yet

I found another case of this in some code a friend wrote

[ilevkivskyi]

That special-casing is really fragile, I wouldn't modify it (either way) unless really needed.

I think the problem here may be that kind of Any gets lost somehow. We should never count AnyType(TypeOfAny.special_form) as a real Any (and we should already do this, see HasAnyType visitor in checkexpr.py). I guess what happens is that during inference we get a new Any, and it may be the new Any gets e.g. kind TypeOfAny.from_another_any. So that visitor should be fixed to also exclude TypeOfAny.from_another_any, where source_any.type_of_any == TypeOfAny.special_form (no need to recurse btw since constructor already handles this).

Btw I think is_special_form_any() in stats.py may also need updating, but that is more tricky, because for stats we sometimes do count special form Any as a "real" Any.

[DetachHead]

here's a more minimal example:

def foo() -> None:
    _ = 1
    assert _ == 1 # error

[ilevkivskyi]

Hm, unfortunately my idea uncovered a big underlying issue, see #15497 (comment)