Skip to content

Invalid exported type for lambda arg to overloaded function #12773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JukkaL opened this issue May 12, 2022 · 0 comments
Open

Invalid exported type for lambda arg to overloaded function #12773

JukkaL opened this issue May 12, 2022 · 0 comments
Assignees
Labels
bug mypy got something wrong

Comments

@JukkaL
Copy link
Collaborator

JukkaL commented May 12, 2022

This typexport-basic.test test case fails, as an incorrect type is stored for the lambda and the final NameExpr:

[case testExportOverloadArgType]
## LambdaExpr|NameExpr
from typing import List, overload, Callable
@overload
def f(x: int, f: Callable[[int], int]) -> None: ...
@overload
def f(x: str, f: Callable[[str], str]) -> None: ...
def f(x): ...
f(
    1, lambda x: x)
[builtins fixtures/list.pyi]
[out]
NameExpr(8) : Overload(def (x: builtins.int, f: def (builtins.int) -> builtins.int), def (x: builtins.str, f: def (builtins.str) -> builtins.str))
LambdaExpr(9) : def (builtins.int) -> builtins.int
NameExpr(9) : builtins.int

The invalid exported type for the lambda is (str) -> str. The NameExpr has type str, instead of int.

We attempt to infer a type for the lambda once for each overload item. (The type context is taken from the overload item.) The type based on the final overload item is the one that takes precedence, since it gets evaluated last, even if it's not the matching item.

This can cause mypyc to generate incorrect code (see test failures in #12766), since it assumes that the exported types are correct and checks runtime values against them.

I see a few plausible fixes:

  1. Perform another type checking pass using the matching overload item as context. This has the drawback of being potentially quite expensive.
  2. When type checking as part of a search for a matching overload item, store inferred types in a temporary location. Discard the inferred types from each unsuccessful search attempt, and only record the types from the successful attempt.

Option 2 sounds more appealing to me, but it sounds a bit more difficult to implement.

@JukkaL JukkaL added the bug mypy got something wrong label May 12, 2022
@JukkaL JukkaL self-assigned this May 12, 2022
JukkaL added a commit that referenced this issue May 13, 2022
Previously we could export an invalid type for a lambda passed to an
overloaded function. We find the matching overload variant by type checking
the arguments against all overload items. We exported types for lambdas
always from the final potential overload item, even if that wasn't the matching
one. This could result in mypyc adding invalid type coercions that could result in
bogus TypeErrors.

The fix is to store types inferred when looking for matching overload items
into temporary type maps, and only the type map from the matching item
gets merged into the exported types.

This doesn't fully solve the problem -- if there are Any types in the
arguments, we can still export invalid types. This should be enough to
unblock syncing typeshed (#12766). Typeshed started triggering the issue
in compiled mypy when re.sub was changed to an overloaded function.

Work on #12773.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong
Projects
None yet
Development

No branches or pull requests

1 participant