Crash in version 1.11 lambda processing (regression since version 1.10)

[acamatcisco]

Crash Report

We maintain a large (1M+ LoC), proprietary Python code base, currently using mypy version 1.10. During an upgrade to version 1.11, I noticed some INTERNAL ERROR output so decided to investigate.

Traceback

bug.py:11: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.11.0
Traceback (most recent call last):
  File "mypy/checkexpr.py", line 5830, in accept
  File "mypy/nodes.py", line 2278, in accept
  File "mypy/checkexpr.py", line 5294, in visit_lambda_expr
  File "/opt/homebrew/Cellar/[email protected]/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "mypy/binder.py", line 436, in frame_context
AssertionError:

To Reproduce

Since I am unable to provide our full source code, I created the following small stand-alone example:

def foo(x):
    pass


class Bar:
    def baz(self, x):
        pass


class Qux(Bar):
    @foo(lambda x: None)
    def baz(self, x) -> None:
        pass

The crash seems to occur when a method decorator defined with a lambda overrides another method of the same name in a base class and there's an annotation mismatch.

That is, for the above example, the crash can be avoided by adding a return value annotation to baz() in the base class:

 class Bar:
-    def baz(self, x):
+    def baz(self, x) -> None:
         pass

Investigation

I narrowed the regression down to the following commit (the crash is avoided if I revert this change):

5059ffd

The crash can also be avoided if I adjust the assertion at the crash site:

--- binder.py.bak	2024-07-26 10:54:02
+++ binder.py	2024-07-26 10:54:07
@@ -433,7 +433,7 @@
         whether any types changed in the newly-topmost frame as a result
         of popping this frame.
         """
-        assert len(self.frames) > 1
+        assert len(self.frames) > 0

         if break_frame:
             self.break_frames.append(len(self.frames) - break_frame)

pytest -q mypy passes after this change, and a full run of mypy 1.11 on our large code base also succeeds without any tracebacks or obvious ill-effects, but I'm otherwise not familiar enough with the innards of mypy to have confidence in its correctness (hence I felt more comfortable providing the FYI here instead of submitting a PR).

Your Environment

• Mypy version used: 1.11
• Mypy command-line flags: None (defaults)
• Mypy configuration options from mypy.ini (and other config files): Reproduced with defaults
• Python version used: 3.11.9
• Operating system and version: macOS Sonoma 14.5

[andersk]

This variant test case, using a generator instead of a lambda, crashes even before 5059ffd, going back all the way to b41bb66 (#5637, v0.700~405).

def foo(x):
    pass


class Bar:
    def baz(self, x):
        pass


class Qux(Bar):
    @next(f for f in [foo])
    def baz(self, x) -> None:
        pass

Playground

[acamatcisco]

Oh wow! Interesting it went unnoticed for so long. We've been slowly adding typing annotations for years and still don't enforce them, so it's not too surprising in our case that we triggered this bug with a mixture of typed and untyped defs.