Adding a classmethod to TypedDict with "# type: ignore" crashes mypy

[achimnol]

Crash Report / To Reproduce

I know that currently allowing adding class methods to TypedDict is not on the priority (#5653),
so I tried to use it anyway with # type: ignore.

from __future__ import annotations
from typing import Literal, Mapping, Optional, TypedDict
import trafaret as t  # https://github.com/Deepwalker/trafaret

__all__ = (
    'HardwareMetadata',
)

class HardwareMetadata(TypedDict):
    status: Literal["healthy", "degraded", "offline", "unavailable"]
    status_info: Optional[str]
    metadata: Mapping[str, str]

    @classmethod  # type: ignore
    def check(cls, value: Any) -> HardwareMetadata:
        try:
            return t.Dict({
                t.Key('status'): t.Enum("healthy", "degraded", "offline", "unavailable"),
                t.Key('status_info'): t.Null | t.String,
                t.Key('metadata'): t.Mapping(t.String, t.String),
            }).check(value)
        except t.DataError as e:
            raise ValueError("The give value does not conform with the target typed dict.", e.as_dict())

I'm referring this as HardwareMetadata.check(...) in another module.
My intention is to validate JSON-like semi-structured data at runtime and use it safely when passing around the codebase with static type check.
Then, running mypy began to crash.

I think it should not crash anyway even though it does not allow this pattern.

Traceback

/home/joongi/backend.ai-dev/common-2009/src/ai/backend/common/types.py: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.790
Traceback (most recent call last):
  File "/home/joongi/.pyenv/versions/3.8.5/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/joongi/.pyenv/versions/3.8.5/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "mypy/build.py", line 1922, in wrap_context
  File "mypy/semanal_main.py", line 360, in check_type_arguments
  File "mypy/nodes.py", line 295, in accept
  File "mypy/semanal_typeargs.py", line 39, in visit_mypy_file
  File "mypy/traverser.py", line 35, in visit_mypy_file
  File "mypy/nodes.py", line 940, in accept
  File "mypy/semanal_typeargs.py", line 50, in visit_class_def
  File "mypy/mixedtraverser.py", line 28, in visit_class_def
  File "mypy/traverser.py", line 71, in visit_class_def
  File "mypy/nodes.py", line 1005, in accept
  File "mypy/semanal_typeargs.py", line 54, in visit_block
  File "mypy/traverser.py", line 39, in visit_block
  File "mypy/nodes.py", line 767, in accept
  File "mypy/traverser.py", line 76, in visit_decorator
  File "mypy/nodes.py", line 677, in accept
  File "mypy/traverser.py", line 54, in visit_func_def
  File "mypy/semanal_typeargs.py", line 46, in visit_func
  File "mypy/mixedtraverser.py", line 23, in visit_func
  File "mypy/mixedtraverser.py", line 91, in visit_optional_type
  File "mypy/types.py", line 1098, in accept
  File "mypy/typetraverser.py", line 52, in visit_callable_type
  File "mypy/types.py", line 794, in accept
  File "mypy/semanal_typeargs.py", line 70, in visit_instance
AttributeError: attribute 'defn' of 'TypeInfo' undefined
/home/joongi/backend.ai-dev/common-2009/src/ai/backend/common/types.py: : note: use --pdb to drop into pdb

Your Environment

• Mypy version used: 0.790
• Mypy command-line flags: python -m mypy src/ai/backend tests --show-traceback
  • Note: ai.backend is the namespace pkg.
• Mypy configuration options from mypy.ini (and other config files): I have the following in setup.cfg:

[mypy]
ignore_missing_imports = true
namespace_packages = true

• Python version used: Python 3.8.5
• Operating system and version: Ubuntu 20.04

[achimnol]

Rewriting the type definitions like this:

class HardwareMetadata(TypedDict):
    status: Literal["healthy", "degraded", "offline", "unavailable"]
    status_info: Optional[str]
    metadata: Mapping[str, str]

def check(cls, value: Any) -> HardwareMetadata:
    try:
        return t.Dict({
            t.Key('status'): t.Enum("healthy", "degraded", "offline", "unavailable"),
            t.Key('status_info'): t.Null | t.String,
            t.Key('metadata'): t.Mapping(t.String, t.String),
        }).check(value)
    except t.DataError as e:
        raise ValueError("The give value does not conform with the target typed dict.", e.as_dict())

HardwareMetadata.check = classmethod(check)  # type: ignore

"fixes" the crash, but I need to put # type: ignore whenever I call HardwareMetadata.check().

[achimnol]

It seems that the current "correct" workaround is to just separate the check method as an independent function.
I understand that TypedDict is not a real type but more like a meta-type, which leads to the design decision not allowing method addition.
Would there be good alternative ways to combine runtime input validation and static typing for semi-structured data?

[achimnol]

For instance, maybe we could just extend mypy to recognize trafaret validators as static type definitions of corresponding Python types. Though, I don't know the details about mypy extensions, so I'm not sure that this would be a good approach for this issue.

[achimnol]

I ended up combining the original TypedDict definition and check_typed_dict() from typeguard for runtime input validation for now. It seems to validate my above case very well, including Literal choices.

[gaozhidf]

u can try about Self (ref: https://docs.python.org/3/library/typing.html#typing.Self)

from typing import Self

class MyClass:
    def method(self) -> Self:
        return self