Make Any a proper class instead of an alias to object()

[grievejia]

At runtime, typing.Any is an actual class, not an object. Currently typeshed says Any = object() which seems wrong given it's not valid Python to use object() as type annotations, e.g.

def test0() -> object(): ...  # ERROR! `object()` is not a valid type

MyAny = object()
def test1() -> MyAny:  # ERROR! `MyAny` is not a valid type

from typing import Any
def test2() -> Any: ...  # This should be OK, but it's not if we strictly follow the `Any = object()` alias

I'm guessing that existing type checkers probably had some special cases for typing.Any to make it work. But if we define it right we could probably get rid of the special cases.

We ran into this issue when working on Pyre2.

[JelleZijlstra]

This is technically more accurate but as you can see it interacts poorly with the existing special casing in some type checkers. Any is a type system primitive and needs to be special-cased in type checkers anyway; it might make more practical sense to stick with the existing definition.

[hauntsaninja]

From a mypy perspective, based on primer this is actually probably an improvement (at least on 3.11+, and maybe depending on what you think TypeForm semantics should be).

pyright seems pretty unhappy though.

[srittau]

Stubtest errors are unrelated.

[grievejia]

Pyright has been updated and now it's OK with the change.

[srittau]

Seems good to me.

[AlexWaygood]

For red-knot, I'd honestly prefer for us to be forced to special-case Any explicitly wherever we need to. While it is technically a class at runtime, it behaves differently from every other class in the type system from a type checker's perspective. I'm worried that if it becomes a class in the stubs, it will be easy for us to forget to manually add in some special casing for it somewhere in red-knot where we need to.

We've already implemented the necessary special-casing in red-knot that allows users to inherit from Any even though it's an instance of object in the stubs, so I don't think this change would offer any advantages to red-knot.

[AlexWaygood]

(Disclaimer: I'm obviously just one developer working on red-knot; I can't claim to speak for everybody on the red-knot team!)

[srittau]

I prefer annotations to follow implementation if possible. Since this needs to be special-cased by type checkers anyway, this shouldn't be a problem for them. But nowadays type annotations are not only used by type checkers.

[AlexWaygood]

At runtime, typing.Any is an actual class, not an object.

Note that this is only true on Python 3.11+, but typeshed supports Python 3.8+. It was changed in python/cpython@5a4973e.

I'm still not enthusiastic about this PR, as I don't see it as bringing significant practical benefits to users. But having discussed it with other people on the red-knot team, I no longer want to block it. We'll be able to work around it without too much difficulty in red-knot, and it's true that it is more accurate to the runtime implementation on Python 3.11+. I'm happy for other maintainers to land it.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

sphinx (https://github.com/sphinx-doc/sphinx)
+ sphinx/config.py:631: error: Unused "type: ignore" comment  [unused-ignore]

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/operations/__init__.py:13: error: Incompatible import of "Any" (imported name has type "type[ibis.expr.operations.reductions.Any]", local name has type "<typing special form>")  [assignment]
+ ibis/expr/operations/__init__.py:13: error: Incompatible import of "Any" (imported name has type "type[ibis.expr.operations.reductions.Any]", local name has type "type[typing.Any]")  [assignment]
- ibis/expr/types/logical.py:333: error: "<typing special form>" not callable  [operator]
+ ibis/expr/types/logical.py:333: error: Too many arguments for "Any"  [call-arg]
+ ibis/expr/types/logical.py:333: error: Unexpected keyword argument "where" for "Any"  [call-arg]
+ note: "Any" defined here
+ ibis/expr/types/logical.py:340: error: "Any" has no attribute "to_expr"  [attr-defined]
- ibis/backends/polars/compiler.py:745: error: No overload variant of "register" of "_SingleDispatchCallable" matches argument types "object", "Callable[[Any, KwArg(Any)], Any]"  [call-overload]
- ibis/backends/polars/compiler.py:745: note: Possible overload variants:
- ibis/backends/polars/compiler.py:745: note:     def register(self, cls: type[Any] | UnionType, func: None = ...) -> Callable[[Callable[..., Any]], Callable[..., Any]]
- ibis/backends/polars/compiler.py:745: note:     def register(self, cls: Callable[..., Any], func: None = ...) -> Callable[..., Any]
- ibis/backends/polars/compiler.py:745: note:     def register(self, cls: type[Any] | UnionType, func: Callable[..., Any]) -> Callable[..., Any]

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/server/events/actions.py:101: error: Unsupported left operand type for | ("<typing special form>")  [operator]
+ src/prefect/server/events/actions.py:101: error: Unsupported left operand type for | ("type[Any]")  [operator]

artigraph (https://github.com/artigraph/artigraph)
- src/arti/internal/type_hints.py:54: error: Non-overlapping identity check (left operand type: "type", right operand type: "<typing special form>")  [comparison-overlap]
- src/arti/internal/type_hints.py:55: error: Non-overlapping identity check (left operand type: "type", right operand type: "<typing special form>")  [comparison-overlap]

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/structured_configs/_implementations.py:3252: error: List comprehension has incompatible type List[tuple[str, Any | <typing special form>, Any | Field[Any]]]; expected List[tuple[str, type] | tuple[str, type, Any]]  [misc]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/ext/commands/converters.py:510: error: Incompatible types in assignment (expression has type "<typing special form>", variable has type "type[T]")  [assignment]
+ steam/ext/commands/converters.py:510: error: Incompatible types in assignment (expression has type "type[Any]", variable has type "type[T]")  [assignment]

mypy (https://github.com/python/mypy)
+ mypy/stubgenc.py:768: error: Unused "type: ignore" comment  [unused-ignore]

[rchen152]

I'm in favor of this change. It better reflects the runtime, which I think is a good goal to strive for in general (although it obviously needs to be balanced against other considerations).