Strange errors when trying to type-check asyncio

[gvanrossum]

When running mypy over the stand-alone asyncio package (https://github.com/python/asyncio) using mypy asyncio I get some unexpected errors about uses of AnyStr in the stub for struct.pyi. I can't repro this easily in a smaller context, and in fact it doesn't repro of I point mypy at just asyncio/windows_events.py either. @JukkaL, do you have any idea what might be going on here?

$ mypy asyncio
asyncio/__init__.py:11: error: No library stub file for standard library module 'selectors'
asyncio/__init__.py:11: note: (Stub files are from https://github.com/python/typeshed)
asyncio/__init__.py:18: error: Cannot find module named '_overlapped'
asyncio/__init__.py:18: note: (Perhaps setting MYPYPATH would help)
asyncio/windows_utils.py:10: error: Cannot find module named '_winapi'
asyncio/windows_utils.py:37: error: Name 'socketpair' already defined
asyncio/windows_events.py:3: error: Cannot find module named '_winapi'
asyncio/windows_events.py:7: note: In module imported here:
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi: note: In function "pack":
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi:9: error: Invalid type "typing.AnyStr"
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi: note: In function "pack_into":
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi:11: error: Invalid type "typing.AnyStr"
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi: note: In function "unpack":
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi:14: error: Invalid type "typing.AnyStr"
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi: note: In function "unpack_from":
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi:15: error: Invalid type "typing.AnyStr"
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi: note: In function "calcsize":
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi:17: error: Invalid type "typing.AnyStr"
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi: note: In function "__init__":
/Users/guido/v3/lib/mypy/typeshed/stdlib/3/struct.pyi:23: error: Invalid type "typing.AnyStr"
[the rest is not relevant to this issue]

[JukkaL]

Hmm.. this looks like it could be caused a potential bug we found when browsing mypy code (mypy/build.py) with @gnprice earlier this week. It's probably something related to cyclic module dependencies and the order in which mypy processes the different files. typing is part of a cycle with builtins, and it seems that this may cause the dependency resolution logic to break down and typing gets semantically analyzed later than things that import it. You can see this by settings debug = True in mypy/build.py and looking for lines starting with next in mypy output. It shows that struct.pyi gets semantically analyzed before typing.pyi which is not right.

This is still a guess, but fixing dependency handling for dependency cycles may fix this. I'll look into this in more detail. The involved code is unclear anyway so even if this isn't the cause of this bug, it's worth refactoring or commenting the relevant code.

[JukkaL]

I think I figured this out. typing.pyi imports asyncio.futures which results in a dependency cycle. We'll probably want to break this, such as by having the definition of Future live in typing.asyncio.

[JukkaL]

Anyway we probably won't want typing to import asyncio as it'll bring in a ton of code at runtime that'll slow down program startup for most programs.

[gvanrossum]

Eww, this was introduced by python/typeshed#30 . I'll look into a better way of doing this.

[gvanrossum]

(So, if you're interested, running mypy on asyncio gets a lot of errors due to missing stubs and mypy bugs (asyncio loves conditional definitions) but it also found two bugs: a missing import and a missing exception definition, both in asyncio/test_support.py. Interestingly, that module is a "backup" for functionality that's usually in the stdlib, so it clearly didn't get exercised as much...)