Stubs not found with Guix Package Manager

[falloutphil]

Feature

Similar to #10633, since the changes for modular typeshed, mypy doesn't seem to pick-up type packages explicitly installed with the GNU Guix package manager (https://guix.gnu.org).

Case in point I can install types-requests using Guix - for reference I used the following package definiton (derivied straight from guix import pypi types-requests):

(define-public python-types-requests
  (package
    (name "python-types-requests")
    (version "2.25.11")
    (source
     (origin
       (method url-fetch)
       (uri (pypi-uri "types-requests" version))
       (sha256
        (base32
         "17msk3yhxjh82m5bx8625m9ry249szj6b5idw67f6s7na572hydj"))))
    (build-system python-build-system)
    (home-page "https://github.com/python/typeshed")
    (synopsis "Typing stubs for requests")
    (description "Typing stubs for requests")
    (license #f)))

I can then install that in an environment with mypy 0910:

guix environment --ad-hoc python-mypy python-types-requests

If I look at the site-packages structure I can see both modules installed just like in the case when using a venv and pip install. A requests-stubs directory is present:

$ echo $PYTHONPATH
/gnu/store/kycvw5fpj4mcz378ka0qvfriwcwpcy1b-profile/lib/python3.8/site-packages
$ cd $PYTHONPATH
$ ll
total 0
lrwxrwxrwx 72 root root 100 Jan  1  1970 easy_install.py -> /gnu/store/hc2nql01h78qqxlcg4qril9c314m33zg-python-3.8.2/lib/python3.8/site-packages/easy_install.py
lrwxrwxrwx  7 root root  94 Jan  1  1970 mypy -> /gnu/store/kpv69xmyaab5wql7rn2pqq4666zzhpvc-python-mypy-0.910/lib/python3.8/site-packages/mypy
lrwxrwxrwx  7 root root 115 Jan  1  1970 mypy-0.910-py3.8.egg-info -> /gnu/store/kpv69xmyaab5wql7rn2pqq4666zzhpvc-python-mypy-0.910/lib/python3.8/site-packages/mypy-0.910-py3.8.egg-info
lrwxrwxrwx  7 root root  95 Jan  1  1970 mypyc -> /gnu/store/kpv69xmyaab5wql7rn2pqq4666zzhpvc-python-mypy-0.910/lib/python3.8/site-packages/mypyc
lrwxrwxrwx 14 root root 137 Jan  1  1970 mypy_extensions-0.4.3-py3.8.egg-info -> /gnu/store/slhjav54195kbavimdvvsghwv6993ipb-python-mypy-extensions-0.4.3/lib/python3.8/site-packages/mypy_extensions-0.4.3-py3.8.egg-info
lrwxrwxrwx 14 root root 119 Jan  1  1970 mypy_extensions.py -> /gnu/store/slhjav54195kbavimdvvsghwv6993ipb-python-mypy-extensions-0.4.3/lib/python3.8/site-packages/mypy_extensions.py
lrwxrwxrwx 72 root root  88 Jan  1  1970 pip -> /gnu/store/hc2nql01h78qqxlcg4qril9c314m33zg-python-3.8.2/lib/python3.8/site-packages/pip
lrwxrwxrwx 72 root root 105 Jan  1  1970 pip-19.2.3.dist-info -> /gnu/store/hc2nql01h78qqxlcg4qril9c314m33zg-python-3.8.2/lib/python3.8/site-packages/pip-19.2.3.dist-info
lrwxrwxrwx 51 root root  98 Jan  1  1970 pkg_resources -> /gnu/store/hc2nql01h78qqxlcg4qril9c314m33zg-python-3.8.2/lib/python3.8/site-packages/pkg_resources
dr-xr-xr-x  2 root root 201 Jan  1  1970 __pycache__
lrwxrwxrwx 72 root root  95 Jan  1  1970 README.txt -> /gnu/store/hc2nql01h78qqxlcg4qril9c314m33zg-python-3.8.2/lib/python3.8/site-packages/README.txt
lrwxrwxrwx  4 root root 116 Jan  1  1970 requests-stubs -> /gnu/store/lgn23rxb63sl54i4px4lrrns3mv40jnn-python-types-requests-2.25.11/lib/python3.8/site-packages/requests-stubs
lrwxrwxrwx 51 root root  95 Jan  1  1970 setuptools -> /gnu/store/hc2nql01h78qqxlcg4qril9c314m33zg-python-3.8.2/lib/python3.8/site-packages/setuptools
lrwxrwxrwx 72 root root 112 Jan  1  1970 setuptools-41.2.0.dist-info -> /gnu/store/hc2nql01h78qqxlcg4qril9c314m33zg-python-3.8.2/lib/python3.8/site-packages/setuptools-41.2.0.dist-info
lrwxrwxrwx 27 root root  95 Jan  1  1970 toml -> /gnu/store/caia95dd5c2bm9qigp2am4fx9xfkfhpz-python-toml-0.10.2/lib/python3.8/site-packages/toml
lrwxrwxrwx 27 root root 117 Jan  1  1970 toml-0.10.2-py3.8.egg-info -> /gnu/store/caia95dd5c2bm9qigp2am4fx9xfkfhpz-python-toml-0.10.2/lib/python3.8/site-packages/toml-0.10.2-py3.8.egg-info
lrwxrwxrwx 14 root root 104 Jan  1  1970 typed_ast -> /gnu/store/ji2xnngipynyarqkpv6s96043x2r0vvp-python-typed-ast-1.4.0/lib/python3.8/site-packages/typed_ast
lrwxrwxrwx 14 root root 125 Jan  1  1970 typed_ast-1.4.0-py3.8.egg-info -> /gnu/store/ji2xnngipynyarqkpv6s96043x2r0vvp-python-typed-ast-1.4.0/lib/python3.8/site-packages/typed_ast-1.4.0-py3.8.egg-info
lrwxrwxrwx  4 root root 139 Jan  1  1970 types_requests-2.25.11-py3.8.egg-info -> /gnu/store/lgn23rxb63sl54i4px4lrrns3mv40jnn-python-types-requests-2.25.11/lib/python3.8/site-packages/types_requests-2.25.11-py3.8.egg-info
lrwxrwxrwx 31 root root 145 Jan  1  1970 typing_extensions-3.7.4.3-py3.8.egg-info -> /gnu/store/6xk9kxwzajh1q0l7q0c5lx4554r6203r-python-typing-extensions-3.7.4.3/lib/python3.8/site-packages/typing_extensions-3.7.4.3-py3.8.egg-info
lrwxrwxrwx 31 root root 125 Jan  1  1970 typing_extensions.py -> /gnu/store/6xk9kxwzajh1q0l7q0c5lx4554r6203r-python-typing-extensions-3.7.4.3/lib/python3.8/site-packages/typing_extensions.py

Everything looks the same as the venv equivalent apart from the obvious:

• Guix softlinks the actual packages
• Guix sets the mtime to 01/01/70 for every file

Looking at the verbose output I can see that stubs under here are found:
/gnu/store/kpv69xmyaab5wql7rn2pqq4666zzhpvc-python-mypy-0.910/lib/python3.8/site-packages/mypy/typeshed/stdlib/

But it ignores the stubs softlinked under:
requests-stubs -> /gnu/store/lgn23rxb63sl54i4px4lrrns3mv40jnn-python-types-requests-2.25.11/lib/python3.8/site-packages/requests-stubs

End of the log looks like this:

LOG:  Metadata fresh for posixpath: file /gnu/store/kpv69xmyaab5wql7rn2pqq4666zzhpvc-python-mypy-0.910/lib/python3.8/site-packages/mypy/typeshed/stdlib/posixpath.pyi
LOG:  Metadata fresh for genericpath: file /gnu/store/kpv69xmyaab5wql7rn2pqq4666zzhpvc-python-mypy-0.910/lib/python3.8/site-packages/mypy/typeshed/stdlib/genericpath.pyi
LOG:  Loaded graph with 47 nodes (0.054 sec)
LOG:  Found 24 SCCs; largest has 24 nodes
LOG:  Processing 23 queued fresh SCCs
LOG:  Processing SCC singleton (src.dss.dss_api) as inherently stale
src/dss/dss_api.py:4: error: Library stubs not installed for "requests" (or incompatible with Python 3.6)
    import requests
    ^
src/dss/dss_api.py:4: note: Hint: "python3 -m pip install types-requests"
src/dss/dss_api.py:4: note: (or run "mypy --install-types" to install all missing stub packages)
src/dss/dss_api.py:4: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
src/dss/dss_api.py:11: error: Library stubs not installed for "requests.adapters" (or incompatible with Python 3.6)

Pitch

Identify what is making the current implementation ignore walking the softlinked stub directories under Guix and include them.

In the meantime offer any workaround we could employ?

For example I've tried exporting MYPYPATH to the absolute directory targeted by the requests-stubs softlink as according to the docs this should then include that directory in the search path for stubs - but it doesn't seem to work.

One final option would be to allow us to include all stubs at build-time in the mypy package via a switch - then we could just build the old-style package?

[emmatyping]

I've never heard of Guix, but it seems like Nix. I know that the Nix folks have figured out how to run mypy with the symlinked environment like you have, so I might suggest reporting this to Guix and seeing what Nix does.