Skip to content

Switch to modular typeshed #9971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
JukkaL opened this issue Jan 26, 2021 · 2 comments
Closed

Switch to modular typeshed #9971

JukkaL opened this issue Jan 26, 2021 · 2 comments
Assignees
@JukkaL
Labels
feature priority-0-high

Comments

@JukkaL
Copy link
Collaborator

JukkaL commented Jan 26, 2021

The structure of typeshed is changing, and tools are no longer expected to bundle the entire typeshed. Instead, users will install relevant PEP 561 stub distributions for the packages that they need.

Typeshed will continue to host third-party package stubs, and updates to third-party stubs will automatically uploaded to PyPI.

This will affect mypy in several ways:

  • Mypy will only ship with stdlib stubs.
  • Stubs for most third-party packages must be installed using pip, e.g. pip install types-requests for stubs for requests.
  • Typeshed will no longer be a git submodule. Instead, we'll copy the stdlib stubs to the mypy repository.

This has some major benefits:

  • It's easy to start using new stubs, or a new version of existing stubs, contributed to typeshed. Users don't need to wait for the next mypy release -- they can just install or update the relevant stubs.
  • Users no longer have to update to the latest stubs when updating mypy. This can save a lot of friction, since new stubs may have errors, and updating to stubs with stricter types (even if they are precise) may require significant changes to code that uses the relevant features.
  • This makes things more consistent. Currently some third-party stubs are bundled with mypy, while some must be installed from PyPI. In the future all third-party stubs are installed in a similar way using pip.
  • It will be easier to maintain multiple versions of stubs (for example, separate stubs for versions of 1.x and 2.x of a library).

We can make the transition relatively smooth by suggesting how to install the relevant stubs when the stubs are missing. We can also support automatically installing the necessary stubs (mypy --install-stubs).

Discussion about the typeshed change, including motivation: python/typeshed#2491
@JukkaL JukkaL self-assigned this Jan 26, 2021
@JukkaL JukkaL mentioned this issue Jan 26, 2021
Merged
JukkaL added a commit that referenced this issue Jan 26, 2021
@JukkaL
Remove the git submodule at mypy/typeshed
ba03788 
I'll add mypy/typeshed back in another commit as a copy of a
typeshed subset.

This is preparations for #9971 and #9973.
@JukkaL
Copy link
Collaborator Author

JukkaL commented Jan 26, 2021

After removing the typeshed submodule, we'll use the script misc/sync-typeshed.py to sync typeshed. It just copies the stdlib/ directory from typeshed to the mypy git repository.

JukkaL added a commit that referenced this issue Jan 26, 2021
@JukkaL
Use modular typeshed (unbundle third-party package stubs) (#9973)
0e7b867 
Support the new structure of typeshed (python/typeshed#2491) and only
bundle stdlib stubs with mypy. 

Most stubs for third-party packages now need to be installed using pip (for 
example, `pip install types-requests`). Add stubs for `typing_extensions` 
and `mypy_extensions` as mypy dependencies, since these are 
needed for basic operation.

Suggest a pip command to run if we encounter known missing stubs.

Add `--install-types` option that installs all missing stub packages. This
can be used as `mypy --install-types` to install missing stub packages from
the previous mypy run (no need to provide a full mypy command line).

This also replaces the typeshed git submodule with a partial copy of the
typeshed that only includes stdlib stubs. 

Add a script to sync stubs from typeshed (`misc/sync-typeshed.py`). 
This is still incomplete since typeshed hasn't actually migrated to the new 
structure yet.

Work towards #9971.
@sobolevn sobolevn mentioned this issue Apr 13, 2021
Merged
@JukkaL
Copy link
Collaborator Author

JukkaL commented May 19, 2021

This has been working for some time now (though not publicly released yet).

@JukkaL JukkaL closed this as completed May 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JukkaL JukkaL

Labels
feature priority-0-high
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JukkaL