Skip to content

Prioritize .pyi from -stubs packages over bundled .pyi #19001

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 1, 2025
Merged

Prioritize .pyi from -stubs packages over bundled .pyi #19001

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
48fa12a
reproduce #18997
jorenham Apr 29, 2025
e85acf6
prioritize stubs from `-stubs` packages over bundled stubs
jorenham Apr 29, 2025
561fb7e
validate PEP 561 precendence for `.pyi` modules
jorenham Apr 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions mypy/modulefinder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -506,21 +506,24 @@ def _find_module(self, id: str, use_typeshed: bool) -> ModuleSearchResult:
dir_prefix = base_dir
for _ in range(len(components) - 1):
dir_prefix = os.path.dirname(dir_prefix)

# Stubs-only packages always take precedence over py.typed packages
path_stubs = f"{base_path}-stubs{sepinit}.pyi"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm this assumes there is an __init__.pyi, I don't think that's required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty much the same as the path_stubs = base_path + "-stubs" + sepinit + extension from before before 🤷🏻

Should I change it to something else?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea you're right, apparently this isn't the whole story:
I added some new tests for a.py[i], but those are failing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm somehow the tests only fail if it's called a.pyi, and not when called spam.pyi. I'm guessing it's a caching thing. Either way, it's probably out out scope, and might be limited to these unit-tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm this assumes there is an __init__.pyi, I don't think that's required?

See the updated PR description

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I believe namespace packages are handled on L525 afterwards.

if fscache.isfile_case(path_stubs, dir_prefix):
if verify and not verify_module(fscache, id, path_stubs, dir_prefix):
near_misses.append((path_stubs, dir_prefix))
else:
return path_stubs

# Prefer package over module, i.e. baz/__init__.py* over baz.py*.
for extension in PYTHON_EXTENSIONS:
path = base_path + sepinit + extension
path_stubs = base_path + "-stubs" + sepinit + extension
if fscache.isfile_case(path, dir_prefix):
has_init = True
if verify and not verify_module(fscache, id, path, dir_prefix):
near_misses.append((path, dir_prefix))
continue
return path
elif fscache.isfile_case(path_stubs, dir_prefix):
if verify and not verify_module(fscache, id, path_stubs, dir_prefix):
near_misses.append((path_stubs, dir_prefix))
continue
return path_stubs

# In namespace mode, register a potential namespace package
if self.options and self.options.namespace_packages:
Expand Down
6 changes: 6 additions & 0 deletions mypy/test/testmodulefinder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,9 @@ def test__packages_with_ns(self) -> None:
("pkg_typed.b", self.path("pkg_typed", "b", "__init__.py")),
("pkg_typed.b.c", self.path("pkg_typed", "b", "c.py")),
("pkg_typed.a.a_var", ModuleNotFoundReason.NOT_FOUND),
# Regular package with py.typed, bundled stubs, and external stubs-only package
("pkg_typed_w_stubs", self.path("pkg_typed_w_stubs-stubs", "__init__.pyi")),
("pkg_typed_w_stubs.spam", self.path("pkg_typed_w_stubs-stubs", "spam.pyi")),
# Regular package without py.typed
("pkg_untyped", ModuleNotFoundReason.FOUND_WITHOUT_TYPE_HINTS),
("pkg_untyped.a", ModuleNotFoundReason.FOUND_WITHOUT_TYPE_HINTS),
Expand Down Expand Up @@ -250,6 +253,9 @@ def test__packages_without_ns(self) -> None:
("pkg_typed.b", self.path("pkg_typed", "b", "__init__.py")),
("pkg_typed.b.c", self.path("pkg_typed", "b", "c.py")),
("pkg_typed.a.a_var", ModuleNotFoundReason.NOT_FOUND),
# Regular package with py.typed, bundled stubs, and external stubs-only package
("pkg_typed_w_stubs", self.path("pkg_typed_w_stubs-stubs", "__init__.pyi")),
("pkg_typed_w_stubs.spam", self.path("pkg_typed_w_stubs-stubs", "spam.pyi")),
# Regular package without py.typed
("pkg_untyped", ModuleNotFoundReason.FOUND_WITHOUT_TYPE_HINTS),
("pkg_untyped.a", ModuleNotFoundReason.FOUND_WITHOUT_TYPE_HINTS),
Expand Down
1 change: 1 addition & 0 deletions test-data/packages/modulefinder-site-packages/foo-stubs/qux.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
qux_var: int
1 change: 1 addition & 0 deletions test-data/packages/modulefinder-site-packages/pkg_typed_w_stubs-stubs/__init__.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pkg_typed_w_stubs_var: str = ...
1 change: 1 addition & 0 deletions test-data/packages/modulefinder-site-packages/pkg_typed_w_stubs-stubs/spam.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
spam_var: str
1 change: 1 addition & 0 deletions test-data/packages/modulefinder-site-packages/pkg_typed_w_stubs/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pkg_typed_w_stubs_var = "pkg_typed_w_stubs"
1 change: 1 addition & 0 deletions test-data/packages/modulefinder-site-packages/pkg_typed_w_stubs/__init__.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pkg_typed_w_stubs_var: object
Empty file added test-data/packages/modulefinder-site-packages/pkg_typed_w_stubs/py.typed
View file
Open in desktop
Empty file.
1 change: 1 addition & 0 deletions test-data/packages/modulefinder-site-packages/pkg_typed_w_stubs/spam.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
spam_var = "spam"
1 change: 1 addition & 0 deletions test-data/packages/modulefinder-site-packages/pkg_typed_w_stubs/spam.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
spam_var: object