Skip to content

Invalidate cache when previously missing stubs are added #5465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 13, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion mypy/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -1884,6 +1884,25 @@ def __init__(self,
self.child_modules = set(self.meta.child_modules)
if temporary:
self.load_tree(temporary=True)
if not manager.use_fine_grained_cache():
# Special case: if there were a previously missing package imported here
# and it is not present, then we need to re-calculate dependencies.
# This is to support patterns like this:
# from missing_package import missing_module # type: ignore
# At first mypy doesn't know that `missing_module` is a module
# (it may be a variable, a class, or a function), so it is not added to
# suppressed dependencies. Therefore, when the package with module is added,
# we need to re-calculate dependencies.
# NOTE: see comment below for why we skip this in fine grained mode.
new_packages = False
for dep in self.suppressed:
path = manager.find_module_cache.find_module(dep, manager.search_paths,
manager.options.python_executable)
if path and '__init__.py' in path:
new_packages = True
if new_packages:
self.parse_file() # This is safe because the cache is anyway stale.
self.compute_dependencies()
else:
# When doing a fine-grained cache load, pretend we only
# know about modules that have cache information and defer
Expand Down Expand Up @@ -2696,12 +2715,15 @@ def load_graph(sources: List[BuildSource], manager: BuildManager,
# (since direct dependencies reflect the imports found in the source)
# but A's cached *indirect* dependency on C is wrong.
dependencies = [dep for dep in st.dependencies if st.priorities.get(dep) != PRI_INDIRECT]
added = [dep for dep in st.suppressed
if manager.find_module_cache.find_module(dep, manager.search_paths,
manager.options.python_executable)]
for dep in st.ancestors + dependencies + st.suppressed:
# We don't want to recheck imports marked with '# type: ignore'
# so we ignore any suppressed module not explicitly re-included
# from the command line.
ignored = dep in st.suppressed and dep not in entry_points
if ignored:
if ignored and dep not in added:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought there was already logic to pick up added modules? Does it not work for these purposes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that the problem is that freshness of a file is determined by whether the suppressed deps are present in the graph. But they are never added to the graph in the first place because of this code (unless they are added as entry points). So I only add them (them == previously suppressed files that are now found) to the graph here in load_graph, and the invalidation is done by the code in process_graph.

manager.missing_modules.add(dep)
elif dep not in graph:
try:
Expand Down
94 changes: 94 additions & 0 deletions test-data/unit/check-incremental.test
Original file line number Diff line number Diff line change
Expand Up @@ -4790,3 +4790,97 @@ tmp/c.py:1: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-impor
[out2]
tmp/c.py:1: error: Cannot find module named 'a.b.c'
tmp/c.py:1: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)

[case testAddedMissingStubs]
# flags: --ignore-missing-imports
from missing import f
f(int())
[file missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:3: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsPackage]
# flags: --ignore-missing-imports
import package.missing
package.missing.f(int())
[file package/__init__.pyi.2]
[file package/missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:3: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsPackageFrom]
# flags: --ignore-missing-imports
from package import missing
missing.f(int())
[file package/__init__.pyi.2]
[file package/missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:3: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsPackagePartial]
# flags: --ignore-missing-imports
import package.missing
package.missing.f(int())
[file package/__init__.pyi]
[file package/missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:3: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsPackagePartialGetAttr]
import package.missing
package.missing.f(int())
[file package/__init__.pyi]
from typing import Any
def __getattr__(attr: str) -> Any: ...
[file package/missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:2: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsIgnore]
from missing import f # type: ignore
f(int())
[file missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:2: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsIgnorePackage]
import package.missing # type: ignore
package.missing.f(int())
[file package/__init__.pyi.2]
[file package/missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:2: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsIgnorePackageFrom]
from package import missing # type: ignore
missing.f(int())
[file package/__init__.pyi.2]
[file package/missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:2: error: Argument 1 to "f" has incompatible type "int"; expected "str"

[case testAddedMissingStubsIgnorePackagePartial]
import package.missing # type: ignore
package.missing.f(int())
[file package/__init__.pyi]
[file package/missing.pyi.2]
def f(x: str) -> None: pass
[out]
[out2]
main:2: error: Argument 1 to "f" has incompatible type "int"; expected "str"