ignore-missing-imports does not suppress missing imports from existing modules

[carljm]

Filed as follow-up to python/typeshed#1827.

Given the following tree structure:

.
├── real
│   ├── __init__.py
│   └── sub.py
├── stub
│   ├── __init__.pyi
│   └── sub.pyi
└── test.py

with all files empty except for test.py, which contains:

from stub import noexist1
from stub.sub import noexist2
from stub.noexist3 import noexist4

from real import noexist5
from real.sub import noexist6
from real.noexist7 import noexist8

from noexist9 import bar
from noexist10 import baz

Running mypy --ignore-missing-imports test.py results in:

test.py:1: error: Module 'stub' has no attribute 'noexist1'
test.py:2: error: Module 'stub.sub' has no attribute 'noexist2'
test.py:5: error: Module 'real' has no attribute 'noexist5'
test.py:6: error: Module 'real.sub' has no attribute 'noexist6'

That is, missing module errors are suppressed, but imports of missing attributes of existing modules are not suppressed.

This makes incomplete stubs more troublesome than they would otherwise be, because you can't suppress them globally with something like this in mypy.ini:

[mypy-stub.*]
ignore_missing_imports = True

There is no option but to either extend the incomplete stub, or else apply individual # type: ignore to every single erroring import.

It seems to me that the name ignore-missing-imports implies that all errors due to importing something that doesn't exist should be suppressed; it shouldn't matter whether the thing that is missing is an entire module or an attribute of a module. So I think this is a bug.

All behaviors above checked on current mypy master, Python 3.6.

[gvanrossum]

The problem here is how would you distinguish between an attribute that's somehow missing from the stub and a typo in an attribute name? And technically it's not a missing import (which really refers to the situation where a module is not found) but a missing attribute. I agree it's a pain though, and any suggestion you have on how to suppress such errors with the most precision would be appreciated!

[carljm]

How is the "it could be a typo" argument different for module attributes than for module names themselves? Losing this typo protection is just a built-in cost of choosing to apply ignore_missing_imports; fortunately the per-module syntax in mypy.ini already allows it to be applied selectively.

Not sure about the semantic technicalities, I think a lot of people would call a missing attribute error specifically in the context of a from ... import ... to be an "import error". At runtime this raises ImportError, not AttributeError.

I'm not proposing that "module has no attribute X" errors in any other context should be suppressed. I guess that is slightly weird (and still means some # type: ignore could be required for an incomplete stub), but "suppress type errors that would be ImportError at runtime" seems to me as reasonable as any other definition for ignore-missing-imports. FWIW in our case with the thrift stubs, all of the errors were in from thrift.something import context. But this would depend on the preferred import style for a project.

One alternative that could suppress both from ... import ... ImportError and later module AttributeError: have a way to say "imports from this module are considered to be Any" that works even if the module exists or has stubs. Effectively it'd be like follow-imports=skip but when applied per-module in mypy.ini it would match to the imported module, not the module doing the importing. Not sure how to name such an option in a way that isn't totally confusing, though.

[carljm]

One option would be to spell it ignore_stubs = True and have it work only for stubs. It's harder to imagine a good use case for this with real code modules.

[gvanrossum]

I guess we look at this from different angles. I am looking at it from the POV of how the import is processed -- if the stub is found in my view it's no longer import's problem. (Even though it's spelled using import, internally in Python it's just an attribute access.)

But I'm fine with an option to change that default. It would be a nice touch if it only applied to stubs since then you can set it globally, but you should also be able to set it for a specific package or module, and then it should apply to all imports from that module/package (not all imports in that package).

[msullivan]

follow_imports does apply to the imported module, not the module doing the importing, but by default (and until recently always) is ignored for stub files. Now you can do

[mypy-some_stub.*]
follow_imports=skip
follow_imports_for_stubs=True

and it will suppress the import of the stub and assume any for all its contents.

I'm closing this issue because I think that is a satisfactory solution here, but if somebody disagrees feel free to reopen.