"Invalid Type" error in v0.600 on uncached runs involving file with same name as parent package

[Strilanc]

When I run mypy on my project, I sometimes get an invalid type error. This error seems to go away over time (some kind of caching issue?), or if I avoid having a file in a package of the same name, or if I do an explicit direct import of the type from the file instead of importing it via the package.

This is kind of a complicated bug to reproduce, but here is my best attempt at making a succinct repro. We will need five files.

0. Create a fresh virtual environment and test project directory.

   mkvirtualenv repro --python=/usr/bin/python3
   pip install mypy
   mkdir /tmp/whatever
   cd /tmp/whatever
   mkdir project

1. Have a project/__init__.py file containing

   from project.study import CustomType
   

2. Have an empty project/root.py file.

3. Have a project/neighbor/__init__.py file containing

   from project.study import CustomType
   def m(arg: CustomType) -> str:
       del arg
       return 'test'

4. Have a project/study/__init__.py file containing

   from project.study.study import CustomType

5. Have a project/study/study.py file containing

   from project import root
   CustomType = str

   Note: renaming this file to something that doesn't match its directory's name will cause the repro to fail.

6. (If you already ran mypy in this directory.) Clear the mypy cache.

   rm -rf .mypy_cache

7. Run mypy on exactly these files in exactly this order.

   mypy project/root.py project/study/study.py project/neighbor/__init__.py

8. And you will get this output:

   project/neighbor/__init__.py:4: error: Invalid type "project.study.study.CustomType"
   

   If you run mypy again without clearing its cache, then there will be no error.

   or, depending on factors I don't understand you may get this different error instead:

   project/neighbor/__init__.py:2: error: Internal error (node is None, kind=1)
   

An easy workaround for this issue is to avoid defining types in files that have the same name as their parent directory. Everyone likes avoiding cache invalidation issues by naming things, right?

[JelleZijlstra]

Thanks for the detailed reproduction instructions!

[gvanrossum]

Hm, I wonder what the cache contents are for those five files?

[msullivan]

I was able to get this error to appear (including follow up internal errors) even if I renamed the submodule?

[msullivan]

Bisecting shows that this is a regression from 0.590 introduced by #4910. I'll take a look.

[thomascellerier]

This introduced many errors when switching from 0.590 to 0.600 on some proprietary code base at $DAYJOB. I verified that the fix from the import-cycles branch fixes all these issues #5016

The situation that we have is the following (I couldn't reproduce a minimal example). A being a module and a a submodule:

• A
  • a
  • b
• B
  • a
  • b

A.a import B.b and B.b imports A.a. So there is no cycle at the submodule level.

[JelleZijlstra]

Is your issue the same as #5024?

[thomascellerier]

It could be, the package has many sub modules and sub-sub modules depending on each other (needs refactoring :)), though no direct cycles. I can't produce a repro unfortunately.
However this PR fixed it all #5016