Unresolved import when opening a subdirectory of an import root as the workspace root

[luabud]

@Anti-Distinctlyminty commented on Mon May 06 2019

Environment data

• VS Code version: 1.33.1
• Extension version (available under the Extensions sidebar): 2019.4.12954
• OS and version: Windows_NT x64 10.0.17134
• Python version (& distribution if applicable, e.g. Anaconda): 2.7.11 64-bit

Expected behaviour

With python.autoComplete.extraPaths set correctly, modules should import without issue.

Actual behaviour

I receive a Unresolved import 'moduleName' when using syntax of the form
import workspace_folder.module as alt_name
and if the .code-workspace file is in the same directory.
e.g.

Steps to reproduce:

1. Extract the following to a suitable location:
   myLib.zip
2. Inside myLib, open the Import_Bug.code-workspace file
3. Go to workspace settings and change the python.autoComplete.extraPaths to the folder containing myLib.
4. open libFunctions.py and see that import myLib.importMe as im has an unresolved import error.

I am aware that the organization of this workspace is sub-optimal, with the code-workspace file being inside the python library folder. However, with all paths being set correctly, the import should still succeed.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Blank.

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

Blank.

[AlexanderSher]

There are two issues with this workspace:

• Workspace file itself contains path:

"folders": [
    {
        "path": "."
    }
]

which defines workspace root as workspace_folder. path has priority over python.autoComplete.extraPaths setting, so if extra path is parent folder to the path folder, it will be ignored for any module inside path. To fix the workspace, you can change path to ...

• myLib.zip in attachment contains modules and packages that don't match naming conventions: https://www.python.org/dev/peps/pep-0008/#package-and-module-names We still support these files, but they can't be referenced as imports.

[Anti-Distinctlyminty]

Hi Alexander,
Thanks for the feedback.

To fix the workspace, you can change path to ...

Would moving the workspace file out of the library, up one level, have the same effect?

• We still support these files, but they can't be referenced as imports.

Could you be more specific about what part is incorrect in the example file? The link to PEP8 mentions that modules and packages should have short, lower case names, and the myLib example file clearly has an upper case 'L'. Is it that the cause of imports being unresolved? Is it the structure of import e.g. import some.thing as st?

[AlexanderSher]

Hi @Anti-Distinctlyminty

Would moving the workspace file out of the library, up one level, have the same effect?

Yes

Is it that the cause of imports being unresolved?

It may be the case. In your example, imports are unresolved because of the path. Language Server will be resolving imports with upper cases, but it is better to follow the conversion. Google has a good summary for it: http://google.github.io/styleguide/pyguide.html#3164-guidelines-derived-from-guidos-recommendations

[Anti-Distinctlyminty]

Thanks @AlexanderSher. Unfortunately, changing path to '..' did not completely resolve the issue in my real life project.
Even though there is no warning about unresolved imports, the language server still seems unable to effectively manage this module. In this example you can see that the language server is seemingly parsing the file and picking up the docstring, but is unable to navigate to its definition

I'll take a look at the link you provided and tidy things up tomorrow and report back.

[AlexanderSher]

houdini_io_globals - is it a module?

[Anti-Distinctlyminty]

Yes. nymus3d is the package.
Folder structure is

nymus3d
    |----__init__.py
    |----houdini_io_globals.py
    |----...```

[egabrum]

All the standard libraries are giving me
unresolved import 'time'Python(unresolved-import), for example.

IMHO this bug should be very high in the priority list. I certainly can't work with the language server in these conditions. Fortunately there is jedi as a backup.

[johnybx]

Seems to be related to #1156

[AlexanderSher]

@Anti-Distinctlyminty, we had several improvements recently. Can you try the daily build "python.analysis.downloadChannel": "daily"?

[AlexanderSher]

@egabrum, your bug is most likely different from what @Anti-Distinctlyminty had. If you still have it on "python.analysis.downloadChannel": "daily" channel, can you please file a separate bug?

[Anti-Distinctlyminty]

@AlexanderSher I put "python.analysis.downloadChannel": "daily" in my settings.json, but I'm not sure I'm getting daily builds. The current is 2019.6.24221 from 3 days ago and the next on the list of available version is from 18 days ago.

The behaviour is still quite strange as shown here

If I import as, then I have to use the alias for any action to be taken. But then the 'go to definition' action takes me to another file with the same import statement import nymus3d.data_io as dio. If I do the same from there, then I get to the actual file.
data_io.py, houdini_io_gui.py and houdini_io_objects.py are all files within the nymus3d folder.

[MikhailArkhipov]

Language server version is unrelated to extension version. It is displayed in the output on startup. Something like 0.3.x

[Anti-Distinctlyminty]

Thanks @MikhailArkhipov , here it is...
Microsoft Python Language Server version 0.3.30.0

[nullyfae]

I have this, with files in the same directory (that was oppened with the option at context menu).

[jakebailey]

You have a src directory. Have you set extraPaths as in our troubleshooting doc? https://github.com/microsoft/python-language-server/blob/master/TROUBLESHOOTING.md#unresolved-import-warnings

[nullyfae]

Thanks! that works!
Why vscode can't do it by it self? 🤔

[jakebailey]

It's not a straightforward problem, src may be a common import root, but there are many others. microsoft/vscode-python#924 is related. Reducing the configuration required in these situation is of course desired, but requires some extra investigation into what's common, how to present something in the UI, how to disable auto detection if that gets added, among other issues to do with how our import resolver behaves on nested roots, etc.

[nullyfae]

I didn't mean to add the paths by itself, but to determinate "oh, i'm on this path, so this import must refer to that file".

[jakebailey]

If it were that simple, we'd probably be doing it, but that's unfortunately not how Python's going to go load the code either.

• The way modules are laid out can mean they are imported differently in different versions of Python.
• Namepsace packages change this behavior as well, where multiple completely different directories can provide modules underneath other modules.
• A file being run as a script means its sys.path changes.
• pth files can modify paths even more.
• If we go start looking for imports up and up and up, we'll eventually start leaving the workspace and scanning through the drive (which has bad implications).
• One name could end up meaning two things in two different places, which has its own bad implications in our analysis (persistence, for one, along with other things).

All of these mean we have to make tradeoffs for what behavior we want/can feasibly do; we could be very, very loose, but then you may go to run the code and it break and you'd wonder why the editor didn't tell you something was wrong beforehand.

[egabrum]

I'm sorry, but I'm at a loss. I've "python.analysis.downloadChannel": "daily" enabled and a bunch of packages installed as editable installs.

• Some are correctly imported, even if they are not in my extraPaths.
• Some are never correctly imported, even if I add them to my extraPaths.
• Some are imported but, when I try to go to the Definition, it takes me to the init.py file in the package (Jedi takes me to the actual definition).

I can't publish all this code and it could take me a some time to abstract it, making sure that the same behavior is still reproduced. The worst part is the inconsistent behavior.

Maybe the language server has additional requirements for the way the components of a package are imported in the init.py within the package? Something that the interpreter and Jedi tolerate, but the language server is more picky about?

[Anti-Distinctlyminty]

I also tested this again with the simple myLib example files in the original report.
The results are still the same with an unresolved import 'myLib'.

[jakebailey]

Some are imported but, when I try to go to the Definition, it takes me to the init.py file in the package (Jedi takes me to the actual definition).

That is #1138.

EDIT: No, it's not, sorry. That sounds like something different where something's location is lost and set to the default.

[jakebailey]

That original example doesn't work, no... It's using a VS Code workspace to make a project appear to have a folder called myLib, when it actually doesn't. VS Code tells us the root is the folder that's opened, which is myLib, so that means trying to import myLib.importMe is going to fail, because that's something outside the workspace.

With our current import resolver (which places the workspace root as the first import root), I'm not sure how we could make this work. All other scenarios we've looked at involve code in different folders inside the workspace; trying to open a subfolder and then import above (or make the workspace "look" like there are more folders than there actually are) seems really tricky to get right.

[SHxKM]

Post edited: unrelated.

[jakebailey]

That is unrelated. See #1345.

Check your language server version, it may be fixed in 0.3.46+. In any case, #1345 is the place to help us pin that issue down.

A reload usually fixes it, from what I've seen.

[datalifenyc]

You have a src directory. Have you set extraPaths as in our troubleshooting doc? https://github.com/microsoft/python-language-server/blob/master/TROUBLESHOOTING.md#unresolved-import-warnings

1. As stated in the instructions above, modify .vscode/settings.json to include "python.autoComplete.extraPaths": ["./src"]
2. In Visual Studio Explorer, right click on your project folder (assuming it is the parent of the folder that contains your sub-directory you want to import from) and click Copy Relative Path. Replace ./src with the relative path value.
3. In your main script, you should be able to type from sub-directory.module import * without receiving the Warning message in the Problems tab.

Thanks @jakebailey

[webdog]

I encountered this issue by copying my python project to a nested folder inside a new VSCode workspace. My Python interpreter's behavior does not change, nor does my Python Application. My imports resolve successfully when running my flask application.

As a nested folder:

When not opened as a nested folder (Same interpreter):

This behavior consumed about 30 minutes of the morning attempting to debug/understand this behavior, scratching my head why VSCode would complain about these imports when my Flask application ran normally. Finally, I started googling around and came across this Issue.

Confusingly, VSCode will attempt to prefix imports by the root workspace folder name, and autocomplete works with this behavior. Thinking I may have understood imports incorrectly, I attempted to bandaid my application based on the Lang Server's suggestion, which then the Language Server clears itself of the import error.

However, this fix will then break the Python interpreter as the import no longer evaluates the module and returns ModuleNotFoundError:

Traceback (most recent call last):
  File "/Users/cwebdog/anaconda3/lib/python3.7/site-packages/flask/cli.py", line 240, in locate_app
    __import__(module_name)
  File "/Users/cwebdog/github/workspace/application/quickweather.py", line 3, in <module>
    app = create_app()
  File "/Users/cwebdog/github/workspace/application/app/__init__.py", line 7, in create_app
    from app.api import bp as api_bp
  File "/Users/cwebdog/github/workspace/application/app/api/__init__.py", line 5, in <module>
    from application.app.api import routes
ModuleNotFoundError: No module named 'application'

The local Debugger also seems to understand these "fixed" imports and won't error, though I was under the impression that this debugger was based on my environment's interpreter (In my case, conda using python 3.7.8). I don't have a saved screenshot, but if I can replicate this behavior, I'll update with another comment.

Additionally, the papercut here is the Logging and Tooling didn't provide any breadcrumbs or feedback that the issue was the Python Language Server running into an issue deconstructing the import tree.

The suggested fix of adding another import path to the User Settings seems to have fixed it:

The little papercut here, IMO:

• This now lives in my user settings across many workspaces (Based on the way I understand how User settings propagate in VSCode, if wrong, please correct)
• If I add this as a workspace setting, I'll have to remember to do this each time for a python project that I choose to include in a nested folder or workspace (Thinking off the cuff, a git submodule would probably be a common use case pattern here, even though that wasn't what uncovered this behavior for me). This type of workflow, for me, will be just infrequent enough that I may struggle to remember this behavior exists.

Updating this setting also now seems to break/resolve the previous behavior, if I attempt to import from the root level folder workspace name, this now returns an unresolved import:

...Which I'm not sure if this would be expected or not, I was under the impression the setting was adding a path to the current module evaluator, so I would have expected for the bug to still be present. My understanding is limited, and I'm probably flawed in that thought process somewhere.

I'm not sure what the fix here is specifically, as I don't understand the evaluation path of the Language Server, but if the Server could traverse the workspace tree, searching for the existence of local modules, hunting for for files like /*/**/__init__.py (Please excuse my quick and rough regex), then perhaps those could could be added to the workspace path programmatically?

Thanks for working on the Language Server! It's my most utilized VSCode tool 😺