Python conda run incorrect environment path

[tbenthompson]

Behaviour

I get the following error when running pytest test discovery. It looked like the environment path is getting duplicated: envs/kevlar/envs/kevlar. I'm on the latest conda 4.11 and fully updated to the latest VSCode version. I've seen #18486 and this seems related but the updates and solutions there have done nothing for me. I've also recreated the environment to see if a brand new environment would avoid this error.

 conda run -n kevlar --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ./inla_explore -s --cache-clear
cwd: ./inla_explore
[ERROR 2022-1-9 14:32:9.956]: Error discovering pytest tests:
 r [Error]: 
EnvironmentLocationNotFound: Not a conda environment: /Users/tbent/.mambaforge/envs/kevlar/envs/kevlar


    at ChildProcess.<anonymous> (/Users/tbent/.vscode/extensions/ms-python.python-2022.0.1814523869/out/client/extension.js:32:39235)
    at Object.onceWrapper (events.js:422:26)
    at ChildProcess.emit (events.js:315:20)
    at maybeClose (internal/child_process.js:1048:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)

Expected vs. Actual

XXX

Steps to reproduce:

1. Open the "Testing" panel.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.2
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
• Value of the python.languageServer setting: Default

User Settings

defaultLS: {"defaultLSType":"Pylance"}

downloadLanguageServer: true

envFile: "<placeholder>"

venvPath: "<placeholder>"

venvFolders: "<placeholder>"

condaPath: "<placeholder>"

pipenvPath: "<placeholder>"

poetryPath: "<placeholder>"

languageServer: "Pylance"

linting
• enabled: true
• cwd: "<placeholder>"
• Flake8Args: "<placeholder>"
• flake8Enabled: false
• flake8Path: "<placeholder>"
• lintOnSave: true
• banditArgs: "<placeholder>"
• banditEnabled: false
• banditPath: "<placeholder>"
• mypyArgs: "<placeholder>"
• mypyEnabled: false
• mypyPath: "<placeholder>"
• pycodestyleArgs: "<placeholder>"
• pycodestyleEnabled: false
• pycodestylePath: "<placeholder>"
• prospectorArgs: "<placeholder>"
• prospectorEnabled: false
• prospectorPath: "<placeholder>"
• pydocstyleArgs: "<placeholder>"
• pydocstyleEnabled: false
• pydocstylePath: "<placeholder>"
• pylamaArgs: "<placeholder>"
• pylamaEnabled: false
• pylamaPath: "<placeholder>"
• pylintArgs: "<placeholder>"
• pylintPath: "<placeholder>"

sortImports
• args: "<placeholder>"
• path: "<placeholder>"

formatting
• autopep8Args: "<placeholder>"
• autopep8Path: "<placeholder>"
• provider: "black"
• blackArgs: "<placeholder>"
• blackPath: "<placeholder>"
• yapfArgs: "<placeholder>"
• yapfPath: "<placeholder>"

testing
• cwd: "<placeholder>"
• debugPort: 3000
• nosetestArgs: "<placeholder>"
• nosetestsEnabled: undefined
• nosetestPath: "<placeholder>"
• promptToConfigure: true
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• pytestPath: "<placeholder>"
• unittestArgs: "<placeholder>"
• unittestEnabled: false
• autoTestDiscoverOnSaveEnabled: true

terminal
• activateEnvironment: true
• executeInFileDir: "<placeholder>"
• launchArgs: "<placeholder>"

experiments
• enabled: true
• optInto: []
• optOutFrom: []

insidersChannel: "off"

tensorBoard
• logDirectory: "<placeholder>"

[calebho]

+1, also seeing the same error.

Extension version: v2022.0.1814523869
Conda version: 4.11.0

[karthiknadig]

That is an issue with conda itself. Previous versions of the extension were using a hack. That hack no longer works starting conda 4.10. we decided to use the conda run as that was recommend way to run scripts using conda. Please file a bug https://github.com/conda/conda/issues

[karthiknadig]

If you need to you can revert to an older version of the extension using the gear icon next to the extension in the extensions view

[tbenthompson]

That is an issue with conda itself. Previous versions of the extension were using a hack. That hack no longer works starting conda 4.10. we decided to use the conda run as that was recommend way to run scripts using conda. Please file a bug https://github.com/conda/conda/issues

Are you sure about this? When I run the exact same command as vscode in a terminal, the command runs just fine. That leads me to believe this is some kind of error in the extension's shell environment setup.

>  conda run -n kevlar --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ./inla_explore -s --cache-clear

>>>PYTHON-EXEC-OUTPUT
[]
<<<PYTHON-EXEC-OUTPUT

[karthiknadig]

Is there a difference between the environment variables in the VS Code terminal VS the external terminal? What happens if you activate the environment in a terminal and then run the above command? The reason I believe this is a conda issue is because we don't compute the environment path for named environments. I believe that something in going on with run when it is run from an already activated environment.

related mamba-org/mamba#1467

[tbenthompson]

Thanks for thinking about this!

Here, I'm running the exact test discovery command first from the internal VSCode terminal, both with the base environment activated and then with the relevant environment activated. And then, in the second block, I run both commands again from an external terminal (mac, kitty, zsh). In both situations, the commands run without error. Whereas when run from the pytest test discovery tool, the command fails with the error from the original issue post. This is why I lean towards thinking there's something wrong in the test discovery conda call.

My only theory is that perhaps the test discovery is running in a different shell and hasn't had the shell code that gets added by conda init? I have no idea why that would result in the doubling of the environment path in relation to the conda root directory. I'll investigate this possibility a little bit.

VSCode built-in terminal:

kevlar 01_2022_tbt_inla !4 ?6 > conda run -n kevlar --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ./inla_explore -s --cache-clear py base
>>>PYTHON-EXEC-OUTPUT
[]
<<<PYTHON-EXEC-OUTPUT
kevlar 01_2022_tbt_inla !4 ?6 > conda activate kevlar                                                                                                                                                                                                                                                                                                    py base
kevlar 01_2022_tbt_inla !4 ?6 > conda run -n kevlar --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ./inla_explore -s --cache-clear
>>>PYTHON-EXEC-OUTPUT
[]
<<<PYTHON-EXEC-OUTPUT

External terminal:

kevlar 01_2022_tbt_inla !4 ?6 > conda run -n kevlar --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ./inla_explore -s --cache-clear
>>>PYTHON-EXEC-OUTPUT
[]
<<<PYTHON-EXEC-OUTPUT
kevlar 01_2022_tbt_inla !4 ?6 > conda activate kevlar                                                                                                                                   py base
kevlar 01_2022_tbt_inla !4 ?6 > conda run -n kevlar --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ./inla_explore -s --cache-clear
>>>PYTHON-EXEC-OUTPUT
[]

pytest test discovery:

> conda run -n kevlar --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ./inla_explore -s --cache-clear
cwd: ./inla_explore
[ERROR 2022-1-9 14:50:25.80]: Error discovering pytest tests:
 r [Error]: 
EnvironmentLocationNotFound: Not a conda environment: /Users/tbent/.mambaforge/envs/kevlar/envs/kevlar

[tbenthompson]

Ok, this is solved for me. The only thing I changed is that I cleared the python.condaPath setting. And the error went away. But that setting was absolutely correct and I don't have any other conda installations on my system, so I'm still confused. But, I'm now happy and confused. Based on the upvotes and other comments, I would prefer not to close the issue since it's clear others are also seeing these errors.

[karthiknadig]

@tbenthompson This is strange, and thanks for digging into it. Can you share what you had for python.condaPath? Does it have any unicode characters in the path?

[tbenthompson]

I had python.condaPath as /Users/tbent/.mambaforge/bin/conda. There aren't any unicode characters.

Also, this is hilarious. I tried setting python.condaPath again to the above path in order to see if I could reproduce the error. I can reproduce the error. But, now when I clear the setting again, it does not resolve. So, I'm back to where I started with a screwed up pytest + vscode integration. I also tried out seven other active environments and three of those work fine and four fail. I don't see a useful pattern though!

[karthiknadig]

@tbenthompson Can you tell me what you have for this setting?

[tbenthompson]

It is unchecked. If I check it doesn't change any of the previously discussed behavior.

[cswartzvi]

This might not be directly related, but I am able to trigger the same error as @tbenthompson in the pytest test discovery if both the following conditions are met:

• the conda package is installed in a local environment (I realize this isn't recommended, but I was testing a conda CLI plugin)
• vscode is launched from outside the local environment

Steps to reproduce:

1. Create a new environment with conda create -n vscode-conda-repro python=3.9 pytest
2. Fire up vscode via a launcher, taskbar, or command line without the vscode-conda-repro environment activated
3. Make a simple test and select the vscode-conda-repro interpreter
4. Run the pytest test discovery - OK
5. Install conda conda install -n vscode-conda-repro conda and restart vscode
6. Refresh pytest test discovery - ERROR
7. Unstall conda conda uninstall -n vscode-conda-repro conda and restart vscode
8. Refresh pytest test discovery - OK

Some notes:

• If you activate vscode-conda-repro and launch vscode from the command line (code .) the error goes away
• If you set python.condaPath to the correct conda path the error goes away
• If you revert to v2021.12.1559732655 the error goes away
• I also ran similar commands as Python conda run incorrect environment path #18493 (comment) from Output > Python with similar results - no errors when executed from the command line inside or outside the vscode-conda-repro environment

Perhaps this is a known issue? It seems to be mentioned here: https://devblogs.microsoft.com/python/python-in-visual-studio-code-february-2022-release/#improvements-when-using-anaconda

Edit: I am running ubuntu 20.04 on WSL2 if that matters for "special characters in file paths" as mentioned in the blog.

[tbenthompson]

@cswartzvi Thanks for the super useful info and explanation. I just checked and you are absolutely right that I have the conda package installed in all the environments that are causing the issue. Uninstalling conda from an env solves the issues. Now that I think about the symptoms it seems obvious that this is the problem!

While this is probably user error, it seems like a type of user error that the VSCode could easily either correct for or yell about.

[karthiknadig]

@cswartzvi @tbenthompson Thanks for the investigation and patience. We will discuss this internally and update on this.

[vancleve]

this is the conda issue:
conda/conda#11174

it seems pretty serious from the point of view of using conda run in vscode...its certainly causing my vscode-python extension to run the wrong python version

[tbenthompson]

Just wanted to update the thread that I've run into this problem again. After I remembered that I'd dealt with it a few weeks ago, I was able to solve it the same way again: uninstall conda from the environment. This is a bit unfortunate and annoying because there are common use cases and real benefits to having conda installed in an environment.

[lneisenman]

I'm setting up VSCode on a new computer and I think I am having a similar problem with pytest test discovery. I'm developing a package in a Conda environment called seeg that includes pytest. When I run the test discovery, I get an error indicating that pytest was not found. Looking at the output, it seems that the error occurs with a command that starts with conda run python instead of conda run -n seeg python so the wrong python is being called.

Version: 1.66.0 (user setup)
Commit: e18005f0f1b33c29e81d732535d8c0e47cafb0b5
Date: 2022-03-30T05:50:14.623Z
Electron: 17.2.0
Chromium: 98.0.4758.109
Node.js: 16.13.0
V8: 9.8.177.11-electron.0
OS: Windows_NT x64 10.0.19042

Here is the output from the discovery:
conda run -n seeg --no-capture-output --live-stream python ~.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\get_output_via_markers.py ~.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir . -s --cache-clear
cwd: .
[ERROR 2022-3-6 12:3:27.775]: Error discovering pytest tests:
[r [Error]: Traceback (most recent call last):
File "c:\Users\leisenman.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\get_output_via_markers.py", line 26, in
runpy.run_path(module, run_name="main")
File "C:\Users\leisenman\Miniconda3\lib\runpy.py", line 269, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\leisenman\Miniconda3\lib\runpy.py", line 96, in _run_module_code
run_code(code, mod_globals, init_globals,
File "C:\Users\leisenman\Miniconda3\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "c:\Users\leisenman.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\testing_tools\run_adapter.py", line 17, in
from testing_tools.adapter.main import parse_args, main
File "c:\Users\leisenman.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\testing_tools\adapter_main.py", line 9, in
from . import pytest, report
File "c:\Users\leisenman.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\testing_tools\adapter\pytest_init.py", line 7, in
from ._discovery import discover
File "c:\Users\leisenman.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\testing_tools\adapter\pytest_discovery.py", line 8, in
import pytest
ModuleNotFoundError: No module named 'pytest'
ERROR conda.cli.main_run:execute(41): conda run python c:\Users\leisenman\.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\get_output_via_markers.py c:\Users\leisenman\.vscode\extensions\ms-python.python-2022.4.0\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\leisenman\Documents\Python\seeg -s --cache-clear failed. (See above for error)

at ChildProcess.<anonymous> (c:\Users\leisenman\.vscode\extensions\ms-python.python-2022.4.0\out\client\extension.js:32:38969)
at Object.onceWrapper (node:events:510:26)
at ChildProcess.emit (node:events:390:28)
at maybeClose (node:internal/child_process:1064:16)
at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)]

[karthiknadig]

Looks like this is addressed in conda, conda/conda#11174.

The fix is available in the canary version of conda.

conda update -c conda-canary/label/dev conda

We plan on continuing with conda run for running tests.

[xarpy]

Hi guys, @karthiknadig, I tried this solution but it didn't work for me. What ended up solving it, was to perform the installation of an earlier version of the ms-python package, it being version v2022.2.1924087327. But even so, I still can't run any more tests with plugin Test Explorer UI only in terminal by command line,now with the previous version package for ms-python I can run only the debug mode.

Because of this, I created a new issue, related to this, for a newer version to try to understand the problem! First of all, I use vscode on linux, in the version described in the issue.

[karrtikr]

Created a bug on conda: conda/conda#12082, feel free to upvote that to raise priority. Closing in favor of conda.