"Test framework pytest is not installed. Install?" even though it *is* installed.

[ericsnowcurrently]

I have a workspace with the following:

• valid tests
• pytest installed in the configured interpreter
• pytest configured in the workspace settings

With the most recent insiders build of the extension I clicked on the "Discover Tests" button in the test explorer. A window popped up saying Test framework pytest is not installed. Install? I verified that pytest was indeed really installed for the current interpreter. Then I looked at the "Python Test Log" Output panel. It showed that pytest discovery had failed (due to a import errors in my test files).

So when pytest discovery fails, the extension is erroneously acting as though pytest hasn't been installed yet.

Given that pytest discovered most of the tests, I would have expected those tests to show up in the test explorer. I would also have expected a warning to pop up saying that there was a failure during test discovery (with a button to open the "Python Test Log" panel).

[ericsnowcurrently]

I've marked this as a regression, as I'm sure this wasn't a problem before.

[ericsnowcurrently]

Here's the output from the "Python Test Log" Output panel:

python /home/esnow/.vscode/extensions/ms-python.python-2019.6.18460-dev/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear .
Test Discovery failed: 
Error: ============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-4.6.2, py-1.8.0, pluggy-0.12.0
rootdir: /home/esnow/projects/work/test_projects/testing
collected 110 items / 2 errors / 108 selected
<Package /home/esnow/projects/work/test_projects/testing/complex/tests>
  <Module test_42-43.py>
    <Function test_simple>
  <Module test_42.py>
    <Function test_simple>
  <DoctestTextfile test_doctest.txt>
    <DoctestItem test_doctest.txt>
  <Module test_foo.py>
    <Function test_simple>
  <Module test_mixed.py>
    <Function test_top_level>
    <Function test_skipped>
    <Class TestMySuite>
        <Function test_simple>
    <UnitTestCase MyTests>
      <TestCaseFunction test_simple>
      <TestCaseFunction test_skipped>
  <Module test_pytest.py>
    <Function test_simple>
    <Function test_failure>
    <Function test_runtime_skipped>
    <Function test_runtime_failed>
    <Function test_raises>
    <Function test_skipped>
    <Function test_maybe_skipped>
    <Function test_known_failure>
    <Function test_warned>
    <Function test_custom_marker>
    <Function test_multiple_markers>
    <Function test_dynamic_1>
    <Function test_dynamic_2>
    <Function test_dynamic_3>
    <Class TestSpam>
        <Function test_simple>
        <Function test_skipped>
        <Class TestHam>
            <Class TestEggs>
                <Function test_simple>
    <Class TestEggs>
        <Function test_simple>
    <Function test_param_01[]>
    <Function test_param_11[x0]>
    <Function test_param_13[x0]>
    <Function test_param_13[x1]>
    <Function test_param_13[x2]>
    <Function test_param_13_repeat[x0]>
    <Function test_param_13_repeat[x1]>
    <Function test_param_13_repeat[x2]>
    <Function test_param_33[1-1-1]>
    <Function test_param_33[3-4-5]>
    <Function test_param_33[0-0-0]>
    <Function test_param_33_ids[v1]>
    <Function test_param_33_ids[v2]>
    <Function test_param_33_ids[v3]>
    <Function test_param_23_13[1-1-z0]>
    <Function test_param_23_13[1-1-z1]>
    <Function test_param_23_13[1-1-z2]>
    <Function test_param_23_13[3-4-z0]>
    <Function test_param_23_13[3-4-z1]>
    <Function test_param_23_13[3-4-z2]>
    <Function test_param_23_13[0-0-z0]>
    <Function test_param_23_13[0-0-z1]>
    <Function test_param_23_13[0-0-z2]>
    <Function test_param_13_markers[x0]>
    <Function test_param_13_markers[???]>
    <Function test_param_13_markers[2]>
    <Function test_param_13_skipped[x0]>
    <Function test_param_13_skipped[x1]>
    <Function test_param_13_skipped[x2]>
    <Function test_param_23_raises[1-None]>
    <Function test_param_23_raises[1.0-None]>
    <Function test_param_23_raises[2-catch2]>
    <Class TestParam>
        <Function test_simple>
        <Function test_param_13[x0]>
        <Function test_param_13[x1]>
        <Function test_param_13[x2]>
    <Class TestParamAll>
        <Function test_param_13[x0]>
        <Function test_param_13[x1]>
        <Function test_param_13[x2]>
        <Function test_spam_13[x0]>
        <Function test_spam_13[x1]>
        <Function test_spam_13[x2]>
    <Function test_fixture>
    <Function test_mark_fixture>
    <Function test_param_fixture[x0]>
    <Function test_param_fixture[x1]>
    <Function test_param_fixture[x2]>
    <Function test_param_mark_fixture[x0]>
    <Function test_param_mark_fixture[x1]>
    <Function test_param_mark_fixture[x2]>
    <Function test_fixture_param[spam]>
    <Function test_fixture_param[eggs]>
  <Module test_pytest_param.py>
    <Function test_param_13[x0]>
    <Function test_param_13[x1]>
    <Function test_param_13[x2]>
    <Class TestParamAll>
        <Function test_param_13[x0]>
        <Function test_param_13[x1]>
        <Function test_param_13[x2]>
        <Function test_spam_13[x0]>
        <Function test_spam_13[x1]>
        <Function test_spam_13[x2]>
  <Module test_unittest.py>
    <UnitTestCase MyTests>
      <TestCaseFunction test_dynamic_>
      <TestCaseFunction test_failure>
      <TestCaseFunction test_known_failure>
      <TestCaseFunction test_maybe_not_skipped>
      <TestCaseFunction test_maybe_skipped>
      <TestCaseFunction test_simple>
      <TestCaseFunction test_skipped>
      <TestCaseFunction test_skipped_inside>
      <TestCaseFunction test_with_nested_subtests>
      <TestCaseFunction test_with_subtests>
    <UnitTestCase OtherTests>
      <TestCaseFunction test_simple>
<Package /home/esnow/projects/work/test_projects/testing/complex/tests/v>
  <Module test_eggs.py>
    <Function test_simple>
    <Class TestSimple>
        <Function test_simple>
  <Module test_ham.py>
    <Function test_simple>
    <Function test_not_hard>
  <Module test_spam.py>
    <Function test_simple>
    <Function test_simpler>
<Module complex/tests/w/test_spam.py>
  <Function test_simple>
<Module complex/tests/w/test_spam_ex.py>
  <Function test_simple>
<Package /home/esnow/projects/work/test_projects/testing/complex/tests/x/y/z>
  <Module test_ham.py>
    <Function test_simple>
<Package /home/esnow/projects/work/test_projects/testing/complex/tests/x/y/z/a>
  <Module test_spam.py>
    <Function test_simple>
<Package /home/esnow/projects/work/test_projects/testing/complex/tests/x/y/z/b>
  <Module test_spam.py>
    <Function test_simple>

==================================== ERRORS ====================================
__________________ ERROR collecting simple/tests/test_spam.py __________________
ImportError while importing test module '/home/esnow/projects/work/test_projects/testing/simple/tests/test_spam.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
ModuleNotFoundError: No module named 'tests.test_spam'
_______________ ERROR collecting syntax-error/tests/test_spam.py _______________
ImportError while importing test module '/home/esnow/projects/work/test_projects/testing/syntax-error/tests/test_spam.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
ModuleNotFoundError: No module named 'tests.test_spam'
=============================== warnings summary ===============================
.venv/lib/python3.7/site-packages/_pytest/mark/structures.py:337
  /home/esnow/projects/work/test_projects/testing/.venv/lib/python3.7/site-packages/_pytest/mark/structures.py:337: PytestUnknownMarkWarning: Unknown pytest.mark.spam - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
===================== 1 warnings, 2 error in 0.10 seconds ======================

Traceback (most recent call last):
  File "/home/esnow/.vscode/extensions/ms-python.python-2019.6.18460-dev/pythonFiles/testing_tools/run_adapter.py", line 16, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/home/esnow/.vscode/extensions/ms-python.python-2019.6.18460-dev/pythonFiles/testing_tools/adapter/__main__.py", line 90, in main
    parents, result = run(toolargs, **subargs)
  File "/home/esnow/.vscode/extensions/ms-python.python-2019.6.18460-dev/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 35, in discover
    raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 2)

[ericsnowcurrently]

Per @DonJayamanne, this is because the extension looks for "ModuleNotFoundError" (or "ImportError"?) in the output to decide if pytest is installed or not. Apparently that is too naive.

[adamryczkowski]

I confirm ericsnowcurrently's diagnosis. I wasted full working day installing-reinstalling-purging-installing again pytest, python, conda, vscode... I just had bad luck in that my test was failing because of the missing module...

Please note, that my "bad luck" is actually a very probable scenario caused by another usability problem: when you write a unit test in pytest, just importing the module you want to test with import tobetested at the beginning of the test file does not work, and triggers the ModuleNotFoundError: No module named 'tobetested'error... then you are bombarded with the apparent problem with the pytest installation...

[DonJayamanne]

Prescribed Solution

• Modify the following code

                if (isNotInstalledError(reason as Error) && !quietMode) {
                    this.installer.promptToInstall(this.product, this.workspaceFolder)
                        .catch(ex => traceError('isNotInstalledError', ex));
                }

Check whether the module is installed using the code const isInstalled = await pythonExecutionService.isModuleInstalled(moduleName); instead of using isNotInstalledError

[ericsnowcurrently]

I've verified that the extension no longer says pytest isn't installed (even though it is). The text in "Python Test Log" output panel did not change (as expected).