Missing Pytest results due to pytest.ini location changing expected root folder

[bschnurr]

Environment data

• VS Code version: XXX
• Extension version (available under the Extensions sidebar): 2019.8.30787 (13 August 2019)
• OS and version: windows 18362.239
• Python version (& distribution if applicable, e.g. Anaconda): python 3.7.4
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: pytest 5.0.1
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version #3977): XXX

Expected behaviour

Running a discovered test should show result icon in test viewer

Actual behaviour

Test passes in output window, but doesn't update in test viewer

Steps to reproduce:

1. Create a base folder "PythonApplication15" with a subfolder "PythonApplication15" (names shouldn't matter)
2. In the subfolder add a unittest and an empty pytest.ini
3. use VSCode open folder on the main folder and run pytest with root "."
4. The test will be discovered under the subfolder
5. Right click run test
6. output shows results..However in the pytest output the root folder has changed from base to subfolder (see below)
7. no result shown in test viewer

Same issue is happening in PTVS..
discovery is running with root base folder
junitxml results have a root as subfolder and filepaths are missing "PythonApplication15" in front

cwd: c:\Users\bschnurr\source\repos\PythonApplication15
rootdir: c:\Users\bschnurr\source\repos\PythonApplication15\pythonapplication15, inifile: pytest.ini

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

============================= test session starts =============================
platform win32 -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: c:\Users\bschnurr\source\repos\PythonApplication15\pythonapplication15, inifile: pytest.ini
collected 1 item

pythonapplication15\test_3.py .                                          [100%]

- generated xml file: C:\Users\bschnurr\AppData\Local\Temp\tmp-106246BVUoGxIZF9n.xml -
========================== 1 passed in 0.02 seconds ===========================

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

> ~\AppData\Local\Programs\Python\Python37\python.exe -m pytest --junitxml=C:\Users\bschnurr\AppData\Local\Temp\tmp-106246BVUoGxIZF9n.xml .\pythonapplication15\test_3.py::Test_test_3::test_A
cwd: c:\Users\bschnurr\source\repos\PythonApplication15

[bschnurr]

It seems pytest wont search in child folders for pytest.ini.. when you run pytest directly from command line it wont find the pytest.ini in the subfolder, unless we tell it to run a specific test, then it changes to search that test's folder.

C:\Users\bschnurr\source\repos\PythonApplication15>C:\Users\bschnurr\source\repos\PythonApplication8\PythonApplication8\pytestenv\Scripts\python.exe -m pytest
======================================================================================= test session starts ========================================================================================
platform win32 -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: C:\Users\bschnurr\source\repos\PythonApplication15
collected 1 item

PythonApplication15\test_3.py .                                                                                                                                                               [100%]

===================================================================================== 1 passed in 0.04 seconds =====================================================================================

C:\Users\bschnurr\source\repos\PythonApplication15>C:\Users\bschnurr\source\repos\PythonApplication8\PythonApplication8\pytestenv\Scripts\python.exe -m pytest .\PythonApplication15\test_3.py
======================================================================================= test session starts ========================================================================================
platform win32 -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: C:\Users\bschnurr\source\repos\PythonApplication15\PythonApplication15, inifile: pytest.ini
collected 1 item

PythonApplication15\test_3.py .                                                                                                                                                               [100%]

===================================================================================== 1 passed in 0.02 seconds =====================================================================================
``

[bschnurr]

One solution might be to always pass the root folder found from discovery to the pytest execution call using
--rootdir=path****

[ericsnowcurrently]

I have verified that with a pytest.ini in the subdirectory, neither the test explorer nor the code lenses show the test results even though the test passes.

[bschnurr]

Similar issue with azure-cli repro microsoft/PTVS#5626

[bschnurr]

For now VS is setting --rootdir as a command line arg for both pytest discovery and execute so that they always match. change here. microsoft/PTVS#5628

[gramster]

Bill, is there a better option than your workaround? Else we could close this (after documenting if necessary).

[bschnurr]

I think the options could be.

1. my fix that forces them to match
2. detect when discovery/execution result's root directories don't match and message to the user.
3. Allow users to specify additional command line arguments for discovery/execution ( can't rely on pytest.ini settings in this case)
4. Just document this limitation

[ericsnowcurrently]

FYI, this is related to #6548. For that issue we need to adjust how we use the "--rootdir" CLI arg to pytest. I expect it will affect the solution here.

[kimadeline]

Fixed in 2019.11.41015-dev as part of #6548.