Refactor pytest discovery code in extension

[drorata]

Environment data

• VS Code version: 1.30.1 (dea8705087adb1b5e5ae1d9123278e178656186a)
• Extension version (available under the Extensions sidebar): 2018.12.1
• OS and version: macOS 10.14.2 (18C54)
• Python version (& distribution if applicable, e.g. Anaconda): 3.6.8 (Anaconda)
• Type of virtual environment used : conda
• Relevant/affected Python packages and their versions: pytest 4.1.0

Local setting is:

{
    "python.pythonPath": "/Users/dror/anaconda3/envs/tmp/bin/python",
    "python.unitTest.pyTestArgs": [
        "."
    ],
    "python.unitTest.unittestEnabled": false,
    "python.unitTest.nosetestsEnabled": false,
    "python.unitTest.pyTestEnabled": true
}

Expected behaviour

When running Python: Discover Unit Tests I expect not to receive an error message.

Actual behaviour

When I run pytest at the root of my project (from the terminal) everything works as expected: all tests are found and executed. However, when I try to run Python: Discover Unit Tests, I get the following error No tests discovered, please check the configuration settings for the tests.. The output is odd:

============================= test session starts ==============================
platform darwin -- Python 3.6.8, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/dror/tmp, inifile:
collected 2 items
<Module test_foo.py>
  <Function test_a>
  <Function test_b>

========================= no tests ran in 0.01 seconds =========================```

Moreover, when I do Python: Run All Tests I get the above output, followed by actually running the tests and passing them (as if I was running pytest directly in the console).

Steps to reproduce:

1. Create a conda virtual environment (tmp) and install pytest using pip
2. Create foo.py and test_foo.py (see below).
3. Run pytest in the console and see the two tests pass.
4. In vscode, run Python: Discover Unit Tests.
5. Then run Python: Run All Unit Tests.

#  foo.py
def bar(x):
    return x + 10

############################################

#  test_foo.py 
import foo


def test_a():
    assert 1 == 1


def test_b():
    assert foo.bar(5) == 15

Logs

Output for Python in the Output: see above.

The output of running the tests is:

============================= test session starts ==============================
platform darwin -- Python 3.6.8, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/dror/tmp, inifile:
collected 2 items
<Module test_foo.py>
  <Function test_a>
  <Function test_b>

========================= no tests ran in 0.01 seconds =========================
============================= test session starts ==============================
platform darwin -- Python 3.6.8, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/dror/tmp, inifile:
collected 2 items

test_foo.py ..                                                           [100%]

 generated xml file: /var/folders/w6/6m9g0bjn1q17bqdw33mqdqb40000gp/T/tmp-20946jC8DYWY6C8bR.xml 
=========================== 2 passed in 0.01 seconds ===========================

Output from Console under the Developer Tools panel: I couldn't find relevant information here.

[michaelmalonenz]

I can confirm I am also seeing the behaviour after the pytest 4.1.0 upgrade. It worked with pytest 4.0.1

[DonJayamanne]

Previous versions would contain quotes around file names:

<Module 'test_foo.py'>
  <Function 'test_a'>
  <Function 'test_b'>

New version of PyTest doesn't.
Hence the code used to parse the output fails.

[drorata]

I can confirm that downgrading pytest to 4.0.1 solves the problem. It seems, if I get it right, that the cause on pytest's side is:

https://github.com/pytest-dev/pytest/blob/a4c426b1a891a22ae1b63d0a0fa2dcdf690e69db/src/_pytest/terminal.py#L614-L617

[DonJayamanne]

The solution is fairly simple:

• Modify test discovery to use python code directly to discover tests
• Pass the pytest args to the python code.
• Use python code as follows:

import pytest
class MyPlugin(object):
    def pytest_collection_modifyitems(sessino, config, items):
        print(items, "items from collected list")
    def pytest_collection_finish(self, session):
        print(list(session.items), "items from session")

pytest.main(["-v", "tests", "--collect-only"], plugins=[MyPlugin()])

• Should be a simple enough change.