Debugging unittest tests doesn't work

[DonJayamanne]

From @yRefl3x on March 6, 2017 11:11

Environment data

VS Code version: 1.10.1
Python Extension version: 0.5.9
Python Version: 2.7.5
OS and version: Windows 10

Actual behavior

After clicking on the "Debug Test" lens of a test method, debugging doesn't start. "Running Tests" is displayed at the bottom of the VSCode window indefinitely.

Expected behavior

Debugging starts.

Steps to reproduce:

I have the following folder structure:

Issue
    cool_package
        __init__.py
        cool_module.py
    test
        __init__.py
        test_cool_module.py

cool_module.py

def foo(a, b):
    return a + b

test_cool_module.py

import unittest

from cool_package.cool_module import foo

class TestCoolModule(unittest.TestCase):
    def test_foo(self):
        res = foo(1, 2)
        self.assertEquals(res, 3)

• Set a breakpoint at line res = foo(1, 2)
• Click "Debug Test" in test method code lens

Settings

settings.json:

{
    "python.unitTest.unittestArgs": [
        "-v",
        "-s",
        "test",
        "-p",
        "test_*.py"
    ],
    "python.unitTest.unittestEnabled": true
}

Logs

Output from Python Test Log output panel

READY

Copied from original issue: DonJayamanne/pythonVSCode#797

[DonJayamanne]

From @syagev on March 14, 2017 8:56

Same exact symptoms here.
win10, python ext 0.6.0, VSC 1.10.2, Python 3.5.3

EDIT: "reload window" VSC command seems to solve this every time this happens

[DonJayamanne]

From @yRefl3x on March 14, 2017 9:56

@syagev I just tried reloading the window before debugging as you suggested. The test actually runs, but it doesn't debug. The breakpoint just changes to a grey circle for a short time but execution isn't stopped. The test just runs like when choosing "Run Tests".

[DonJayamanne]

From @alexwhittemore on March 29, 2017 17:14

Confirmed, but I can't try "reload window" to solve it, because "discovering tests" has stopped completing for me.

[DonJayamanne]

From @gpassero on April 1, 2017 14:5

I can debug tests when I run them with the following command line (in the end of the test file script). It may be a temporary solution for you.

if __name__ == '__main__':
    unittest.main()

[DonJayamanne]

From @alexwhittemore on April 1, 2017 15:59

Update: I can get unit tests to discover if I run "discover unit tests," then, while it works, run "run all tests," which causes discovery to alert failure. At this point, I can "run unit tests", which will trigger discovery successfully. At least, I think that's the flow. Will verify back at a computer.

[DonJayamanne]

From @xqliu on April 18, 2017 16:55

@alexwhittemore How did you run "discover unit tests"? method?

I can not find this command on the command panel nor any place ...

Thanks :)

[DonJayamanne]

From @alexwhittemore on April 18, 2017 18:25

In the bottom bar, bottom left, where it usually says "run tests" - I think it'll say "discovery failed" when such is the case, and clicking it will bring up the command palette with an option to discover.

[DonJayamanne]

@alexwhittemore Sorry, please run the command Run All Unittests

[DonJayamanne]

From @VicLiang621 on May 28, 2017 7:38

I have a similar issue where when you click "Debug test" the running tests never stop and breakpoints do not matter. When trying to implement the if __name__ == '__main__': unittest.main(), seems to be a temporary fix.

[DonJayamanne]

From @sdtom on June 19, 2017 15:6

Similar issue, my workaround.. keep clicking "debug test" over and over.. first time gets READY, second time generates an error.. but maybe 1 out of 10 times instead of just getting READY, things turn orange and debugging works fine

[DonJayamanne]

From @MattMorgis on July 22, 2017 0:56

I am having the same issue with both the unittest and pytest options, both generate the same error. If you can point me in the right direction, I can get a fix merged in.

I created a sample project to help identify the issue: https://github.com/MattMorgis/python-tests

Each time I hit Debug Test inline in VS Code, I get this output:

FAILED (errors=1)
Traceback (most recent call last):
  File "/Users/morgism/.vscode/extensions/donjayamanne.python-0.6.9/pythonFiles/PythonTools/visualstudio_py_testlauncher.py", line 343, in <module>
    main()
  File "/Users/morgism/.vscode/extensions/donjayamanne.python-0.6.9/pythonFiles/PythonTools/visualstudio_py_testlauncher.py", line 238, in main
    enable_attach(opts.secret, ('127.0.0.1', getattr(opts, 'port', DEFAULT_PORT)), redirect_output = True)
  File "/Users/morgism/.vscode/extensions/donjayamanne.python-0.6.9/pythonFiles/PythonTools/ptvsd/attach_server.py", line 167, in enable_attach
    server.bind(address)
  File "/Users/morgism/anaconda/envs/py2Env/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 48] Address already in use

I am using a conda environment and have my Python path set both in settings.json and as an ENV variable. pytest is installed there when trying to run that configuration.

[DonJayamanne]

Will look into this over the weekend

[DonJayamanne]

From @samiraguiar on October 18, 2017 1:26

This issue still persists, at least for me on Fedora 26.
Apparently it hangs waiting for a debugger to be attached, since no timeout is passed to wait_for_attach() in visualstudio_py_testlauncher.py (of course, manually setting a timeout just makes it run the tests without debugging).

Maybe this is some kind of concurrency issue?

[DonJayamanne]

@MattMorgis @samiraguiar can you try changing the port to something different such as 3001 or similar in your user or workspace settings (settings.json) as follows:

"python.unitTest.debugPort": 3000

[DonJayamanne]

@yRefl3x how about your self, are you still experiencing the same issue

[DonJayamanne]

@MattMorgis found a problem with your code, assertEqual needs to be qualified by self

class MetamapAPITest(unittest.TestCase):
    def test_one(self):
        self.assertEqual(true, false)

[DonJayamanne]

From @MattMorgis on October 30, 2017 12:50

@DonJayamanne I committed both changes to my project: adding self and changed debug port to 3001.

Looks like it lead to a new error:

There was an error in starting the debug server. 
Error = 
{
  "code":"ECONNREFUSED",
  "errno":"ECONNREFUSED",
  "syscall":"connect",
  "address":"127.0.0.1",
  "port":3001
}

[DonJayamanne]

From @samiraguiar on October 31, 2017 0:46

@DonJayamanne changing the port didn't help, the issue persisted (though no errors similar to that of @MattMorgis happened).

I'm now also getting Error: undefined when running these tests. Here's my configuration for the linked repo:

{
    "python.linting.pylintEnabled": false,
    "python.unitTest.unittestEnabled": true,
    "python.unitTest.debugPort": 3000,
    "python.pythonPath": "/bin/python3",
    "python.unitTest.unittestArgs": [
        "-v",
        "-f",
        "-s",
        "./tests",
        "-p",
        "test_*.py"
    ]
}

Edit: managed to get a stack trace:

visualstudio_py_testlauncher.py:302:main:TypeError: format_exception() missing 3 required positional arguments: 'etype', 'value', and 'tb'

Traceback (most recent call last):
  File "/<user>/.vscode/extensions/donjayamanne.python-0.7.0/pythonFiles/PythonTools/visualstudio_py_testlauncher.py", line 294, in main
    for m in cls._tests:
  File "/usr/lib64/python3.6/unittest/loader.py", line 32, in __getattr__
    return super(_FailedTest, self).__getattr__(name)
AttributeError: 'super' object has no attribute '__getattr__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/<user>/.vscode/extensions/donjayamanne.python-0.7.0/pythonFiles/PythonTools/visualstudio_py_testlauncher.py", line 343, in <module>
    main()
  File "/<user>/.vscode/extensions/donjayamanne.python-0.7.0/pythonFiles/PythonTools/visualstudio_py_testlauncher.py", line 302, in main
    errorMessage = traceback.format_exception()
TypeError: format_exception() missing 3 required positional arguments: 'etype', 'value', and 'tb'

Local variables in innermost frame:
cls: <unittest.loader._FailedTest testMethod=json.test_output>
suite: <unittest.suite.TestSuite tests=[<unittest.loader._FailedTest testMethod=json.test_output>]>
suites: <unittest.suite.TestSuite tests=[<unittest.suite.TestSuite tests=[]>, <unittest.suite.TestSuite tests=[<unittest.loader._FailedTest testMethod=json.test_output>]>, <unittest.suite.TestSuite tests=[]>, <unittest.suite.TestSuite tests=[]>, <unittest.suite.TestSuite tests=[]>]>
loader: <unittest.loader.TestLoader object at 0x7f571fa48cc0>
tests: None
cov: None
_: []
opts: <Values at 0x7f571fa48668: {'secret': None, 'port': None, 'mixed_mode': None, 'tests': ['json.test_output.TestValidJson.test_olevba_analysis'], 'testFile': '/<user>/oletools/tests/json/test_output.py', 'coverage': None, 'result_port': 39325, 'us': './tests', 'up': 'test_*.py', 'ut': None, 'uvInt': 2, 'uf': None, 'uc': None}>
parser: <optparse.OptionParser object at 0x7f57209249e8>
OptionParser: <class 'optparse.OptionParser'>
unittest: <module 'unittest' from '/usr/lib64/python3.6/unittest/__init__.py'>
sys: <module 'sys' (built-in)>
os: <module 'os' from '/usr/lib64/python3.6/os.py'>

Edit 2: sometimes when clicking Run on a test instead of Debug I get a VS Code warning: No tests ran, please check the configuration settings for the tests. It's weird since I clicked on that Run link that floats over the test, so it means it did find a test.

[DonJayamanne]

@samiraguiar Hmm, seems to work at my end in the dev version. Will check the current released version.

[DonJayamanne]

From @yRefl3x on November 2, 2017 11:46

My experience got better, I can debug unittests now, but it will only work on the first or second tries (kind of inconsistent). Meaning I click the code lens (both class and method lenses work here) for debugging the first time, everything works as expected. If I click a second time after stopping debugging, "Running Tests" is displayed at the bottom indefinitely but nothing happens. To be able to debug again, I have to reload the window.
Btw, I'm now on VSCode v1.17.1 and extension v0.7.0

[thomsavage]

I think I'm still seeing this issue. Trying to use the inline "Debug Tests" within a unit test module shows "Running Tests" in the status bar, but nothing happens.

• VSCode 1.20.1 x64
• Python 2018.1.0
• Win 10

[veuncent]

I have the same issue and same versions as @thomsavage lists (except my Python extension = 2018.2.0).

Edit: although I use nose instead of unittest:

{
    "python.unitTest.nosetestsEnabled": true,  
    "python.unitTest.unittestEnabled": false,  
    "python.unitTest.pyTestEnabled": false,  
    "python.unitTest.promptToConfigure": true,  
}

But the same problem persist if I only enable unittestEnabled.

[brettcannon]

@veuncent so you're saying that your example config is fine but it becomes an issue if you set "python.unitTest.unittestEnabled": true" while having "python.unitTest.nosetestsEnabled": true` as well? IOW what is the exact setting you have which consistently triggers the problem?

[veuncent]

@brettcannon No that's not what I mean. Under no circumstances debugging works. I tried having only one of the testing frameworks enabled at a time (as well as having multiple enabled, which obviously shouldn't work).

Clicking Debug Test above a test class or method does nothing, besides showing the Running Tests throbber at the bottom of the window.

[veuncent]

P.s. When I add

if __name__ == "__main__":
    unittest.main()

to the bottom of the file I can use the Start Debugging button in VSC to run the unit tests, but it does not pause at break points or anything.

[DonJayamanne]

@veuncent Please could you create a separate issue with a sample code that we can use to replicate this at our end.
If you can point us to a git repo that would be great.
Please provide the following info in the new issue:

• Python version
• OS version
• Sample code to replicate this issue

[veuncent]

Done: #1009