Test execution randomly ends prematurely

[pewscorner]

Environment data

• VS Code version: 1.60.0
• Extension version (available under the Extensions sidebar): v2021.9.1191016588
• OS and version: Windows 10
• Python version (& distribution if applicable, e.g. Anaconda): 3.9.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: N/A
• Relevant/affected Python-related VS Code extensions and their versions: Pylance v2021.9.0
• Value of the python.languageServer setting: Pylance

Expected behaviour

When clicking on the "Run Tests" button, all discovered tests should run.

Actual behaviour

A random number of tests are run (sometimes all of them). There is nothing in the UI indicating why the testing stopped (but there's an error in the Python output log, see below). This behavior started after upgrading VS Code to 1.60.0.

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

I simply run my 64-test test suite.

Logs

Test output terminal:

Running tests (unittest): d:\Documents\PythonProjects\PythonQuickRefTest\unittests
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_about_classes_types_and_objects
./classes_unittest.py::TestClasses::test_about_classes_types_and_objects Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_class_creation_and_instantiation
./classes_unittest.py::TestClasses::test_class_creation_and_instantiation Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_class_inheritance
./classes_unittest.py::TestClasses::test_class_inheritance Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_descriptors
./classes_unittest.py::TestClasses::test_descriptors Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_property_attributes
Total number of tests expected to run: 64
Total number of tests run: 64
Total number of tests passed: 4
Total number of tests failed: 0
Total number of tests failed with errors: 0
Total number of tests skipped: 0
Finished running tests!

Python output

User belongs to experiment group 'pythonaa'
User belongs to experiment group 'pythonJediLSP'
User belongs to experiment group 'pythonSendEntireLineToREPL'
User belongs to experiment group 'pythonNotDisplayLinterPrompt'
User belongs to experiment group 'pythonDiscoveryModuleWithoutWatcher'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'PythonPyTorchProfiler'
Python interpreter path: D:\Python\Python39\python.exe
> pyenv root
> conda --version
> python3.7 -c "import sys;print(sys.executable)"
> python3.6 -c "import sys;print(sys.executable)"
> python3 -c "import sys;print(sys.executable)"
> python2 -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> py -3.7 -c "import sys;print(sys.executable)"
> py -3.6 -c "import sys;print(sys.executable)"
> py -3 -c "import sys;print(sys.executable)"
> py -2 -c "import sys;print(sys.executable)"
> D:\Python\Python39\python.exe -c "import sys;print(sys.executable)"
> conda info --json
Error 2021-09-04 19:21:02: Detection of Python Interpreter for Command python3 and args  failed as file  does not exist
Error 2021-09-04 19:21:03: Failed to get interpreter information for 'D:\Python\Python25\python.exe' [Error: Command failed: "D:\\Python\Python25\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Traceback (most recent call last):
  File "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py", line 4, in <module>
    import json
ImportError: No module named json

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)] {
  killed: false,
  code: 1,
  signal: null,
  cmd: '"D:\\\\Python\\Python25\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:03: Failed to get interpreter information for 'D:\Python\Python24\python.exe' [Error: Command failed: "D:\\Python\Python24\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Traceback (most recent call last):
  File "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py", line 4, in ?
    import json
ImportError: No module named json

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)] {
  killed: false,
  code: 1,
  signal: null,
  cmd: '"D:\\\\Python\\Python24\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Starting Pylance language server.
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python3.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python3.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
> conda --version
> D:\Python\Python39\python.exe -c "
import unittest
import inspect

def get_sourceline(obj):
    s, n = inspect.getsourcelines(obj)
    for i, v in enumerate(s):
        if v.strip().startswith('def'):
            return str(n+i)
    return '*'

def generate_test_cases(suite):
    for test in suite:
        if isinstance(test, unittest.TestCase):
            yield test
        else:
            for test_case in generate_test_cases(test):
                yield test_case

loader = unittest.TestLoader()
suite = loader.discover("./unittests", pattern="*unittest.py")

print("start")  # Don't remove this line
loader_errors = []
for s in generate_test_cases(suite):
    tm = getattr(s, s._testMethodName)
    testId = s.id()
    if testId.startswith("unittest.loader._FailedTest"):
        loader_errors.append(s._exception)
    else:
        print(testId.replace(".", ":") + ":" + get_sourceline(tm))

for error in loader_errors:
    try:
        print("=== exception start ===")
        print(error.msg)
        print("=== exception end ===")
    except:
        pass
"
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe -c "
import unittest
import inspect

def get_sourceline(obj):
    s, n = inspect.getsourcelines(obj)
    for i, v in enumerate(s):
        if v.strip().startswith('def'):
            return str(n+i)
    return '*'

def generate_test_cases(suite):
    for test in suite:
        if isinstance(test, unittest.TestCase):
            yield test
        else:
            for test_case in generate_test_cases(test):
                yield test_case

loader = unittest.TestLoader()
suite = loader.discover("./unittests", pattern="*unittest.py")

print("start")  # Don't remove this line
loader_errors = []
for s in generate_test_cases(suite):
    tm = getattr(s, s._testMethodName)
    testId = s.id()
    if testId.startswith("unittest.loader._FailedTest"):
        loader_errors.append(s._exception)
    else:
        print(testId.replace(".", ":") + ":" + get_sourceline(tm))

for error in loader_errors:
    try:
        print("=== exception start ===")
        print(error.msg)
        print("=== exception end ===")
    except:
        pass
"
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_about_classes_types_and_objects --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_about_classes_types_and_objects --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:21: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_creation_and_instantiation --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_creation_and_instantiation --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:21: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_inheritance --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_inheritance --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:21: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_descriptors --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_descriptors --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:22: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_property_attributes --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_property_attributes --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:22: Error: read ECONNRESET
Error 2021-09-04 19:21:22: undefined

[karthiknadig]

@pewscorner Are you using pytest or unittest?

[pewscorner]

@karthiknadig I'm using unittest.

[Josephkready]

I am also experiencing this after updating to 1.60.0

[pvenderbosch]

Using unittest and have the same issue since the update to 1.60.0

[mrucker]

I'm using untitest and have the same issue.

I have over 300 unit tests in my project. I haven't had it finish all 300+ test in a single pass one time since the update.

[mrucker]

One more small thing, I know this probably isn't the place for it, but I do really really like the look of the new test interface. Thanks for all your hard work!!!