PyTest 5.2.3 causes no-data-collected with coverage.py

[chrisrink10]

I submitted a PR this morning for a project of mine and found that it was collecting 0% of lines from our coverage.py Tox job which was, at the time, using PyTest 5.2.3. This was occurring on my local computer (a Mac) and on CircleCI in a Debian based Docker container.

I perused my last builds to find I had previously been using PyTest 5.2.1 and that covered lines were being collected fine using 5.2.1 for that version.

I was able to confirm that collection works on 5.2.1 and 5.2.2 by setting my PyTest version locally (in tox.ini) and running tox -e py36,py37,py38 --recreate and each time.

As far as I can tell, this is not related to any changes in coverage.py since that version has been 4.5.4 between all of the various test runs.

This should be trivial to recreate using the linked repo (if you have Pyenv):

git clone https://github.com/coverahealth/dataspec.git
cd dataspec
pyenv install 3.6.6 3.7.4 3.8.0
pyenv local 3.6.6 3.7.4 3.8.0
python -m venv .venv
source .venv/bin/activate
pip install -rrequirements.txt -rrequirements.dev.txt
sed -i '' 's/pytest==5.2.2/pytest==5.2.3/g' tox.ini   # macOS only (?)
# sed 's/pytest==5.2.2/pytest==5.2.3/g' tox.ini
tox -e py36,py37,py38

Each of the test runs should result in something like:

Coverage.py warning: No data was collected. (no-data-collected)

Python Version(s): 3.6, 3.7, 3.8
PyTest Version: 5.2.3
Operating System: Mac OS Mojave (10.14.6) and a Debian Stretch-based Docker image running on CircleCI

Results of pip list: (from the Mac OS virtual environment)

Package            Version   Location
------------------ --------- ----------------------------------
appdirs            1.4.3
atomicwrites       1.3.0
attrs              19.3.0
black              19.3b0
bleach             3.1.0
certifi            2019.9.11
chardet            3.0.4
Click              7.0
dataspec           0.2.0     /Users/chris/Projects/dataspec/src
docutils           0.15.2
filelock           3.0.12
idna               2.8
importlib-metadata 0.23
isort              4.3.21
more-itertools     7.2.0
packaging          19.2
phonenumbers       8.10.21
pip                19.3.1
pkginfo            1.5.0.1
pluggy             0.13.0
py                 1.8.0
Pygments           2.4.2
pyparsing          2.4.2
pytest             5.2.1
python-dateutil    2.8.0
readme-renderer    24.0
requests           2.22.0
requests-toolbelt  0.9.1
setuptools         41.4.0
six                1.12.0
toml               0.10.0
tox                3.14.0
tqdm               4.36.1
twine              2.0.0
urllib3            1.25.6
virtualenv         16.7.6
wcwidth            0.1.7
webencodings       0.5.1
wheel              0.33.6
zipp               0.6.0

[asottile]

can confirm, this is the same root cause as #6194 so let's continue the discussion there 👍

[chrisrink10]

Thanks @asottile !