Ensure COV_CORE_SRC is an absolute path before exporting to the environment

[ctheune]

Summary

When COV_CORE_SRC is a relative directory and a subprocess first changes its working directory
before invoking Python then coverage won't associate the

Expected vs actual result

Get proper coverage reporting, but coverage is not reported properly.

Reproducer

• specifiy the test directory with a relative path, i.e. bin/py.test src
• Create wrap a subprocess call in a shell script that first changes its work directory before calling bin/python src/something.py

Versions

Output of relevant packages pip list, python --version, pytest --version etc.

Python 3.8.5
pytest 6.1.2
pytest-asyncio==0.14.0
pytest-cache==1.0
pytest-cov==2.11.1
pytest-flake8==1.0.6
pytest-timeout==1.4.2

Config

Include your tox.ini, pytest.ini, .coveragerc, setup.cfg or any relevant configuration.

[run]
branch = True

[pytest]
addopts = --timeout=30 --tb=native --cov=src --cov-report=html src -r w
markers = slow: This is a non-unit test and thus is not run by default. Use ``-m slow`` to run these, or ``-m 1`` to run all tests.
log_level = NOTSET


filterwarnings =
    ignore::DeprecationWarning:telnetlib3.*:

Code

See https://github.com/flyingcircusio/backy/blob/master/src/backy/tests/test_backy.py#L99

I'm currently working around this by explicitly making COV_CORE_SRC absolute before calling the subprocess. I guess this could/should be done in general, too.

    os.environ['COV_CORE_SOURCE'] = os.path.abspath(
        os.environ['COV_CORE_SOURCE'])

[masenf]

Have you tried changing --cov=src to --cov=backy?

In my experiements, I found that subprocess coverage reporting works better when specifying source packages instead of filesystem paths.

In my case, this wasn't easy using a "flat" repo structure where the package directory (relative to repo root) and the package name are the same string. coverage.py seems to prefer interpreting blah as a filesystem path (if it exists), even if blah might be a package name.

A solution is to explicitly use source_pkgs = in the [coverage:run] section of the config file, and then simply pass --cov to pytest

[jfly]

Have you tried changing --cov=src to --cov=backy?

In my experiements, I found that subprocess coverage reporting works better when specifying source packages instead of filesystem paths.

I personally haven't had issues using filesystem paths for reporting. I actually prefer it, because it's one less place where I have my package name specified (which can get out of a sync if I rename a project, or copy stuff to a new project with a different name).

I'm currently working around this by explicitly making COV_CORE_SRC absolute before calling the subprocess. I guess this could/should be done in general, too.

    os.environ['COV_CORE_SOURCE'] = os.path.abspath(
        os.environ['COV_CORE_SOURCE'])

For the record, this implementation doesn't work if you are using package names rather than source dirs, because this will mange the package name into an absolute path.

The robust fix would be to replicate this isdir ternary logic from coveragepy itself: https://github.com/nedbat/coveragepy/blob/7.7.1/coverage/inorout.py#L198-L201, and only make absolute paths out of source elements that are themselves directories.

IMO, a cleaner fix would be for coveragepy to have an explicit configuration knob for source_dirs in addition to source_pkgs. I filed a feature request for that over here: nedbat/coveragepy#1942