Deleted working directory in pytest-cov 2.6.0+

[effigies]

For pytest-cov>=2.6, the following error results at the end of execution:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/_pytest/main.py", line 213, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/_pytest/main.py", line 257, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/coverage/control.py", line 675, in load
INTERNALERROR>     self._init()
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/coverage/control.py", line 223, in _init
INTERNALERROR>     set_relative_directory()
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/coverage/files.py", line 28, in set_relative_directory
INTERNALERROR>     RELATIVE_DIR = os.path.normcase(abs_file(os.curdir) + os.sep)
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/site-packages/coverage/files.py", line 163, in abs_file
INTERNALERROR>     path = os.path.realpath(path)
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/posixpath.py", line 389, in realpath
INTERNALERROR>     return abspath(path)
INTERNALERROR>   File "/usr/local/miniconda/lib/python3.7/posixpath.py", line 376, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

This appears to indicate that somewhere we change directories, then that directory gets deleted, and then os.getcwd() gets called via coverage.coverage(...).stop() before any chdir to an existing directory.

Downgrading to 2.5.1 resolves the problem. For reference, our call looks like:

pytest --junit-xml=/tmp/pytest.xml \
         --cov niworkflows --cov-report xml:/tmp/unittests.xml \
         --ignore=/src/niworkflows/niworkflows/tests/ \
         --ignore=/src/niworkflows/niworkflows/interfaces/ants.py \
         /src/niworkflows/niworkflows

Related: nedbat/coveragepy#750

[blueyed]

Downgrading to 2.5.1 resolves the problem

Can you git-bisect it then maybe?

[effigies]

Took some time but: 6b1e878

[ionelmc]

Oooof. That commit :|

There are to ways here: revert that and figure out a different way to deal with the problem of #130 (perhaps fix it in coveragepy) or make pytest-cov stash and restore cwd at the right time.

[effigies]

make pytest-cov stash and restore cwd at the right time.

I see a couple options, not being well acquainted with the internals of pytest or pytest-cov:

1. Get pytest to stash and restore cwd before handing off to plugins.
2. Stash and restore in pytest-cov
3. Try to get coverage to disregard the cwd, or allow the intended working directory to be passed as an argument.

[ionelmc]

3. Pretty sure coverage can't avoid using abspath, realpath, or whatever.

1. @pytest-dev/core any opinion on making pytest having builtin protection against tests doing silly things with cwd changes? IOW: do some cwd saving/restoring to avoid having plugins blow up then tests mess up the cwd.

Either way, if they agree having that it'd still only be in the pytest 5.x release (py3 only).

[asottile]

It's arguably not a bug for a test to change the working directory

Consider this which I consider well formed:

import pytest


class Test:
    @pytest.fixture(scope='class', autouse=True)
    def in_tmpdir(self, tmpdir_factory):
        d = tmpdir_factory.mktemp('d')
        with d.as_cwd():
            yield d

    def test(self): ...

pytest-cov should be keeping absolute paths on construction and using those throughout

[asottile]

that'll fix things due to the implementation of abspath:

def abspath(path):
    """Return an absolute path."""
    path = os.fspath(path)
    if not isabs(path):
        if isinstance(path, bytes):
            cwd = os.getcwdb()
        else:
            cwd = os.getcwd()
        path = join(cwd, path)
    return normpath(path)

[simonw]

I just ran into this bug while trying to test out Coverage 5.0 beta. Here are my steps to reproduce:

~ $ cd /tmp
/tmp $ git clone git@github.com:simonw/datasette
Cloning into 'datasette'...
/tmp $ cd datasette/
/tmp/datasette $ python3 -m venv venv
/tmp/datasette $ source venv/bin/activate
(venv) /tmp/datasette $ pip install -e .[test]
Obtaining file:///private/tmp/datasette
...
(venv) /tmp/datasette $ pip install coverage==5.0b1 pytest-cov==2.8.1
Collecting coverage==5.0b1
  Downloading https://files.pythonhosted.org/packages/f6/6d
...
(venv) /tmp/datasette $ pytest --cov=datasette
============================================================= test session starts =============================================================
platform darwin -- Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /private/tmp/datasette, inifile: pytest.ini
plugins: asyncio-0.10.0, cov-2.8.1
collected 379 items                                                                                                                           

tests/test_cli.py ...                                                                                                                   [  0%]
...
tests/test_utils.py ......................................................................                                              [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/_pytest/main.py", line 193, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/_pytest/main.py", line 237, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/pytest_cov/plugin.py", line 254, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/pytest_cov/engine.py", line 197, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/control.py", line 595, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/control.py", line 649, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/sqldata.py", line 436, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/sqldata.py", line 493, in _choose_lines_or_arcs
INTERNALERROR>     with self._connect() as con:
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/sqldata.py", line 298, in _connect
INTERNALERROR>     self._create_db()
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/sqldata.py", line 247, in _create_db
INTERNALERROR>     with db:
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/sqldata.py", line 1008, in __enter__
INTERNALERROR>     self._connect()
INTERNALERROR>   File "/private/tmp/datasette/venv/lib/python3.7/site-packages/coverage/sqldata.py", line 982, in _connect
INTERNALERROR>     filename = os.path.relpath(self.filename)
INTERNALERROR>   File "/usr/local/bin/../Cellar/python/3.7.4_1/bin/../Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py", line 475, in relpath
INTERNALERROR>     start_list = [x for x in abspath(start).split(sep) if x]
INTERNALERROR>   File "/usr/local/bin/../Cellar/python/3.7.4_1/bin/../Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py", line 383, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

[nedbat]

I can confirm that the datasette instructions do reproduce the problem.