Pytest performs collection under cwd rather than under specified rootdir

[LewisGaul]

When specifying --rootdir (or using -c to give a config file that might implicitly be used to determine the root dir?), pytest still collects tests under the current working directory, rather than under the specified root.

This means that at least one of the following conditions must be met to avoid pytest searching in the wrong place:

• Must specify the dir to search under, i.e. pytest <path_to_dir>
• Must ensure cwd corresponds to the path intended to be searched

Say we have a test package structured as follows:

/some/path/to/tests/
|-- __init__.py
|-- conftest.py
|-- pytest.ini
|-- sub1/
|   |-- __init__.py
|   `-- test_1.py
`-- sub2/
    |-- __init__.py
    `-- test_2.py

Then from any cwd it's possible to run pytest /some/path/to/tests/ to run all the tests, or pytest /some/path/to/tests/sub1/ to run a specific subpackage. The problem is that there's no way to run from another cwd without specifying <path_to_dir>, e.g. it cannot be specified as pytest --rootdir=/some/path/to/tests/ since pytest will then search cwd.

The problem with this is it makes it hard to write a convenience wrapper script supporting run-my-tests [<file_or_dir>] [<other_pytest_args>], where if no args are passed it should run all the tests, or if <file_or_dir> is given as a path relative to cwd then only those specified tests will be run. That is, I want to support the following:

1. Run all tests (not specifying <file_or_path>) with run-my-tests <args> -> pytest /some/path/to/tests/ <args>
2. (e.g. with cwd being /some/path/to/) Run a specified subset of tests with run-my-tests tests/sub1/ <args> -> pytest tests/sub1/ <args>

Given point 2 supporting relative paths, the wrapper cannot change cwd. However, under point 1, the wrapper script must pass the test root dir, and if this dir is always passed then all the tests will always be run (ignoring sub-paths that may be passed as in point 2). There is no general way for the wrapper script to know whether a path has been given (checking args to see if they look like a path is not foolproof unless reimplementing full parsing of pytest args, which is not even possible in the case of arbitrary plugins adding args).

The request is for pytest to only search for tests under the rootdir, i.e. the wrapper described above could be implemented as simply adding --rootdir=/some/path/to/tests/, which would satisfy the two cases above.

[RonnyPfannschmidt]

rootdir is only used to determine test nodeids (relative to rootdir)

its not informing a default set of tests

the testpaths option sets the default seach path

[LewisGaul]

Yes this corresponds to my description of the behaviour I'm seeing. Is there any reason it makes more sense for rootdir to have no impact on where collection takes place and for it to only be done via testpaths (positional args on the CLI) and cwd?

Do you have any suggestions for a solution to the problem I described with implementing a wrapper script?

[RonnyPfannschmidt]

You can pass testpaths in via the cli using option overrides

[LewisGaul]

Ahh so -o testpaths=/some/path/to/tests/ rather than --rootdir=/some/path/to/tests/ - that seems to do the job, thanks!

[LewisGaul]

Actually this doesn't quite work - it seems the pytest_addoption() hook is not called early enough in this case.

If I add the following in /some/path/to/tests/conftest.py:

def pytest_addoption(parser: pytest.Parser):
    parser.addoption("--foo", action="store_true", help="foo")

then I get:

$pytest -o testpaths=/some/path/to/tests/ -c /some/path/to/tests/pytest.ini --foo
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --foo
  inifile: /some/path/to/tests/pytest.ini
  rootdir: /some/path/to/tests

whereas this works if I also pass e.g. /some/path/to/tests/sub1/ on the end.

[RonnyPfannschmidt]

pytest-addoption can only be used in the conftests directly under the rootdit/config dirs (as pytest will only look at the root ones before parsing ci args

[LewisGaul]

Hmm but in the example above the rootdir is /some/path/to/tests/, and pytest_addoption() is in the conftest.py within that rootdir (not in a subpackage).

This works:
pytest /some/path/to/tests/ [-o testpaths=/some/path/to/tests/] [-c /some/path/to/tests/pytest.ini] --foo

This does not work:
pytest -o testpaths=/some/path/to/tests/ [-c /some/path/to/tests/pytest.ini] --foo

Even though both cases have the same rootdir.

[kdeldycke]

I can confirm the issue @LewisGaul is reporting. I think this issue should be re-opened.

[RonnyPfannschmidt]

the title needs to change tho, but im not sure abou what would be best

[bluetech]

Closing as a duplicate of #9311.