Test directory structure and import problems

[DevOpsCraftsman]

Hi!

I don't know if problems with the documentation are to be posted here...

In the Test directory structure page:

Typically you can run tests by pointing to test directories or modules:

pytest tests/test_appmodule.py      # for external test dirs
pytest src/tests/test_appmodule.py  # for inlined test dirs
pytest src                          # run tests in all below test directories
pytest                              # run all tests below current dir

These are not working out of the box because of the path problem.
You will have something like:

tests/test_appmodule.py:1: in <module>
    from appmodule import <whatever>
E   ModuleNotFoundError: No module named 'appmodule'

I've wrote a question on stackoverflow here.

[asottile]

I left a comment

there's a bunch of dupes about relative imports on so:

• https://stackoverflow.com/q/279237/812183
• https://stackoverflow.com/q/14132789/812183

(and a few others, didn't have time to track them all down)

hope this helps!

[DevOpsCraftsman]

This helps to understand more the problem, thanks.
But this doesn't help about the issue opened here: the documentation is incomplete and misleading for what I see...

I can't understand how people deal with this and why nobody complained about this earlier. I'm pretty sure I'm missing something.

[asottile]

The mistake wasn't the directory layout but that you were using a relative import which isn't a thing in python3

Is there a code sample in our docs which uses a relative import like yours?

[DevOpsCraftsman]

I don't understand quite well... How is this supposed to work then?

You said:

your __init__.py you should have from .funcs import f -- you are making a package-relative import
oh also your test file should have from my_folder.funcs import f -- $ python -m pytest -q my_folder ==> 1 passed in 0.01 seconds

How can someone understand that from the doc?

[asottile]

the mistakes you made have nothing to do with pytest. that's just how you make modules and import statements in python.

the docs aren't intended to teach you how to write import statements (they don't show any code at all so they couldn't have misled you here)

[DevOpsCraftsman]

I don't agree:

pytest supports two common test layouts:
[...]

and

Typically you can run tests by pointing to test directories or modules:
pytest tests/test_appmodule.py
[...]

It's obvious that you have to give at least a simple example if you don't want people to think that imports will be handled nicely out of the box without extra painful work. I don't call this support...

[asottile]

here's both layouts working fine. There's no pain from imports, this is how any library would be arranged. The imports inside the tests are exactly as they would be if one were a consumer of the library.

inside package

$ tree my_pkg/
my_pkg/
├── foo.py
├── __init__.py
└── tests
    └── foo_test.py

1 directory, 3 files

$ find my_pkg/ -type f | xargs tail -n999
==> my_pkg/foo.py <==
def square(x: int) -> int:
    return x * x

==> my_pkg/__init__.py <==

==> my_pkg/tests/foo_test.py <==
from my_pkg.foo import square


def test_square():
    assert square(2) == 4
    assert square(16) == 256

$ python -m pytest my_pkg/
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /tmp/t/in-mod, inifile:
collected 1 item                                                               

my_pkg/tests/foo_test.py .                                               [100%]

=========================== 1 passed in 0.03 seconds ===========================

outside package

(adopted from the top by mv my_pkg/tests .)

$ tree my_pkg/ tests
my_pkg/
├── foo.py
└── __init__.py
tests
└── foo_test.py

0 directories, 3 files

$ find my_pkg/ tests -type f | xargs tail -n999
==> my_pkg/foo.py <==
def square(x: int) -> int:
    return x * x

==> my_pkg/__init__.py <==

==> tests/foo_test.py <==
from my_pkg.foo import square


def test_square():
    assert square(2) == 4
    assert square(16) == 256

$ python -m pytest tests
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /tmp/t/in-mod, inifile:
collected 1 item                                                               

tests/foo_test.py .                                                      [100%]

=========================== 1 passed in 0.01 seconds ===========================

[DevOpsCraftsman]

Works with python -m pytest but not with pytest:

python -m pytest my_pkg/tests
====================================== test session starts ======================================
platform linux -- Python 3.7.0, pytest-3.8.1, py-1.6.0, pluggy-0.7.1
rootdir: /home/yahya/Tmp/my_project, inifile:
plugins: xdist-1.23.2, forked-0.2
collected 1 item                                                                                

my_pkg/tests/foo_test.py .                                                                [100%]

=================================== 1 passed in 0.01 seconds ====================================

pytest my_pkg/tests
====================================== test session starts ======================================
platform linux -- Python 3.7.0, pytest-3.8.1, py-1.6.0, pluggy-0.7.1
rootdir: /home/yahya/Tmp/my_project, inifile:
plugins: xdist-1.23.2, forked-0.2
collected 0 items / 1 errors                                                                    

============================================ ERRORS =============================================
___________________________ ERROR collecting my_pkg/tests/foo_test.py ___________________________
ImportError while importing test module '/home/yahya/Tmp/my_project/my_pkg/tests/foo_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
my_pkg/tests/foo_test.py:1: in <module>
    from my_pkg.foo import square
E   ModuleNotFoundError: No module named 'my_pkg'
!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 1 error in 0.07 seconds ====================================

Why?
So there seems to be a problem with the doc: it should recommend python -m pytest.

[asottile]

ah the docs assume you're actually installed (notice how I left out setup.py and the installation setup)

[DevOpsCraftsman]

I don't know what do you mean by ' installed', could you please elaborate?

[asottile]

$ cat setup.py 
from setuptools import find_packages
from setuptools import setup

setup(
    name='my-mod',
    version='0',
    packages=find_packages('.', exclude=('tests*',)),
)

$ pip install .
Processing /tmp/t/in-mod
Building wheels for collected packages: my-mod
  Running setup.py bdist_wheel for my-mod ... done
  Stored in directory: /home/asottile/.cache/pip/wheels/66/fd/40/fa5c6e075909ac6d0bd0d099b5cbdf71f98904a16b95b73c4e
Successfully built my-mod
Installing collected packages: my-mod
Successfully installed my-mod-0

$ pytest my_pkg/
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /tmp/t/in-mod, inifile:
collected 1 item                                                               

my_pkg/tests/foo_test.py .                                               [100%]

=========================== 1 passed in 0.01 seconds ===========================

[DevOpsCraftsman]

Thanks for your explanations!

[asottile]

also I'm not sure how you found that docs page, it's from a two years old feature branch 😆

[asottile]

@The-Compiler any opposition to me deleting that branch (since it's causing confusion / isn't maintained)?

[RonnyPfannschmidt]

@asottile this was started as part of the doc reorg from the last pytest sprint - we might want to coordinate with @obestwalter and @pfctdayelise wrt the details (i recall some of those changes being a much needed enhancement, it would be sad if personal time constraints would cause them to be left over/ended

[asottile]

@RonnyPfannschmidt I think you're referring to this other branch which is only ~ a month old

[RonnyPfannschmidt]

@asottile that may be possible as well - i believe @nicoddemus should know then (im a bit out of touch with those branches - i just wanted to ensure we don't loose something we don't want to loose

[nicoddemus]

https://github.com/pytest-dev/pytest/commits/documentation-restructure has been created so @evildmp could work on the restructuring of the docs (tutorial, how-to, reference, explanation), from a Python talk and summarized in this post: https://www.divio.com/blog/documentation

https://github.com/pytest-dev/pytest/commits/reorganize-docs is the branch that @obestwalter, @pfctdayelise, and @hackebrot worked on our sprint. Not sure if they want to keep this branch around or not, given that is probably riddled with conflicts by this point.

[nicoddemus]

Created issues to track what to do with those branches: #4119 and #4120. 👍