Attempt to clarify the confusion regarding __init__ files and unique test names

doc/en/goodpractices.rst

@@ -30,68 +30,106 @@ Within Python modules, ``pytest`` also discovers tests using the standard
 
 
 Choosing a test layout / import rules
-------------------------------------------
+-------------------------------------
 
 ``pytest`` supports two common test layouts:
 
-* putting tests into an extra directory outside your actual application
-  code, useful if you have many functional tests or for other reasons
-  want to keep tests separate from actual application code (often a good
-  idea)::
+Tests outside application code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-    setup.py   # your setuptools Python package metadata
+Putting tests into an extra directory outside your actual application code
+might be useful if you have many functional tests or for other reasons want
+to keep tests separate from actual application code (often a good idea)::
+
+    setup.py
     mypkg/
         __init__.py
-        appmodule.py
+        app.py
+        view.py
     tests/
         test_app.py
+        test_view.py
         ...
 
+This way your tests can run easily against an installed version
+of ``mypkg``.
+
+Note that using this scheme your test files must have **unique names**, because
+``pytest`` will import them as *top-level* modules since there are no packages
+to derive a full package name from. In other words, the test files in the example above will
+be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to
+``sys.path``.
 
-* inlining test directories into your application package, useful if you
-  have direct relation between (unit-)test and application modules and
-  want to distribute your tests along with your application::
+If you need to have test modules with the same name, you might add ``__init__.py`` files to your
+``tests`` folder and subfolders, changing them to packages::
 
-    setup.py   # your setuptools Python package metadata
+    setup.py
     mypkg/
-        __init__.py
-        appmodule.py
         ...
-        test/
-            test_app.py
-            ...
+    tests/
+        __init__.py
+        foo/
+            __init__.py
+            test_view.py
+        bar/
+            __init__.py
+            test_view.py
+
+Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``, allowing
+you to have modules with the same name. But now this introduces a subtle problem: in order to load
+the test modules from the ``tests`` directory, pytest prepends the root of the repository to
+``sys.path``, which adds the side-effect that now ``mypkg`` is also importable.
+This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment,
+because you want to test the *installed* version of your package, not the local code from the repository.
+
+In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
+sub-directory of your root::
+
+    setup.py
+    src/
+        mypkg/
+            __init__.py
+            app.py
+            view.py
+    tests/
+        __init__.py
+        foo/
+            __init__.py
+            test_view.py
+        bar/
+            __init__.py
+            test_view.py
 
-Important notes relating to both schemes:
 
-- **make sure that "mypkg" is importable**, for example by typing once::
+This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent
+`blog post by Ionel Cristian Mărieș <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`_.
 
-     pip install -e .   # install package using setup.py in editable mode
+Tests as part of application code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-- **avoid "__init__.py" files in your test directories**.
-  This way your tests can run easily against an installed version
-  of ``mypkg``, independently from the installed package if it contains
-  the tests or not.
+Inlining test directories into your application package
+is useful if you have direct relation between tests and application modules and
+want to distribute them along with your application::
 
-- With inlined tests you might put ``__init__.py`` into test
-  directories and make them installable as part of your application.
-  Using the ``pytest --pyargs mypkg`` invocation pytest will
-  discover where mypkg is installed and collect tests from there.
-  With the "external" test you can still distribute tests but they
-  will not be installed or become importable.
+    setup.py
+    mypkg/
+        __init__.py
+        app.py
+        view.py
+        test/
+            __init__.py
+            test_app.py
+            test_view.py
+            ...
+
+In this scheme, it is easy to your run tests using the ``--pyargs`` option::
 
-Typically you can run tests by pointing to test directories or modules::
+    pytest --pyargs mypkg
 
-    pytest tests/test_app.py       # for external test dirs
-    pytest mypkg/test/test_app.py  # for inlined test dirs
-    pytest mypkg                   # run tests in all below test directories
-    pytest                         # run all tests below current dir
-    ...
+``pytest`` will discover where ``mypkg`` is installed and collect tests from there.
+
+Note that this layout also works in conjunction with the ``src`` layout mentioned in the previous section.
 
-Because of the above ``editable install`` mode you can change your
-source code (both tests and the app) and rerun tests at will.
-Once you are done with your work, you can `use tox`_ to make sure
-that the package is really correct and tests pass in all
-required configurations.
 
 .. note::
 
@@ -144,7 +182,13 @@ for installing your application and any dependencies
 as well as the ``pytest`` package itself. This ensures your code and
 dependencies are isolated from the system Python installation.
 
-If you frequently release code and want to make sure that your actual
+You can then install your package in "editable" mode::
+
+     pip install -e .
+
+which lets you change your source code (both tests and application) and rerun tests at will.
+
+Once you are done with your work and want to make sure that your actual
 package passes all tests you may want to look into `tox`_, the
 virtualenv test automation tool and its `pytest support
 <https://tox.readthedocs.io/en/latest/example/pytest.html>`_.
@@ -154,11 +198,6 @@ options.  It will run tests against the installed package and not
 against your source code checkout, helping to detect packaging
 glitches.
 
-Continuous integration services such as Jenkins_ can make use of the
-``--junitxml=PATH`` option to create a JUnitXML file and generate reports (e.g.
-by publishing the results in a nice format with the `Jenkins xUnit Plugin
-<https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin>`_).
-
 
 Integrating with setuptools / ``python setup.py test`` / ``pytest-runner``
 --------------------------------------------------------------------------

tox.ini

@@ -106,6 +106,8 @@ commands=
   pytest -ra {posargs:testing/test_unittest.py}
 
 [testenv:docs]
+skipsdist=True
+usedevelop=True
 basepython=python
 changedir=doc/en
 deps=