Merge master into features

.travis.yml

@@ -23,6 +23,11 @@ env:
     - TOXENV=py37-pluggymaster PYTEST_NO_COVERAGE=1
     - TOXENV=py37-freeze PYTEST_NO_COVERAGE=1
 
+matrix:
+  allow_failures:
+    - python: '3.8-dev'
+      env: TOXENV=py38
+
 jobs:
   include:
     # Coverage tracking is slow with pypy, skip it.
@@ -35,6 +40,8 @@ jobs:
       python: '3.5'
     - env: TOXENV=py36
       python: '3.6'
+    - env: TOXENV=py38
+      python: '3.8-dev'
     - env: TOXENV=py37
     - &test-macos
       language: generic

AUTHORS

@@ -27,6 +27,7 @@ Anthony Shaw
 Anthony Sottile
 Anton Lodder
 Antony Lee
+Arel Cordero
 Armin Rigo
 Aron Coyle
 Aron Curzon
@@ -173,6 +174,7 @@ Nathaniel Waisbrot
 Ned Batchelder
 Neven Mundar
 Nicholas Devenish
+Nicholas Murphy
 Niclas Olofsson
 Nicolas Delaby
 Oleg Pidsadnyi

CHANGELOG.rst

@@ -24,7 +24,7 @@ pytest 4.2.0 (2019-01-30)
 Features
 --------
 
-- `#3094 <https://github.com/pytest-dev/pytest/issues/3094>`_: `Class xunit-style <https://docs.pytest.org/en/latest/xunit_setup.html>`__ functions and methods
+- `#3094 <https://github.com/pytest-dev/pytest/issues/3094>`_: `Classic xunit-style <https://docs.pytest.org/en/latest/xunit_setup.html>`__ functions and methods
   now obey the scope of *autouse* fixtures.
 
   This fixes a number of surprising issues like ``setup_method`` being called before session-scoped
@@ -96,6 +96,9 @@ Trivial/Internal Changes
 - `#4657 <https://github.com/pytest-dev/pytest/issues/4657>`_: Copy saferepr from pylib
 
 
+- `#4668 <https://github.com/pytest-dev/pytest/issues/4668>`_: The verbose word for expected failures in the teststatus report changes from ``xfail`` to ``XFAIL`` to be consistent with other test outcomes.
+
+
 pytest 4.1.1 (2019-01-12)
 =========================
 

changelog/2895.bugfix.rst

@@ -0,0 +1 @@
+The ``pytest_report_collectionfinish`` hook now is also called with ``--collect-only``.

changelog/3899.bugfix.rst

@@ -0,0 +1 @@
+Do not raise ``UsageError`` when an imported package has a ``pytest_plugins.py`` child module.

changelog/3899.doc.rst

@@ -0,0 +1 @@
+Add note to ``plugins.rst`` that ``pytest_plugins`` should not be used as a name for a user module containing plugins.

changelog/4324.doc.rst

@@ -0,0 +1 @@
+Document how to use ``raises`` and ``does_not_raise`` to write parametrized tests with conditional raises.

changelog/4592.bugfix.rst

@@ -0,0 +1 @@
+Fix handling of ``collect_ignore`` via parent ``conftest.py``.

changelog/4700.bugfix.rst

@@ -0,0 +1,2 @@
+Fix regression where ``setUpClass`` would always be called in subclasses even if all tests
+were skipped by a ``unittest.skip()`` decorator applied in the subclass.

changelog/4709.doc.rst

@@ -0,0 +1,2 @@
+Document how to customize test failure messages when using
+``pytest.warns``.

changelog/4739.bugfix.rst

@@ -0,0 +1 @@
+Fix ``parametrize(... ids=<function>)`` when the function returns non-strings.

doc/en/example/parametrize.rst

@@ -565,3 +565,50 @@ As the result:
 - The test ``test_eval[1+7-8]`` passed, but the name is autogenerated and confusing.
 - The test ``test_eval[basic_2+4]`` passed.
 - The test ``test_eval[basic_6*9]`` was expected to fail and did fail.
+
+.. _`parametrizing_conditional_raising`:
+
+Parametrizing conditional raising
+--------------------------------------------------------------------
+
+Use :func:`pytest.raises` with the
+:ref:`pytest.mark.parametrize ref` decorator to write parametrized tests
+in which some tests raise exceptions and others do not.
+
+It is helpful to define a no-op context manager ``does_not_raise`` to serve
+as a complement to ``raises``. For example::
+
+    from contextlib import contextmanager
+    import pytest
+
+    @contextmanager
+    def does_not_raise():
+        yield
+
+
+    @pytest.mark.parametrize('example_input,expectation', [
+        (3, does_not_raise()),
+        (2, does_not_raise()),
+        (1, does_not_raise()),
+        (0, pytest.raises(ZeroDivisionError)),
+    ])
+    def test_division(example_input, expectation):
+        """Test how much I know division."""
+        with expectation:
+            assert (6 / example_input) is not None
+
+In the example above, the first three test cases should run unexceptionally,
+while the fourth should raise ``ZeroDivisionError``.
+
+If you're only supporting Python 3.7+, you can simply use ``nullcontext``
+to define ``does_not_raise``::
+
+    from contextlib import nullcontext as does_not_raise
+
+Or, if you're supporting Python 3.3+ you can use::
+
+    from contextlib import ExitStack as does_not_raise
+
+Or, if desired, you can ``pip install contextlib2`` and use::
+
+    from contextlib2 import ExitStack as does_not_raise

doc/en/plugins.rst

@@ -84,6 +84,11 @@ will be loaded as well.
     :ref:`full explanation <requiring plugins in non-root conftests>`
     in the Writing plugins section.
 
+.. note::
+   The name ``pytest_plugins`` is reserved and should not be used as a
+   name for a custom plugin module.
+
+
 .. _`findpluginname`:
 
 Finding out which plugins are active

doc/en/warnings.rst

@@ -233,7 +233,7 @@ You can also use it as a contextmanager::
 .. _warns:
 
 Asserting warnings with the warns function
------------------------------------------------
+------------------------------------------
 
 .. versionadded:: 2.8
 
@@ -291,7 +291,7 @@ Alternatively, you can examine raised warnings in detail using the
 .. _recwarn:
 
 Recording warnings
-------------------------
+------------------
 
 You can record raised warnings either using ``pytest.warns`` or with
 the ``recwarn`` fixture.
@@ -329,6 +329,26 @@ warnings, or index into it to get a particular recorded warning.
 
 Full API: :class:`WarningsRecorder`.
 
+.. _custom_failure_messages:
+
+Custom failure messages
+-----------------------
+
+Recording warnings provides an opportunity to produce custom test
+failure messages for when no warnings are issued or other conditions
+are met.
+
+.. code-block:: python
+
+    def test():
+        with pytest.warns(Warning) as record:
+            f()
+            if not record:
+                pytest.fail("Expected a warning!")
+
+If no warnings are issued when calling ``f``, then ``not record`` will
+evaluate to ``True``.  You can then call ``pytest.fail`` with a
+custom error message.
 
 .. _internal-warnings:
 

pyproject.toml

@@ -5,6 +5,7 @@ requires = [
   "setuptools-scm",
   "wheel",
 ]
+build-backend = "setuptools.build_meta"
 
 [tool.towncrier]
 package = "pytest"

src/_pytest/config/__init__.py

@@ -559,8 +559,8 @@ def _get_plugin_specs_as_list(specs):
     which case it is returned as a list. Specs can also be `None` in which case an
     empty list is returned.
     """
-    if specs is not None:
-        if isinstance(specs, str):
+    if specs is not None and not isinstance(specs, types.ModuleType):
+        if isinstance(specs, six.string_types):
             specs = specs.split(",") if specs else []
         if not isinstance(specs, (list, tuple)):
             raise UsageError(

src/_pytest/logging.py

@@ -370,6 +370,8 @@ def get_actual_log_level(config, *setting_names):
         )
 
 
+# run after terminalreporter/capturemanager are configured
+@pytest.hookimpl(trylast=True)
 def pytest_configure(config):
     config.pluginmanager.register(LoggingPlugin(config), "logging-plugin")
 
@@ -388,8 +390,6 @@ def __init__(self, config):
 
         # enable verbose output automatically if live logging is enabled
         if self._log_cli_enabled() and not config.getoption("verbose"):
-            # sanity check: terminal reporter should not have been loaded at this point
-            assert self._config.pluginmanager.get_plugin("terminalreporter") is None
             config.option.verbose = 1
 
         self.print_logs = get_option_ini(config, "log_print")
@@ -420,6 +420,54 @@ def __init__(self, config):
 
         self.log_cli_handler = None
 
+        self.live_logs_context = lambda: dummy_context_manager()
+        # Note that the lambda for the live_logs_context is needed because
+        # live_logs_context can otherwise not be entered multiple times due
+        # to limitations of contextlib.contextmanager.
+
+        if self._log_cli_enabled():
+            self._setup_cli_logging()
+
+    def _setup_cli_logging(self):
+        config = self._config
+        terminal_reporter = config.pluginmanager.get_plugin("terminalreporter")
+        if terminal_reporter is None:
+            # terminal reporter is disabled e.g. by pytest-xdist.
+            return
+
+        # FIXME don't set verbosity level and derived attributes of
+        # terminalwriter directly
+        terminal_reporter.verbosity = config.option.verbose
+        terminal_reporter.showheader = terminal_reporter.verbosity >= 0
+        terminal_reporter.showfspath = terminal_reporter.verbosity >= 0
+        terminal_reporter.showlongtestinfo = terminal_reporter.verbosity > 0
+
+        capture_manager = config.pluginmanager.get_plugin("capturemanager")
+        # if capturemanager plugin is disabled, live logging still works.
+        log_cli_handler = _LiveLoggingStreamHandler(terminal_reporter, capture_manager)
+        log_cli_format = get_option_ini(config, "log_cli_format", "log_format")
+        log_cli_date_format = get_option_ini(
+            config, "log_cli_date_format", "log_date_format"
+        )
+        if (
+            config.option.color != "no"
+            and ColoredLevelFormatter.LEVELNAME_FMT_REGEX.search(log_cli_format)
+        ):
+            log_cli_formatter = ColoredLevelFormatter(
+                create_terminal_writer(config),
+                log_cli_format,
+                datefmt=log_cli_date_format,
+            )
+        else:
+            log_cli_formatter = logging.Formatter(
+                log_cli_format, datefmt=log_cli_date_format
+            )
+        log_cli_level = get_actual_log_level(config, "log_cli_level", "log_level")
+        self.log_cli_handler = log_cli_handler
+        self.live_logs_context = lambda: catching_logs(
+            log_cli_handler, formatter=log_cli_formatter, level=log_cli_level
+        )
+
     def _log_cli_enabled(self):
         """Return True if log_cli should be considered enabled, either explicitly
         or because --log-cli-level was given in the command-line.
@@ -430,10 +478,6 @@ def _log_cli_enabled(self):
 
     @pytest.hookimpl(hookwrapper=True, tryfirst=True)
     def pytest_collection(self):
-        # This has to be called before the first log message is logged,
-        # so we can access the terminal reporter plugin.
-        self._setup_cli_logging()
-
         with self.live_logs_context():
             if self.log_cli_handler:
                 self.log_cli_handler.set_when("collection")
@@ -513,7 +557,6 @@ def pytest_sessionfinish(self):
 
     @pytest.hookimpl(hookwrapper=True, tryfirst=True)
     def pytest_sessionstart(self):
-        self._setup_cli_logging()
         with self.live_logs_context():
             if self.log_cli_handler:
                 self.log_cli_handler.set_when("sessionstart")
@@ -533,46 +576,6 @@ def pytest_runtestloop(self, session):
             else:
                 yield  # run all the tests
 
-    def _setup_cli_logging(self):
-        """Sets up the handler and logger for the Live Logs feature, if enabled."""
-        terminal_reporter = self._config.pluginmanager.get_plugin("terminalreporter")
-        if self._log_cli_enabled() and terminal_reporter is not None:
-            capture_manager = self._config.pluginmanager.get_plugin("capturemanager")
-            log_cli_handler = _LiveLoggingStreamHandler(
-                terminal_reporter, capture_manager
-            )
-            log_cli_format = get_option_ini(
-                self._config, "log_cli_format", "log_format"
-            )
-            log_cli_date_format = get_option_ini(
-                self._config, "log_cli_date_format", "log_date_format"
-            )
-            if (
-                self._config.option.color != "no"
-                and ColoredLevelFormatter.LEVELNAME_FMT_REGEX.search(log_cli_format)
-            ):
-                log_cli_formatter = ColoredLevelFormatter(
-                    create_terminal_writer(self._config),
-                    log_cli_format,
-                    datefmt=log_cli_date_format,
-                )
-            else:
-                log_cli_formatter = logging.Formatter(
-                    log_cli_format, datefmt=log_cli_date_format
-                )
-            log_cli_level = get_actual_log_level(
-                self._config, "log_cli_level", "log_level"
-            )
-            self.log_cli_handler = log_cli_handler
-            self.live_logs_context = lambda: catching_logs(
-                log_cli_handler, formatter=log_cli_formatter, level=log_cli_level
-            )
-        else:
-            self.live_logs_context = lambda: dummy_context_manager()
-        # Note that the lambda for the live_logs_context is needed because
-        # live_logs_context can otherwise not be entered multiple times due
-        # to limitations of contextlib.contextmanager
-
 
 class _LiveLoggingStreamHandler(logging.StreamHandler):
     """

src/_pytest/main.py

@@ -607,6 +607,7 @@ def filter_(f):
                 yield y
 
     def _collectfile(self, path, handle_dupes=True):
+        assert path.isfile()
         ihook = self.gethookproxy(path)
         if not self.isinitpath(path):
             if ihook.pytest_ignore_collect(path=path, config=self.config):

src/_pytest/python.py

@@ -599,6 +599,7 @@ def gethookproxy(self, fspath):
         return proxy
 
     def _collectfile(self, path, handle_dupes=True):
+        assert path.isfile()
         ihook = self.gethookproxy(path)
         if not self.isinitpath(path):
             if ihook.pytest_ignore_collect(path=path, config=self.config):
@@ -642,11 +643,12 @@ def collect(self):
             ):
                 continue
 
-            if path.isdir() and path.join("__init__.py").check(file=1):
-                pkg_prefixes.add(path)
-
-            for x in self._collectfile(path):
-                yield x
+            if path.isdir():
+                if path.join("__init__.py").check(file=1):
+                    pkg_prefixes.add(path)
+            else:
+                for x in self._collectfile(path):
+                    yield x
 
 
 def _get_xunit_setup_teardown(holder, attr_name, param_obj=None):
@@ -1144,20 +1146,18 @@ def _find_parametrized_scope(argnames, arg2fixturedefs, indirect):
 
 def _idval(val, argname, idx, idfn, item, config):
     if idfn:
-        s = None
         try:
-            s = idfn(val)
+            generated_id = idfn(val)
+            if generated_id is not None:
+                val = generated_id
         except Exception as e:
             # See issue https://github.com/pytest-dev/pytest/issues/2169
             msg = "{}: error raised while trying to determine id of parameter '{}' at position {}\n"
             msg = msg.format(item.nodeid, argname, idx)
             # we only append the exception type and message because on Python 2 reraise does nothing
             msg += "  {}: {}\n".format(type(e).__name__, e)
             six.raise_from(ValueError(msg), e)
-        if s:
-            return ascii_escaped(s)
-
-    if config:
+    elif config:
         hook_id = config.hook.pytest_make_parametrize_id(
             config=config, val=val, argname=argname
         )