Release 4.5.0 (#5246)

Release 4.5.0

Author: nicoddemus

AUTHORS

@@ -194,6 +194,7 @@ Paweł Adamczak
 Pedro Algarvio
 Pieter Mulder
 Piotr Banaszkiewicz
+Pulkit Goyal
 Punyashloka Biswal
 Quentin Pradet
 Ralf Schmitt
@@ -211,6 +212,7 @@ Ross Lawley
 Russel Winder
 Ryan Wooden
 Samuel Dion-Girardeau
+Samuel Searles-Bryant
 Samuele Pedroni
 Sankt Petersbug
 Segev Finer

CHANGELOG.rst

@@ -18,6 +18,139 @@ with advance notice in the **Deprecations** section of releases.
 
 .. towncrier release notes start
 
+pytest 4.5.0 (2019-05-11)
+=========================
+
+Features
+--------
+
+- `#4826 <https://github.com/pytest-dev/pytest/issues/4826>`_: A warning is now emitted when unknown marks are used as a decorator.
+  This is often due to a typo, which can lead to silently broken tests.
+
+
+- `#4907 <https://github.com/pytest-dev/pytest/issues/4907>`_: Show XFail reason as part of JUnitXML message field.
+
+
+- `#5013 <https://github.com/pytest-dev/pytest/issues/5013>`_: Messages from crash reports are displayed within test summaries now, truncated to the terminal width.
+
+
+- `#5023 <https://github.com/pytest-dev/pytest/issues/5023>`_: New flag ``--strict-markers`` that triggers an error when unknown markers (e.g. those not registered using the `markers option`_ in the configuration file) are used in the test suite.
+
+  The existing ``--strict`` option has the same behavior currently, but can be augmented in the future for additional checks.
+
+  .. _`markers option`: https://docs.pytest.org/en/latest/reference.html#confval-markers
+
+
+- `#5026 <https://github.com/pytest-dev/pytest/issues/5026>`_: Assertion failure messages for sequences and dicts contain the number of different items now.
+
+
+- `#5034 <https://github.com/pytest-dev/pytest/issues/5034>`_: Improve reporting with ``--lf`` and ``--ff`` (run-last-failure).
+
+
+- `#5035 <https://github.com/pytest-dev/pytest/issues/5035>`_: The ``--cache-show`` option/action accepts an optional glob to show only matching cache entries.
+
+
+- `#5059 <https://github.com/pytest-dev/pytest/issues/5059>`_: Standard input (stdin) can be given to pytester's ``Testdir.run()`` and ``Testdir.popen()``.
+
+
+- `#5068 <https://github.com/pytest-dev/pytest/issues/5068>`_: The ``-r`` option learnt about ``A`` to display all reports (including passed ones) in the short test summary.
+
+
+- `#5108 <https://github.com/pytest-dev/pytest/issues/5108>`_: The short test summary is displayed after passes with output (``-rP``).
+
+
+- `#5172 <https://github.com/pytest-dev/pytest/issues/5172>`_: The ``--last-failed`` (``--lf``) option got smarter and will now skip entire files if all tests
+  of that test file have passed in previous runs, greatly speeding up collection.
+
+
+- `#5177 <https://github.com/pytest-dev/pytest/issues/5177>`_: Introduce new specific warning ``PytestWarning`` subclasses to make it easier to filter warnings based on the class, rather than on the message. The new subclasses are:
+
+
+  * ``PytestAssertRewriteWarning``
+
+  * ``PytestCacheWarning``
+
+  * ``PytestCollectionWarning``
+
+  * ``PytestConfigWarning``
+
+  * ``PytestUnhandledCoroutineWarning``
+
+  * ``PytestUnknownMarkWarning``
+
+
+- `#5202 <https://github.com/pytest-dev/pytest/issues/5202>`_: New ``record_testsuite_property`` session-scoped fixture allows users to log ``<property>`` tags at the ``testsuite``
+  level with the ``junitxml`` plugin.
+
+  The generated XML is compatible with the latest xunit standard, contrary to
+  the properties recorded by ``record_property`` and ``record_xml_attribute``.
+
+
+- `#5214 <https://github.com/pytest-dev/pytest/issues/5214>`_: The default logging format has been changed to improve readability. Here is an
+  example of a previous logging message::
+
+      test_log_cli_enabled_disabled.py    3 CRITICAL critical message logged by test
+
+  This has now become::
+
+      CRITICAL root:test_log_cli_enabled_disabled.py:3 critical message logged by test
+
+  The formatting can be changed through the `log_format <https://docs.pytest.org/en/latest/reference.html#confval-log_format>`__ configuration option.
+
+
+- `#5220 <https://github.com/pytest-dev/pytest/issues/5220>`_: ``--fixtures`` now also shows fixture scope for scopes other than ``"function"``.
+
+
+
+Bug Fixes
+---------
+
+- `#5113 <https://github.com/pytest-dev/pytest/issues/5113>`_: Deselected items from plugins using ``pytest_collect_modifyitems`` as a hookwrapper are correctly reported now.
+
+
+- `#5144 <https://github.com/pytest-dev/pytest/issues/5144>`_: With usage errors ``exitstatus`` is set to ``EXIT_USAGEERROR`` in the ``pytest_sessionfinish`` hook now as expected.
+
+
+- `#5235 <https://github.com/pytest-dev/pytest/issues/5235>`_: ``outcome.exit`` is not used with ``EOF`` in the pdb wrapper anymore, but only with ``quit``.
+
+
+
+Improved Documentation
+----------------------
+
+- `#4935 <https://github.com/pytest-dev/pytest/issues/4935>`_: Expand docs on registering marks and the effect of ``--strict``.
+
+
+
+Trivial/Internal Changes
+------------------------
+
+- `#4942 <https://github.com/pytest-dev/pytest/issues/4942>`_: ``logging.raiseExceptions`` is not set to ``False`` anymore.
+
+
+- `#5013 <https://github.com/pytest-dev/pytest/issues/5013>`_: pytest now depends on `wcwidth <https://pypi.org/project/wcwidth>`__ to properly track unicode character sizes for more precise terminal output.
+
+
+- `#5059 <https://github.com/pytest-dev/pytest/issues/5059>`_: pytester's ``Testdir.popen()`` uses ``stdout`` and ``stderr`` via keyword arguments with defaults now (``subprocess.PIPE``).
+
+
+- `#5069 <https://github.com/pytest-dev/pytest/issues/5069>`_: The code for the short test summary in the terminal was moved to the terminal plugin.
+
+
+- `#5082 <https://github.com/pytest-dev/pytest/issues/5082>`_: Improved validation of kwargs for various methods in the pytester plugin.
+
+
+- `#5202 <https://github.com/pytest-dev/pytest/issues/5202>`_: ``record_property`` now emits a ``PytestWarning`` when used with ``junit_family=xunit2``: the fixture generates
+  ``property`` tags as children of ``testcase``, which is not permitted according to the most
+  `recent schema <https://github.com/jenkinsci/xunit-plugin/blob/master/
+  src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd>`__.
+
+
+- `#5239 <https://github.com/pytest-dev/pytest/issues/5239>`_: Pin ``pluggy`` to ``< 1.0`` so we don't update to ``1.0`` automatically when
+  it gets released: there are planned breaking changes, and we want to ensure
+  pytest properly supports ``pluggy 1.0``.
+
+
 pytest 4.4.2 (2019-05-08)
 =========================
 

changelog/5239.trivial.rst

@@ -43,7 +43,7 @@ clean:
 
 regen: REGENDOC_FILES:=*.rst */*.rst
 regen:
-	PYTHONDONTWRITEBYTECODE=1 PYTEST_ADDOPTS=-pno:hypothesis COLUMNS=76 regendoc --update ${REGENDOC_FILES} ${REGENDOC_ARGS}
+	PYTHONDONTWRITEBYTECODE=1 PYTEST_ADDOPTS="-pno:hypothesis -Wignore::pytest.PytestUnknownMarkWarning" COLUMNS=76 regendoc --update ${REGENDOC_FILES} ${REGENDOC_ARGS}
 
 html:
 	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html

doc/en/Makefile

@@ -6,6 +6,7 @@ Release announcements
    :maxdepth: 2
 
 
+   release-4.5.0
    release-4.4.2
    release-4.4.1
    release-4.4.0

doc/en/announce/index.rst

@@ -0,0 +1,35 @@
+pytest-4.5.0
+=======================================
+
+The pytest team is proud to announce the 4.5.0 release!
+
+pytest is a mature Python testing tool with more than a 2000 tests
+against itself, passing on many different interpreters and platforms.
+
+This release contains a number of bugs fixes and improvements, so users are encouraged
+to take a look at the CHANGELOG:
+
+    https://docs.pytest.org/en/latest/changelog.html
+
+For complete documentation, please visit:
+
+    https://docs.pytest.org/en/latest/
+
+As usual, you can upgrade from pypi via:
+
+    pip install -U pytest
+
+Thanks to all who contributed to this release, among them:
+
+* Anthony Sottile
+* Bruno Oliveira
+* Daniel Hahler
+* Floris Bruynooghe
+* Pulkit Goyal
+* Samuel Searles-Bryant
+* Zac Hatfield-Dodds
+* Zac-HD
+
+
+Happy testing,
+The Pytest Development Team

doc/en/announce/release-4.5.0.rst

@@ -27,40 +27,47 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
         name of your plugin or application to avoid clashes with other cache users.
 
         Values can be any object handled by the json stdlib module.
+
     capsys
         Enable text capturing of writes to ``sys.stdout`` and ``sys.stderr``.
 
         The captured output is made available via ``capsys.readouterr()`` method
         calls, which return a ``(out, err)`` namedtuple.
         ``out`` and ``err`` will be ``text`` objects.
+
     capsysbinary
         Enable bytes capturing of writes to ``sys.stdout`` and ``sys.stderr``.
 
         The captured output is made available via ``capsysbinary.readouterr()``
         method calls, which return a ``(out, err)`` namedtuple.
         ``out`` and ``err`` will be ``bytes`` objects.
+
     capfd
         Enable text capturing of writes to file descriptors ``1`` and ``2``.
 
         The captured output is made available via ``capfd.readouterr()`` method
         calls, which return a ``(out, err)`` namedtuple.
         ``out`` and ``err`` will be ``text`` objects.
+
     capfdbinary
         Enable bytes capturing of writes to file descriptors ``1`` and ``2``.
 
         The captured output is made available via ``capfd.readouterr()`` method
         calls, which return a ``(out, err)`` namedtuple.
         ``out`` and ``err`` will be ``byte`` objects.
-    doctest_namespace
+
+    doctest_namespace [session scope]
         Fixture that returns a :py:class:`dict` that will be injected into the namespace of doctests.
-    pytestconfig
+
+    pytestconfig [session scope]
         Session-scoped fixture that returns the :class:`_pytest.config.Config` object.
 
         Example::
 
             def test_foo(pytestconfig):
                 if pytestconfig.getoption("verbose") > 0:
                     ...
+
     record_property
         Add an extra properties the calling test.
         User properties become part of the test report and are available to the
@@ -72,10 +79,26 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
 
             def test_function(record_property):
                 record_property("example_key", 1)
+
     record_xml_attribute
         Add extra xml attributes to the tag for the calling test.
         The fixture is callable with ``(name, value)``, with value being
         automatically xml-encoded
+
+    record_testsuite_property [session scope]
+        Records a new ``<property>`` tag as child of the root ``<testsuite>``. This is suitable to
+        writing global information regarding the entire test suite, and is compatible with ``xunit2`` JUnit family.
+
+        This is a ``session``-scoped fixture which is called with ``(name, value)``. Example:
+
+        .. code-block:: python
+
+            def test_foo(record_testsuite_property):
+                record_testsuite_property("ARCH", "PPC")
+                record_testsuite_property("STORAGE_TYPE", "CEPH")
+
+        ``name`` must be a string, ``value`` will be converted to a string and properly xml-escaped.
+
     caplog
         Access and control log capturing.
 
@@ -85,6 +108,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
         * caplog.records         -> list of logging.LogRecord instances
         * caplog.record_tuples   -> list of (logger_name, level, message) tuples
         * caplog.clear()         -> clear captured records and formatted log output string
+
     monkeypatch
         The returned ``monkeypatch`` fixture provides these
         helper methods to modify objects, dictionaries or os.environ::
@@ -102,15 +126,19 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
         test function or fixture has finished. The ``raising``
         parameter determines if a KeyError or AttributeError
         will be raised if the set/deletion operation has no target.
+
     recwarn
         Return a :class:`WarningsRecorder` instance that records all warnings emitted by test functions.
 
         See http://docs.python.org/library/warnings.html for information
         on warning categories.
-    tmpdir_factory
+
+    tmpdir_factory [session scope]
         Return a :class:`_pytest.tmpdir.TempdirFactory` instance for the test session.
-    tmp_path_factory
+
+    tmp_path_factory [session scope]
         Return a :class:`_pytest.tmpdir.TempPathFactory` instance for the test session.
+
     tmpdir
         Return a temporary directory path object
         which is unique to each test function invocation,
@@ -119,6 +147,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
         path object.
 
         .. _`py.path.local`: https://py.readthedocs.io/en/latest/path.html
+
     tmp_path
         Return a temporary directory path object
         which is unique to each test function invocation,
@@ -130,6 +159,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
 
             in python < 3.6 this is a pathlib2.Path
 
+
     no tests ran in 0.12 seconds
 
 You can also interactively ask for help, e.g. by typing on the Python interactive prompt something like::

doc/en/builtin.rst

@@ -247,7 +247,7 @@ See the :ref:`cache-api` for more details.
 
 
 Inspecting Cache content
--------------------------------
+------------------------
 
 You can always peek at the content of the cache using the
 ``--cache-show`` command line option:
@@ -260,7 +260,7 @@ You can always peek at the content of the cache using the
     cachedir: $PYTHON_PREFIX/.pytest_cache
     rootdir: $REGENDOC_TMPDIR
     cachedir: $PYTHON_PREFIX/.pytest_cache
-    ------------------------------- cache values -------------------------------
+    --------------------------- cache values for '*' ---------------------------
     cache/lastfailed contains:
       {'test_50.py::test_num[17]': True,
        'test_50.py::test_num[25]': True,
@@ -277,8 +277,25 @@ You can always peek at the content of the cache using the
 
     ======================= no tests ran in 0.12 seconds =======================
 
+``--cache-show`` takes an optional argument to specify a glob pattern for
+filtering:
+
+.. code-block:: pytest
+
+    $ pytest --cache-show example/*
+    =========================== test session starts ============================
+    platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
+    cachedir: $PYTHON_PREFIX/.pytest_cache
+    rootdir: $REGENDOC_TMPDIR
+    cachedir: $PYTHON_PREFIX/.pytest_cache
+    ----------------------- cache values for 'example/*' -----------------------
+    example/value contains:
+      42
+
+    ======================= no tests ran in 0.12 seconds =======================
+
 Clearing Cache content
--------------------------------
+----------------------
 
 You can instruct pytest to clear all cache files and values
 by adding the ``--cache-clear`` option like this: