Export types of builtin fixture for type annotations

In order to allow users to type annotate fixtures they request, the
types need to be imported from the `pytest` namespace. They are/were
always available to import from the `_pytest` namespace, but that is
not guaranteed to be stable.

These types are only exported for the purpose of typing. Specifically,
the following are *not* public:

- Construction (`__init__`)
- Subclassing
- staticmethods and classmethods

We try to combat them being used anyway by:

- Marking the classes as `@final` when possible (already done).

- Not documenting private stuff in the API Reference.

- Using `_`-prefixed names or marking as `:meta private:` for private
  stuff.

- Adding a keyword-only `_ispytest=False` to private constructors,
  warning if False, and changing pytest itself to pass True. In the
  future it will (hopefully) become a hard error.

Hopefully that will be enough.

Author: bluetech

changelog/7469.deprecation.rst

@@ -0,0 +1,18 @@
+Directly constructing/calling the following classes/functions is now deprecated:
+
+- ``_pytest.cacheprovider.Cache``
+- ``_pytest.cacheprovider.Cache.for_config()``
+- ``_pytest.cacheprovider.Cache.clear_cache()``
+- ``_pytest.cacheprovider.Cache.cache_dir_from_config()``
+- ``_pytest.capture.CaptureFixture``
+- ``_pytest.fixtures.FixtureRequest``
+- ``_pytest.fixtures.SubRequest``
+- ``_pytest.logging.LogCaptureFixture``
+- ``_pytest.pytester.Pytester``
+- ``_pytest.pytester.Testdir``
+- ``_pytest.recwarn.WarningsRecorder``
+- ``_pytest.recwarn.WarningsChecker``
+- ``_pytest.tmpdir.TempPathFactory``
+- ``_pytest.tmpdir.TempdirFactory``
+
+These have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 7.0.0.

changelog/7469.improvement.rst

@@ -0,0 +1,23 @@
+It is now possible to construct a :class:`MonkeyPatch` object directly as ``pytest.MonkeyPatch()``,
+in cases when the :fixture:`monkeypatch` fixture cannot be used. Previously some users imported it
+from the private `_pytest.monkeypatch.MonkeyPatch` namespace.
+
+The types of builtin pytest fixtures are now exported so they may be used in type annotations of test functions.
+The newly-exported types are:
+
+- ``pytest.FixtureRequest`` for the :fixture:`request` fixture.
+- ``pytest.Cache`` for the :fixture:`cache` fixture.
+- ``pytest.CaptureFixture[str]`` for the :fixture:`capfd` and :fixture:`capsys` fixtures.
+- ``pytest.CaptureFixture[bytes]`` for the :fixture:`capfdbinary` and :fixture:`capsysbinary` fixtures.
+- ``pytest.LogCaptureFixture`` for the :fixture:`caplog` fixture.
+- ``pytest.Pytester`` for the :fixture:`pytester` fixture.
+- ``pytest.Testdir`` for the :fixture:`testdir` fixture.
+- ``pytest.TempdirFactory`` for the :fixture:`tmpdir_factory` fixture.
+- ``pytest.TempPathFactory`` for the :fixture:`tmp_path_factory` fixture.
+- ``pytest.MonkeyPatch`` for the :fixture:`monkeypatch` fixture.
+- ``pytest.WarningsRecorder`` for the :fixture:`recwarn` fixture.
+
+Constructing them is not supported (except for `MonkeyPatch`); they are only meant for use in type annotations.
+Doing so will emit a deprecation warning, and may become a hard-error in pytest 7.0.
+
+Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.

doc/en/reference.rst

@@ -304,11 +304,10 @@ request ``pytestconfig`` into your fixture and get it with ``pytestconfig.cache`
 Under the hood, the cache plugin uses the simple
 ``dumps``/``loads`` API of the :py:mod:`json` stdlib module.
 
-.. currentmodule:: _pytest.cacheprovider
+``config.cache`` is an instance of :class:`pytest.Cache`:
 
-.. automethod:: Cache.get
-.. automethod:: Cache.set
-.. automethod:: Cache.makedir
+.. autoclass:: pytest.Cache()
+   :members:
 
 
 .. fixture:: capsys
@@ -318,12 +317,10 @@ capsys
 
 **Tutorial**: :doc:`capture`.
 
-.. currentmodule:: _pytest.capture
-
-.. autofunction:: capsys()
+.. autofunction:: _pytest.capture.capsys()
     :no-auto-options:
 
-    Returns an instance of :py:class:`CaptureFixture`.
+    Returns an instance of :class:`CaptureFixture[str] <pytest.CaptureFixture>`.
 
     Example:
 
@@ -334,7 +331,7 @@ capsys
             captured = capsys.readouterr()
             assert captured.out == "hello\n"
 
-.. autoclass:: CaptureFixture()
+.. autoclass:: pytest.CaptureFixture()
     :members:
 
 
@@ -345,10 +342,10 @@ capsysbinary
 
 **Tutorial**: :doc:`capture`.
 
-.. autofunction:: capsysbinary()
+.. autofunction:: _pytest.capture.capsysbinary()
     :no-auto-options:
 
-    Returns an instance of :py:class:`CaptureFixture`.
+    Returns an instance of :class:`CaptureFixture[bytes] <pytest.CaptureFixture>`.
 
     Example:
 
@@ -367,10 +364,10 @@ capfd
 
 **Tutorial**: :doc:`capture`.
 
-.. autofunction:: capfd()
+.. autofunction:: _pytest.capture.capfd()
     :no-auto-options:
 
-    Returns an instance of :py:class:`CaptureFixture`.
+    Returns an instance of :class:`CaptureFixture[str] <pytest.CaptureFixture>`.
 
     Example:
 
@@ -389,10 +386,10 @@ capfdbinary
 
 **Tutorial**: :doc:`capture`.
 
-.. autofunction:: capfdbinary()
+.. autofunction:: _pytest.capture.capfdbinary()
     :no-auto-options:
 
-    Returns an instance of :py:class:`CaptureFixture`.
+    Returns an instance of :class:`CaptureFixture[bytes] <pytest.CaptureFixture>`.
 
     Example:
 
@@ -433,7 +430,7 @@ request
 
 The ``request`` fixture is a special fixture providing information of the requesting test function.
 
-.. autoclass:: _pytest.fixtures.FixtureRequest()
+.. autoclass:: pytest.FixtureRequest()
     :members:
 
 
@@ -475,9 +472,9 @@ caplog
 .. autofunction:: _pytest.logging.caplog()
     :no-auto-options:
 
-    Returns a :class:`_pytest.logging.LogCaptureFixture` instance.
+    Returns a :class:`pytest.LogCaptureFixture` instance.
 
-.. autoclass:: _pytest.logging.LogCaptureFixture
+.. autoclass:: pytest.LogCaptureFixture()
     :members:
 
 
@@ -504,9 +501,7 @@ pytester
 
 .. versionadded:: 6.2
 
-.. currentmodule:: _pytest.pytester
-
-Provides a :class:`Pytester` instance that can be used to run and test pytest itself.
+Provides a :class:`~pytest.Pytester` instance that can be used to run and test pytest itself.
 
 It provides an empty directory where pytest can be executed in isolation, and contains facilities
 to write tests, configuration files, and match against expected output.
@@ -519,16 +514,16 @@ To use it, include in your topmost ``conftest.py`` file:
 
 
 
-.. autoclass:: Pytester()
+.. autoclass:: pytest.Pytester()
     :members:
 
-.. autoclass:: RunResult()
+.. autoclass:: _pytest.pytester.RunResult()
     :members:
 
-.. autoclass:: LineMatcher()
+.. autoclass:: _pytest.pytester.LineMatcher()
     :members:
 
-.. autoclass:: HookRecorder()
+.. autoclass:: _pytest.pytester.HookRecorder()
     :members:
 
 .. fixture:: testdir
@@ -541,7 +536,7 @@ legacy ``py.path.local`` objects instead when applicable.
 
 New code should avoid using :fixture:`testdir` in favor of :fixture:`pytester`.
 
-.. autoclass:: Testdir()
+.. autoclass:: pytest.Testdir()
     :members:
 
 
@@ -552,12 +547,10 @@ recwarn
 
 **Tutorial**: :ref:`assertwarnings`
 
-.. currentmodule:: _pytest.recwarn
-
-.. autofunction:: recwarn()
+.. autofunction:: _pytest.recwarn.recwarn()
     :no-auto-options:
 
-.. autoclass:: WarningsRecorder()
+.. autoclass:: pytest.WarningsRecorder()
     :members:
 
 Each recorded warning is an instance of :class:`warnings.WarningMessage`.
@@ -574,13 +567,11 @@ tmp_path
 
 **Tutorial**: :doc:`tmpdir`
 
-.. currentmodule:: _pytest.tmpdir
-
-.. autofunction:: tmp_path()
+.. autofunction:: _pytest.tmpdir.tmp_path()
     :no-auto-options:
 
 
-.. fixture:: tmp_path_factory
+.. fixture:: _pytest.tmpdir.tmp_path_factory
 
 tmp_path_factory
 ~~~~~~~~~~~~~~~~
@@ -589,12 +580,9 @@ tmp_path_factory
 
 .. _`tmp_path_factory factory api`:
 
-``tmp_path_factory`` instances have the following methods:
+``tmp_path_factory`` is an instance of :class:`~pytest.TempPathFactory`:
 
-.. currentmodule:: _pytest.tmpdir
-
-.. automethod:: TempPathFactory.mktemp
-.. automethod:: TempPathFactory.getbasetemp
+.. autoclass:: pytest.TempPathFactory()
 
 
 .. fixture:: tmpdir
@@ -604,9 +592,7 @@ tmpdir
 
 **Tutorial**: :doc:`tmpdir`
 
-.. currentmodule:: _pytest.tmpdir
-
-.. autofunction:: tmpdir()
+.. autofunction:: _pytest.tmpdir.tmpdir()
     :no-auto-options:
 
 
@@ -619,12 +605,9 @@ tmpdir_factory
 
 .. _`tmpdir factory api`:
 
-``tmpdir_factory`` instances have the following methods:
-
-.. currentmodule:: _pytest.tmpdir
+``tmp_path_factory`` is an instance of :class:`~pytest.TempdirFactory`:
 
-.. automethod:: TempdirFactory.mktemp
-.. automethod:: TempdirFactory.getbasetemp
+.. autoclass:: pytest.TempdirFactory()
 
 
 .. _`hook-reference`:

src/_pytest/cacheprovider.py

@@ -25,6 +25,7 @@
 from _pytest.config import ExitCode
 from _pytest.config import hookimpl
 from _pytest.config.argparsing import Parser
+from _pytest.deprecated import check_ispytest
 from _pytest.fixtures import fixture
 from _pytest.fixtures import FixtureRequest
 from _pytest.main import Session
@@ -53,7 +54,7 @@
 
 
 @final
-@attr.s
+@attr.s(init=False)
 class Cache:
     _cachedir = attr.ib(type=Path, repr=False)
     _config = attr.ib(type=Config, repr=False)
@@ -64,26 +65,50 @@ class Cache:
     # sub-directory under cache-dir for values created by "set"
     _CACHE_PREFIX_VALUES = "v"
 
+    def __init__(self, cachedir: Path, config: Config, *, _ispytest: bool) -> None:
+        check_ispytest(_ispytest)
+        self._cachedir = cachedir
+        self._config = config
+
     @classmethod
-    def for_config(cls, config: Config) -> "Cache":
-        cachedir = cls.cache_dir_from_config(config)
+    def for_config(cls, config: Config, *, _ispytest: bool) -> "Cache":
+        """Create the Cache instance for a Config.
+
+        :meta private:
+        """
+        check_ispytest(_ispytest)
+        cachedir = cls.cache_dir_from_config(config, _ispytest=True)
         if config.getoption("cacheclear") and cachedir.is_dir():
-            cls.clear_cache(cachedir)
-        return cls(cachedir, config)
+            cls.clear_cache(cachedir, _ispytest=True)
+        return cls(cachedir, config, _ispytest=True)
 
     @classmethod
-    def clear_cache(cls, cachedir: Path) -> None:
-        """Clear the sub-directories used to hold cached directories and values."""
+    def clear_cache(cls, cachedir: Path, _ispytest: bool) -> None:
+        """Clear the sub-directories used to hold cached directories and values.
+
+        :meta private:
+        """
+        check_ispytest(_ispytest)
         for prefix in (cls._CACHE_PREFIX_DIRS, cls._CACHE_PREFIX_VALUES):
             d = cachedir / prefix
             if d.is_dir():
                 rm_rf(d)
 
     @staticmethod
-    def cache_dir_from_config(config: Config) -> Path:
+    def cache_dir_from_config(config: Config, *, _ispytest: bool) -> Path:
+        """Get the path to the cache directory for a Config.
+
+        :meta private:
+        """
+        check_ispytest(_ispytest)
         return resolve_from_str(config.getini("cache_dir"), config.rootpath)
 
-    def warn(self, fmt: str, **args: object) -> None:
+    def warn(self, fmt: str, *, _ispytest: bool, **args: object) -> None:
+        """Issue a cache warning.
+
+        :meta private:
+        """
+        check_ispytest(_ispytest)
         import warnings
         from _pytest.warning_types import PytestCacheWarning
 
@@ -152,15 +177,15 @@ def set(self, key: str, value: object) -> None:
                 cache_dir_exists_already = self._cachedir.exists()
                 path.parent.mkdir(exist_ok=True, parents=True)
         except OSError:
-            self.warn("could not create cache path {path}", path=path)
+            self.warn("could not create cache path {path}", path=path, _ispytest=True)
             return
         if not cache_dir_exists_already:
             self._ensure_supporting_files()
         data = json.dumps(value, indent=2, sort_keys=True)
         try:
             f = path.open("w")
         except OSError:
-            self.warn("cache could not write path {path}", path=path)
+            self.warn("cache could not write path {path}", path=path, _ispytest=True)
         else:
             with f:
                 f.write(data)
@@ -469,7 +494,7 @@ def pytest_cmdline_main(config: Config) -> Optional[Union[int, ExitCode]]:
 
 @hookimpl(tryfirst=True)
 def pytest_configure(config: Config) -> None:
-    config.cache = Cache.for_config(config)
+    config.cache = Cache.for_config(config, _ispytest=True)
     config.pluginmanager.register(LFPlugin(config), "lfplugin")
     config.pluginmanager.register(NFPlugin(config), "nfplugin")