Add unraisableexception and threadexception plugins

[bluetech]

Fixes #5299.

Please see the added documentation for details. The first two commits just fix a couple of issues this found in our own testsuite.

While the two plugins are very similar, I chose to keep them separate because I think it's cleaner and so they may be disabled separately.

Performance-wise this adds about 3% to the pure pytest overhead, which seems fine.

[nicoddemus]

Awesome work @bluetech!

[nicoddemus]

Did you mean?

Suggested change

	The modules are automatically enabled for pytest runs, unless the
	The warnings are automatically enabled for pytest runs, unless the

[bluetech]

I copy the phrasing from the faulthandler section just above this one, but I agree it's not great. I don't think we should use "warnings" here because enabling warnings is a separate thing. I'll change this to "plugins".

[nicoddemus]

Great docs!

[RonnyPfannschmidt]

would it be sensible to move this into the context manager, then the yield from generator hack wouldn't have to be used

[bluetech]

I prefer to keep the context manager itself standalone from pytest stuff just in case it's useful on its own.

[The-Compiler]

There seems to be quite some code duplication between the two plugins - do they really need to be separate? I think I'd prefer having them in one plugin and sharing more code.

[bluetech]

There seems to be quite some code duplication between the two plugins - do they really need to be separate?

The certainly look similar, but the similarity is just structural, not logical. IMO it would be wrong to couple them -- they couldn't be disabled separately, and I suspect will cause maintainability issues if new requirements ever come up, a process which is nicely described in this article and is pretty accurate from my experience.

[dstansby]

Thanks for the work on this, looking forward to making use of it! Does anyone know which pytest release this is likely to be released in?

[nicoddemus]

It will go in 6.2, which we plan to release this Saturday! 🎉

#8101

[xuhdev]

Could someone explain what this new feature means? I read the document of PytestUnraisableExceptionWarning but still have no idea what it means:

Unraisable exceptions are exceptions raised in __del__ implementations and similar situations when the exception cannot be raised as normal.

Thanks!

[bluetech]

Here is an example from the tests:

        class BrokenDel:
            def __del__(self) -> None:
                raise ValueError("del is broken")

        def test_it() -> None:
            obj = BrokenDel()
            del obj

Previously the ValueError would go unnoticed, since the test does pass and the output is captured. Now pytest issues a PytestUnraisableExceptionWarning warning for this test, so you can better notice the ValueError problem.

[flyser]

Is there a way to propagate this warning to an error to let the tests fail?

[graingert]

@flyser see https://docs.pytest.org/en/stable/warnings.html#warnings-capture

you can set:

# pyproject.toml
[tool.pytest.ini_options]
filterwarnings = ["error"]

I like to use the most strict config I can:

# pyproject.toml
[tool.pytest.ini_options]
addopts = ["--strict-config", "--strict-markers"]
filterwarnings = ["error"]
xfail_strict = true