Make xpass failure again #11498
Make xpass failure again #11498
Conversation
for more information, see https://pre-commit.ci
…arwal28/pytest into make-xpass-failure-again
| item: Item, call: CallInfo[None] | ||
| ) -> Generator[None, TestReport, TestReport]: | ||
| rep = yield | ||
| @hookimpl(hookwrapper=True) |
There was a problem hiding this comment.
please restore the prior wrapper definition as thats the new style hook wrapper declaration from pluggy
| @@ -208,7 +207,7 @@ def evaluate_xfail_marks(item: Item) -> Optional[Xfail]: | |||
| """Evaluate xfail marks on item, returning Xfail if triggered.""" | |||
| for mark in item.iter_markers(name="xfail"): | |||
| run = mark.kwargs.get("run", True) | |||
| strict = mark.kwargs.get("strict", item.config.getini("xfail_strict")) | |||
| strict = mark.kwargs.get(item.config.getini("xfail_strict"), True) | |||
There was a problem hiding this comment.
in order to ensure backward compatibility we cannot go straight from non-strict to strict
instead we have to start by warning if strict was not set to true or false
the warning should indicate that a future major release of pytest will change the default from False to True
and recommend to use struct=True as default and a plugin for actually flaky tests
|
Sorry I'm closing this PR due to misunderstaning of a solution. I have created an another PR. |
|
Closes |
|
Please find below the latest open PR for this same issue with the changes suggested by you as well. |
Problem:
Some years ago, a change was made that caused
xpassnot to be considered a failure. However, there have been multiple cases where it was missed that a test was fixed, and this is not an acceptable default behavior.Solution:
This PR aims to address the issue by modifying the behavior of
xpassin the following way: In this strict parameter of xfail is set to true by default and it will ensure xpass is either a warning or a failure in any case.Checklist:
This change closes issue #11467.
Changelog: Created a new changelog file named
11467.feature.rstand added the entry for this change.Author: Tanya Agarwal