Skip to content

Hypothesis integration raises internal error when collecting RuleBasedStateMachine #758

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Zac-HD opened this issue Jan 18, 2024 · 2 comments

Comments

@Zac-HD
Copy link
Member

Zac-HD commented Jan 18, 2024

Initially reported in HypothesisWorks/hypothesis#3850

The bug can be reproduced by installing pytest_asyncio and hypothesis, then trying to run the following test:

from hypothesis.stateful import RuleBasedStateMachine
class StatefulTest(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
 
TestStorage = StatefulTest.TestCase
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   ...
INTERNALERROR>   File ".../python3.11/site-packages/pytest_asyncio/plugin.py", line 371, in pytest_collection_modifyitems
INTERNALERROR>     and _hypothesis_test_wraps_coroutine(function)
INTERNALERROR>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File ".../python3.11/site-packages/pytest_asyncio/plugin.py", line 377, in _hypothesis_test_wraps_coroutine
INTERNALERROR>     return _is_coroutine(function.hypothesis.inner_test)
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^
INTERNALERROR> AttributeError: 'function' object has no attribute 'hypothesis'

To fix this, we should avoid assuming that the .hypothesis attribute exists in

@staticmethod
def _can_substitute(item: Function) -> bool:
func = item.obj
return getattr(
func, "is_hypothesis_test", False
) and asyncio.iscoroutinefunction(func.hypothesis.inner_test)

As a fix, I'd suggest:

        return (
            getattr(func, "is_hypothesis_test", False)
            and getattr(func, "hypothesis", None)
            and asyncio.iscoroutinefunction(func.hypothesis.inner_test)
        )

Note that there's no simple to support async stateful testing; the hypothesis-trio project is essentially a fork of our upstream stateful testing module with async handling injected in many places. My suggestion therefore continues to treat asyncio stateful tests as out of scope, since I don't think there's sufficient demand to be worth the maintainence burden.

@eivindjahren
Copy link

It seems that this is no longer an issue in pytest-asyncio==0.23.3. I suggest closing the issue.

@Zac-HD Zac-HD closed this as completed Jan 27, 2024
seifertm added a commit that referenced this issue Jan 28, 2024
As suggested in #758 (comment)

Signed-off-by: Michael Seifert <[email protected]>
@seifertm
Copy link
Contributor

Thanks for the cross-reference to the Hypothesis issue and for the improvement suggestion. I amended the check for Hypothesis tests as suggested.

Pytest-asyncio v0.23 had lots issues with test collection, unfortunately. These caused an INTERNALERROR in pytest at collection time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants