Skip to content

Pytest does not notify users when async tests are ignored #11372

@LukeWood

Description

@LukeWood

What's the problem this feature will solve?

When writing async unit-tests, it is extremely easy to forget to install the pytest-async extension and end up ignoring the futures.
This can cause unexpected silence:

async def test_should_fail_but_doesnt():
    assert False

This passes if pytest-async is not installed. Instead, I'd like to propose we make this fail with a nice message indicating that you need too install python-async.

Describe the solution you'd like

I'd like to inspect tests to see if they return a future. If they return a future, and pytest-async is not installed i'd like to error out and tell users to install it.

Alternative Solutions

Ignore the tests.

Additional context

I've been bit by this once~ know another team member of mine who has also been bitten by this.
I'm happy to contribute this.

Activity

RonnyPfannschmidt

RonnyPfannschmidt commented on Aug 30, 2023

@RonnyPfannschmidt
Member

More details needed,pytest notifies when async tests are there without a async plugin

LukeWood

LukeWood commented on Aug 30, 2023

@LukeWood
Author

More details needed,pytest notifies when async tests are there without a async plugin

Oh interesting! I had no idea- maybe I missed it in the logs?

what does notify mean in this context? Error or warning? If a warning maybe we should upgrade it to an error and gate the warning behind an environment variable?

RonnyPfannschmidt

RonnyPfannschmidt commented on Aug 30, 2023

@RonnyPfannschmidt
Member

Warnings

LukeWood

LukeWood commented on Aug 30, 2023

@LukeWood
Author
RonnyPfannschmidt

RonnyPfannschmidt commented on Aug 30, 2023

@RonnyPfannschmidt
Member

@nicoddemus @asottile @bluetech

I'm +1 on turning those into a failure for 8.x and would like your input

added
topic: reportingrelated to terminal output and user-facing messages and errors
on Sep 7, 2023
Zac-HD

Zac-HD commented on Sep 7, 2023

@Zac-HD
Member

Sounds good to me - we could even recommend a particular plugin based on sys.modules:

if pkgs := " or ".join(
    f"pytest-{n}" for n in ("anyio", "asyncio", "trio) 
    if n in sys.modules and f"pytest_{n}" not in sys.modules
):
    error_msg += "  Consider installing {pkgs}."
added this to the 9.0 milestone on May 20, 2024
Zac-HD

Zac-HD commented on Oct 22, 2024

@Zac-HD
Member

Relevant code is here. We should convert from a warning to an error, and include the nodeid in the error message - it's not currently included in this warning!

It'd also be good to turn other non-None returns into an error; that's similarly a few years past when we planned to finish the transition.

4 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic: collectionrelated to the collection phasetopic: reportingrelated to terminal output and user-facing messages and errors

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Participants

      @RonnyPfannschmidt@LukeWood@Zac-HD

      Issue actions

        Pytest does not notify users when async tests are ignored · Issue #11372 · pytest-dev/pytest