Asyncio warning and error log messages should be displayed

[chaoflow]

An exception in an asyncio loop callback leads to an error message being logged by asyncio. Given that the test does not fail, this log message is not displayed, except if log_cli is enabled (see below). However, this leads to log messages from all loggers to be printed.

I think there should be an easy way to enable log messages for asyncio of warning and above and/or these should be visible by default.

Proposal

1. All log messages of asyncio should be visible by default OR there is an easy way to configure this
2. --pdb registers an exception handler for post-mortem debugging of loop callback exceptions

MWE

import asyncio

def raise_exc():
    raise ValueError

async def main():
    loop = asyncio.get_running_loop()
    loop.call_soon(raise_exc)
    await asyncio.sleep(0)

def test_callback_exc():
    asyncio.run(main())

if __name__ == '__main__':
    test_callback_exc()

setup.cfg

[tool:pytest]
log_cli = 1

info

Package    Version                 Location
---------- ----------------------- ----------------------------------
attrs      21.2.0
iniconfig  1.1.1
packaging  20.9
pip        21.1.2
pluggy     0.13.1
py         1.10.0
pyparsing  2.4.7
pytest     6.3.0.dev524+gc675d8054 /path/to/pytest
setuptools 57.0.0
toml       0.10.2

[chaoflow]

Related: It would be great if pytest --pdb would drop into a debugger inside the callback.

[asottile]

unfortunately I believe this would need a hack or a PEP to get solved -- this is similar to sys.unraisablehook (maybe that can be called by python in this case?) except specific to event loops -- there's set_exception_handler but it's insufficient because pytest does not create your event loop

the problem here is pytest has no control over the event loop creation -- that's being created by the test code itself -- perhaps something like pytest-asyncio could implement this though -- there the loop is created by the test machinery itself?

[chaoflow]

@asottile With asyncio we can register the handler whenever a new loop is created. The following in conftest.py works:

import sys

def set_handler_for_new_loop():
    import asyncio
    from functools import wraps

    policy = asyncio.get_event_loop_policy()
    _new_event_loop = policy.new_event_loop

    def post_mortem(*args, **kw):
        import pdb
        pdb.post_mortem(sys.exc_info()[2])

    @wraps(_new_event_loop)
    def new_event_loop():
        loop = _new_event_loop()
        loop.set_exception_handler(post_mortem)
        return loop

    policy.new_event_loop = new_event_loop
    asyncio.set_event_loop_policy(policy)

if '--pdb' in sys.argv:
    set_handler_for_new_loop()

Would something like this fit into pytest itself? I see the need for it independent of whether the actual test functions are async. In the example above pytest-asyncio or something similar is not involved.

Without handler being set, asyncio logs the exception as ERROR with logger name asyncio. Do you see a way to display these log messages specifically instead of enabling log_cli broadly for all loggers?

[chaoflow]

I also updated the description above to reflect the proposal more clearly.

[Zac-HD]

Closing this because I think it's best solved in pytest-asyncio (and see #12465 (comment))