py.test ignores exceptions in fixtures

[Wilfred]

pytest seems to silently ignore any exception inside fixtures.

import pytest

@pytest.yield_fixture(scope='session', autouse=True)
def my_fixture():
    print "inside fixture"
    raise Exception("oh dear")
    yield

def test_foo():
    assert True

Running this with py.test exception.py -s gives:

exception.py inside fixture
.
================================================ 1 passed in 0.01 seconds =================================================

I would expect py.test to at least warn me that my fixture threw an error.

[nicoddemus]

py.test does fail tests if a fixture raises an exception, as expected:

============================= test session starts =============================
platform win32 -- Python 2.7.11, pytest-2.9.2.dev1, py-1.4.31, pluggy-0.3.1
rootdir: X:\pytest, inifile: tox.ini
collected 1 items

foo.py inside fixture
E

=================================== ERRORS ====================================
_________________________ ERROR at setup of test_foo __________________________

    @pytest.yield_fixture(scope='session', autouse=True)
    def my_fixture():
        print("inside fixture")
>       raise Exception("oh dear")
E       Exception: oh dear

foo.py:6: Exception
=========================== 1 error in 0.06 seconds ===========================

(Tried this in both Python 2.7 and Python 3.5)

Which python and pytest versions are you using?

[Wilfred]

I'm using Python 2.7 with pytest 2.6.4.

[nicoddemus]

2.6.4 is quite old... if this was a bug it was fixed already. 😁

[nicoddemus]

(Please try upgrading pytest and reopen this if you still see the issue)

[Wilfred]

Worryingly, I'm seeing this on pytest 2.9.1 as well. I'll see if it's plugin related.

[RonnyPfannschmidt]

please provide full output then, are you perhaps using pytest-sugar?

[Wilfred]

Hmm, seems to be the "flaky" plugin. Uninstalling that makes the exception appear as expected.

$ py.test exception.py -s
===================================================================== test session starts =====================================================================
platform linux2 -- Python 2.7.3, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: redacted, inifile: setup.cfg
plugins: flaky-2.1.2
collected 1 items 

exception.py inside fixture
.
===Flaky Test Report===


===End Flaky Test Report===

================================================================== 1 passed in 0.01 seconds ===================================================================

[nicoddemus]

OK, thanks for narrowing the problem down. Could you please post an issue on their tracker?

[Wilfred]

OK, seems to be fixed in flaky 3.1.1. Sorry for the noise.

[nicoddemus]

No problem!