Function-scoped fixture run before session-scoped with unittest.TestCase

[Lekensteyn]

Currently unittest.TestCase instances cannot use fixtures in their test specifications, reportedly due to "different design philosophies".

Would it be possible to create a class decorator to enable this? See this example (runs both with python -m unittest and pytest):

import unittest
import pytest

def fixture_wrapper(fn, params):
    # XXX discover fixture instead of hardcoding it here
    # pytest seems to add extra @py_assert1@ and @py_format1@ stuff here.
    #assert params == ('program',)
    assert params[0] == 'program'
    return lambda self, *args: fn(self, program())

def uses_fixtures(cls):
    '''Wraps all unittest.TestCase methods in order to inject fixtures.'''
    assert issubclass(cls, unittest.TestCase)
    for name in dir(cls):
        func = getattr(cls, name)
        if not name.startswith('test') or not callable(func):
            continue
        if func.__code__.co_argcount <= 1:
            # ignore methods which do not have fixture dependencies
            continue
        params = func.__code__.co_varnames[1:]
        print(name, params)
        setattr(cls, name, fixture_wrapper(func, params))

@pytest.fixture
def program():
    return 'dummy'

class TestDemo(object):
    def test_1(self, program):
        assert program == 'dummy'

@uses_fixtures
class DemoTestCase(unittest.TestCase):
    def test_2(self, program):
        assert program == 'dummy'

The motivation is to provide a transition path for an existing unittest suite to pytest. Currently I use pytest as test runner for an existing project that otherwise does not depend on pytest. To fix some issues (inability to run part of the tests when some program dependencies are missing) I would like to use fixtures, but without adding a hard dependency on pytest for now. With such a decorator, migration to pytest will require less extensive changes (dropping a decorator vs changing every test function to use the parameter instead of a property that was configured via the unittest setUp method).

Edit: one motivation for using fixtures is the ability to skip tests from it. E.g. if "program" is not available, skip the test.

[nicoddemus]

Hi @Lekensteyn,

I wholeheartedly understand the desire of using fixtures in unittest.TestCase subclasses as a stop gap measure when working with existing unittest-based test suites, I've done this myself countless times.

One approach that works out of the box is using autouse fixtures to inject other fixtures in the class:

class DemoTestCase(unittest.TestCase):

    @pytest.fixture(autouse=True)
    def _inject_fixtures(self, program):
        self.program = program

    def test_2(self):
        assert self.program == 'dummy'

From that, it follows you can then use the same technique to obtain access to request and inject the fixtures into the methods:

import functools
import unittest
import pytest
import inspect

def fixture_wrapper(fn, params):
    params = params[1:]  # skip self

    @functools.wraps(fn)
    def wrapped(self):
        fixtures = (self.request.getfixturevalue(n) for n in params)
        fn(self, *fixtures)

    return wrapped

def uses_fixtures(cls):
    '''Wraps all unittest.TestCase methods in order to inject fixtures.'''
    assert issubclass(cls, unittest.TestCase)
    
    for name in dir(cls):
        func = getattr(cls, name)
        if not name.startswith('test') or not callable(func):
            continue
        argspec = inspect.getfullargspec(func)
        if len(argspec.args) <= 1:
            # ignore methods which do not have fixture dependencies
            continue
        setattr(cls, name, fixture_wrapper(func, argspec.args))

    @pytest.fixture(autouse=True)
    def __inject_request(self, request):
        self.request = request
    cls.__inject_request = __inject_request
    return cls

@pytest.fixture
def program():
    return 'dummy'

class TestDemo(object):
    def test_1(self, program):
        assert program == 'dummy'

@uses_fixtures
class DemoTestCase(unittest.TestCase):
    def test_2(self, program):
        assert program == 'dummy'

This is an interesting idea, and it can be made automatic using a conftest.py file or plugin:

@pytest.hookimpl(hookwrapper=True)
def pytest_pycollect_makeitem(collector, name, obj):
    from _pytest.unittest import UnitTestCase
    outcome = yield
    item = outcome.get_result()
    if isinstance(item, UnitTestCase):
        uses_fixtures(item.obj)

So the user doesn't even need the uses_fixture decorator:

class DemoTestCase(unittest.TestCase):
    def test_2(self, program):
        assert program == 'dummy'

I'm not sure how far we want to take it by adding this to the core, as the possibility of maintenance problems is very real, but this could be made into an external plugin very easily (we can even mention it in the official pytest docs).

[Lekensteyn]

Wow, that's a cool and clever approach! I suppose that the decorator pattern is preferable for compatibility with python -m unittest, but otherwise the conftest.py addition would definitely help with an ongoing transition.

I'm going to play a bit with it and experiment with a (hopefully) small compatiblity layer to avoid breaking those who still rely on unittest only.

[RonnyPfannschmidt]

would you consider your issue resolved?

[digitalresistor]

Is someone working on making this a pytest plugin that is distributed on pypi? This would be fantastic for those of us that unfortunately can't migrate wholesale to pytest just yet.

[nicoddemus]

Is someone working on making this a pytest plugin that is distributed on pypi?

Indeed that would be awesome. If somebody does, please post it in this issue!

[nicoddemus]

https://twitter.com/nicoddemus/status/1052646757061091334, let's see if somebody picks this up. 😉

[Lekensteyn]

@RonnyPfannschmidt My original issue is resolved, but I think that https://docs.pytest.org/en/latest/unittest.html should be updated before this issue is closed. Currently it says that fixtures do not work, but with the class decorator hack it does seem to work!

As for the fallback when pytest is unavailable, I wrote a small module that can be used directly with unittest classes and provides a limited fallback/emulation layer when pytest is not available. See fixtures.py in https://code.wireshark.org/review/30220 (WIP)

[nicoddemus]

Currently it says that fixtures do not work, but with the class decorator hack it does seem to work!

Well I think the documentation is correct as is, because pytest does not support that out of the box. 😉

But I very much like to add a blurb to the docs pointing out to a plugin which enables this, if somebody gets to write one.

[Lekensteyn]

I discovered one issue, higher-scoped fixtures are not executed first:

# <imports and uses_fixture definition elided for brevity>
counters = [0, 0]
def counter_inc(index, what):
    counters[index] += 1
    print("\ncounters[%d] = %d  # %s" % (index, counters[index], what))
    return counters[index]

# Normal test with pytest
@pytest.fixture(scope='session')
def session_scoped1():
    return counter_inc(0, 's')

@pytest.fixture
def function_scoped1():
    return counter_inc(0, 'f')

class TestCasePytest(object):
    def test_1(self, function_scoped1, session_scoped1):
        # ok: session scoped is initialized before function scoped
        assert (session_scoped1, function_scoped1) == (1, 2)

# Same tests, but with unittest.TestCase
@pytest.fixture(scope='session')
def session_scoped2():
    return counter_inc(1, 's')

@pytest.fixture
def function_scoped2():
    return counter_inc(1, 'f')

@uses_fixtures
class UnittestTestCase(unittest.TestCase):
    def test_2(self, function_scoped2, session_scoped2):
        # unexpectedly fails with (2, 1)
        assert (session_scoped2, function_scoped2) == (1, 2)

Output for pytest -s:

test_unittest_fixtures.py 
counters[0] = 1  # s

counters[0] = 2  # f
.
counters[1] = 1  # f

counters[1] = 2  # s
F
....
    def test_2(self, function_scoped2, session_scoped2):
>       assert (session_scoped2, function_scoped2) == (1, 2)
E       AssertionError: assert (2, 1) == (1, 2)