Monkeypatch is changing functions context

[GabrielSalla]

Description

I'm trying to use monkeypath to set a different function from the original code while testing. Not sure if is really a bug or just something intended, but didn't find anything in the documentation.

Monkeypatching works perfectly until I try to implement a way to assert the function was called. When I created a variable to store some flags I would use in the tests (like the class Variables in the following example), I got assertion errors just as they were not set by the function.

Added some prints and running pytest it showed different name for the objects and different ids:

============================= test session starts =============================
platform win32 -- Python 3.6.1, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: D:\mp_bug, inifile:
plugins: asyncio-0.6.0
collected 1 item

tests\test_1.py F

================================== FAILURES ===================================
___________________________________ test_1 ____________________________________

    def test_1():
        my_code.f(10)
        print(Variables, id(Variables))
>       assert Variables.a == 10
E       assert None == 10
E        +  where None = Variables.a

tests\test_1.py:16: AssertionError
---------------------------- Captured stdout call -----------------------------
<class 'tests.test_1.Variables'> 2349607687928
<class 'test_1.Variables'> 2349607690760
========================== 1 failed in 0.07 seconds ===========================

The first line is the print of the f() function of the my_code.py file after the monkeypatch, while the second one is from the test_1.py file. I was expecting it to be exactly the same object but it changed it's context and isn't anymore.

When changing the test_1.py to the same directory as conftest.py it works again.

Pip freeze

aiopg==0.13.0
colorama==0.3.9
email==6.0.0a1
future==0.16.0
numpy==1.13.1
pandas==0.20.3
peewee==2.10.1
peewee-async==0.5.7
pluggy==0.5.2
psycopg2==2.7.1
py==1.4.34
pygame==1.9.3
PyInstaller==3.2.1
pypiwin32==220
pytest==3.2.1
pytest-asyncio==0.6.0
python-dateutil==2.6.1
pytz==2017.2
PyYAML==3.12
requests==2.13.0
six==1.10.0
slackclient==1.0.5
SQLAlchemy==1.1.10
tabulate==0.7.7
tornado==4.5.1
tox==2.9.1
virtualenv==15.1.0
websocket-client==0.40.0

Example code

# ./my_code.py

def f(v):
    pass

# ./conftest.py

import pytest
import my_code
import tests.test_1 as test_1

@pytest.fixture(autouse=True)
def change_f(monkeypatch):
    monkeypatch.setattr(my_code, "f", test_1.new_f)

# ./tests/test_1.py

import my_code


class Variables:
    a = None


def new_f(v):
    print(Variables, id(Variables))
    Variables.a = v


def test_1():
    my_code.f(10)
    print(Variables, id(Variables))
    assert Variables.a == 10

[nicoddemus]

Hmm that's strange, I cannot reproduce this on the latest master or 3.2.1.

My guess is that somehow your fixture is not being executed, but it should.

Could you please change it to:

@pytest.fixture(autouse=True)
def change_f(monkeypatch):
    assert 0
    monkeypatch.setattr(my_code, "f", test_1.new_f)

And see if that assert gets reached?

[GabrielSalla]

Replaced the change_f fixture by the one you suggested and got the expected assertion error:

Just an observation, I used another environment to test now.

============================================= test session starts ==============================================
platform linux -- Python 3.6.2, pytest-3.2.0, py-1.4.34, pluggy-0.4.0
rootdir: /home/gabriel/temp/extest, inifile:
plugins: hypothesis-3.33.0
collected 1 item s

tests/test_1.py E

==================================================== ERRORS ====================================================
___________________________________________ ERROR at setup of test_1 ___________________________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f6a1d78af98>

    @pytest.fixture(autouse=True)
    def change_f(monkeypatch):
>       assert 0
E       assert 0

conftest.py:11: AssertionError
=========================================== 1 error in 0.02 seconds ============================================

[GabrielSalla]

Upgraded pytest to 3.2.3 and got the same result

============================================= test session starts ==============================================
platform linux -- Python 3.6.2, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/gabriel/temp/extest, inifile:
plugins: hypothesis-3.33.0
collected 1 item                                                                                                

tests/test_1.py E

==================================================== ERRORS ====================================================
___________________________________________ ERROR at setup of test_1 ___________________________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff6de033588>

    @pytest.fixture(autouse=True)
    def change_f(monkeypatch):
>       assert 0
E       assert 0

conftest.py:11: AssertionError
=========================================== 1 error in 0.02 seconds ============================================

When removing the assert 0

============================================= test session starts ==============================================
platform linux -- Python 3.6.2, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/gabriel/temp/extest, inifile:
plugins: hypothesis-3.33.0
collected 1 item                                                                                                

tests/test_1.py F

=================================================== FAILURES ===================================================
____________________________________________________ test_1 ____________________________________________________

    def test_1():
        my_code.f(10)
        print(Variables, id(Variables))
>       assert Variables.a == 10
E       assert None == 10
E        +  where None = Variables.a

tests/test_1.py:16: AssertionError
--------------------------------------------- Captured stdout call ---------------------------------------------
<class 'tests.test_1.Variables'> 94222888525784
<class 'test_1.Variables'> 94222888561208
=========================================== 1 failed in 0.02 seconds ===========================================

[nicoddemus]

Strange. You can remove the assert 0 statement, it was there just to ensure change_f was being called.

Could you please try your example in a clean virtualenv with only pytest installed?

Also, after monkeypatch.setattr(...), can you print my_code.f? It should be the new_f function.

[GabrielSalla]

~/temp/extest  virtualenv -p python3.6 venv
Running virtualenv with interpreter /usr/bin/python3.6
Using base prefix '/usr'
New python executable in /home/gabriel/temp/extest/venv/bin/python3.6
Also creating executable in /home/gabriel/temp/extest/venv/bin/python
Installing setuptools, pip, wheel...done.

~/temp/extest  source venv/bin/activate

~/temp/extest  pip install pytest
Collecting pytest
  Using cached pytest-3.2.3-py2.py3-none-any.whl
Collecting py>=1.4.33 (from pytest)
  Using cached py-1.4.34-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./venv/lib/python3.6/site-packages (from pytest)
Installing collected packages: py, pytest
Successfully installed py-1.4.34 pytest-3.2.3

~/temp/extest pip freeze
py==1.4.34
pytest==3.2.3

~/temp/extest pytest
============================================= test session starts ==============================================
platform linux -- Python 3.6.2, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/gabriel/temp/extest, inifile:
plugins: hypothesis-3.33.0
collected 1 item

tests/test_1.py F

=================================================== FAILURES ===================================================
____________________________________________________ test_1 ____________________________________________________

    def test_1():
        my_code.f(10)
        print(Variables, id(Variables))
>       assert Variables.a == 10
E       assert None == 10
E        +  where None = Variables.a

tests/test_1.py:16: AssertionError
-------------------------------------------- Captured stdout setup ---------------------------------------------
<function new_f at 0x7fd20f54b598>
--------------------------------------------- Captured stdout call ---------------------------------------------
<class 'tests.test_1.Variables'> 94206532243416
<class 'test_1.Variables'> 94206532278840
=========================================== 1 failed in 0.03 seconds ===========================================

[GabrielSalla]

Are you sure you used the same file structure as I said? It only happens if it's like this:

.
├── conftest.py
├── my_code.py
└── tests
    └── test_1.py

[nicoddemus]

I could reproduce it, thanks (not sure why I wasn't able before, even with the same file structure).

I can see that the monkeypatch fixture is doing its job, the problem is that by importing the test module yourself into the conftest.py file you are ending with two Variables classes (not instances). If we print Variables.__module__ during new_f and during the test we can see what I mean:

def new_f(v):
    print('new_f:', Variables.__module__, id(Variables))
    ...

def test_1():
    my_code.f(10)
    print('test_1', Variables.__module__, id(Variables))
    ...

new_f: tests.test_1 2020345212040
test_1 test_1 2020345227144

We see that we have the Variables class living in two different modules.

When pytest finds the test_1.py file, because it is not living into a package (its parent directory does not contain an __init__.py file), pytest will import it as an "absolute" module: test_1.

When pytest imports the conftest.py file, it will end up importing tests.test_1 and that will work because of the implement namespace packages in Python 3. You will end up with a new module tests.test_1.

Both modules have their own Variables class, that's why monkeypatch.set does not seem to work.

A quick workaround is to add a __init__.py file to the tests directory; this way pytest will understand that it should load the test file as tests.test_1.

This is all confusing I admit, and the underlying problem is that pytest does not currently support implicit namespace packages unfortunately. Implicit namespace packages is a complicated topic and a lot of tools (setuptools is another one) are having trouble to implement support for it reliably.

[GabrielSalla]

Well, that's sad
The workaround I used was to import a file with no code, just to store variables used in the tests. Like when a function is called, I add it's call to this file and then the caller checks if it really executed.

[nicoddemus]

Thanks for the follow up @GabrielSalla