`pytest-benchmark`: AttributeError: module 'py' has no attribute 'io'

[ocelotl]

When I run test cases with 7.2.0 I get this error:

py39-opentelemetry-sdk installed: asgiref==3.5.2,attrs==22.1.0,Deprecated==1.2.13,exceptiongroup==1.0.0rc9,flaky==3.7.0,iniconfig==1.1.1,opentelemetry-api @ file:///home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-api,opentelemetry-sdk @ file:///home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-sdk,opentelemetry-semantic-conventions @ file:///home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-semantic-conventions,opentelemetry-test-utils @ file:///home/ocelotl/github/ocelotl/opentelemetry-python/tests/opentelemetry-test-utils,packaging==21.3,pluggy==1.0.0,py-cpuinfo==8.0.0,pyparsing==3.0.9,pytest==7.2.0,pytest-benchmark==3.4.1,tomli==2.0.1,typing_extensions==4.4.0,wrapt==1.14.1
	py39-opentelemetry-sdk run-test-pre: PYTHONHASHSEED='2761085170'
	py39-opentelemetry-sdk run-test-pre: commands[0] | python -m pip install -U pip setuptools wheel
	Requirement already satisfied: pip in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (22.3)
	Requirement already satisfied: setuptools in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (65.5.0)
	Requirement already satisfied: wheel in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (0.37.1)
	py39-opentelemetry-sdk run-test-pre: commands[1] | pip install /home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-api /home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-semantic-conventions /home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-sdk /home/ocelotl/github/ocelotl/opentelemetry-python/tests/opentelemetry-test-utils
	Processing /home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-api
	  Installing build dependencies: started
	  Installing build dependencies: finished with status 'done'
	  Getting requirements to build wheel: started
	  Getting requirements to build wheel: finished with status 'done'
	  Preparing metadata (pyproject.toml): started
	  Preparing metadata (pyproject.toml): finished with status 'done'
	Processing /home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-semantic-conventions
	  Installing build dependencies: started
	  Installing build dependencies: finished with status 'done'
	  Getting requirements to build wheel: started
	  Getting requirements to build wheel: finished with status 'done'
	  Preparing metadata (pyproject.toml): started
	  Preparing metadata (pyproject.toml): finished with status 'done'
	Processing /home/ocelotl/github/ocelotl/opentelemetry-python/opentelemetry-sdk
	  Installing build dependencies: started
	  Installing build dependencies: finished with status 'done'
	  Getting requirements to build wheel: started
	  Getting requirements to build wheel: finished with status 'done'
	  Preparing metadata (pyproject.toml): started
	  Preparing metadata (pyproject.toml): finished with status 'done'
	Processing /home/ocelotl/github/ocelotl/opentelemetry-python/tests/opentelemetry-test-utils
	  Installing build dependencies: started
	  Installing build dependencies: finished with status 'done'
	  Getting requirements to build wheel: started
	  Getting requirements to build wheel: finished with status 'done'
	  Preparing metadata (pyproject.toml): started
	  Preparing metadata (pyproject.toml): finished with status 'done'
	Requirement already satisfied: deprecated>=1.2.6 in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (from opentelemetry-api==1.13.0) (1.2.13)
	Requirement already satisfied: setuptools>=16.0 in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (from opentelemetry-api==1.13.0) (65.5.0)
	Requirement already satisfied: typing-extensions>=3.7.4 in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (from opentelemetry-sdk==1.13.0) (4.4.0)
	Requirement already satisfied: asgiref~=3.0 in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (from opentelemetry-test-utils==0.34b0) (3.5.2)
	Requirement already satisfied: wrapt<2,>=1.10 in /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages (from deprecated>=1.2.6->opentelemetry-api==1.13.0) (1.14.1)
	Building wheels for collected packages: opentelemetry-api, opentelemetry-semantic-conventions, opentelemetry-sdk, opentelemetry-test-utils
	  Building wheel for opentelemetry-api (pyproject.toml): started
	  Building wheel for opentelemetry-api (pyproject.toml): finished with status 'done'
	  Created wheel for opentelemetry-api: filename=opentelemetry_api-1.13.0-py3-none-any.whl size=50942 sha256=1901355eef56579ac74db918b4516c56279224c44c0e60e649dc64d9ef7e5c74
	  Stored in directory: /home/ocelotl/.cache/pip/wheels/97/52/38/b1470cb7b99810c0c536deaefbdc2d57a319b66391941669bd
	  Building wheel for opentelemetry-semantic-conventions (pyproject.toml): started
	  Building wheel for opentelemetry-semantic-conventions (pyproject.toml): finished with status 'done'
	  Created wheel for opentelemetry-semantic-conventions: filename=opentelemetry_semantic_conventions-0.34b0-py3-none-any.whl size=26484 sha256=f7173b52a2b82a26871000ad66a7fc4d924bd442bedc7cda7d8000483944437a
	  Stored in directory: /home/ocelotl/.cache/pip/wheels/5e/7b/ea/74985dc81f953e9718c69a2c03d6df36160c9f44c066ce2d36
	  Building wheel for opentelemetry-sdk (pyproject.toml): started
	  Building wheel for opentelemetry-sdk (pyproject.toml): finished with status 'done'
	  Created wheel for opentelemetry-sdk: filename=opentelemetry_sdk-1.13.0-py3-none-any.whl size=91031 sha256=b954e4be9c945ddeb0917d5ef5e1e552c5d456f5ec95b59232dfdb0041c385ea
	  Stored in directory: /home/ocelotl/.cache/pip/wheels/9b/98/37/769fc9cbe6ef9ca97cc67b7b9e050f4c0d4a976fbae98f1764
	  Building wheel for opentelemetry-test-utils (pyproject.toml): started
	  Building wheel for opentelemetry-test-utils (pyproject.toml): finished with status 'done'
	  Created wheel for opentelemetry-test-utils: filename=opentelemetry_test_utils-0.34b0-py3-none-any.whl size=12326 sha256=22a289670a4d4f9ebce121ec0646b0f9e5f0baf7ba5da37d62afa36ed1cb3f20
	  Stored in directory: /home/ocelotl/.cache/pip/wheels/4b/ff/aa/2c99f8acc78fb2eb8b563c2f005c4ec54692fec47266c2ceeb
	Successfully built opentelemetry-api opentelemetry-semantic-conventions opentelemetry-sdk opentelemetry-test-utils
	Installing collected packages: opentelemetry-semantic-conventions, opentelemetry-api, opentelemetry-sdk, opentelemetry-test-utils
	  Attempting uninstall: opentelemetry-semantic-conventions
	    Found existing installation: opentelemetry-semantic-conventions 0.34b0
	    Uninstalling opentelemetry-semantic-conventions-0.34b0:
	      Successfully uninstalled opentelemetry-semantic-conventions-0.34b0
	  Attempting uninstall: opentelemetry-api
	    Found existing installation: opentelemetry-api 1.13.0
	    Uninstalling opentelemetry-api-1.13.0:
	      Successfully uninstalled opentelemetry-api-1.13.0
	  Attempting uninstall: opentelemetry-sdk
	    Found existing installation: opentelemetry-sdk 1.13.0
	    Uninstalling opentelemetry-sdk-1.13.0:
	      Successfully uninstalled opentelemetry-sdk-1.13.0
	  Attempting uninstall: opentelemetry-test-utils
	    Found existing installation: opentelemetry-test-utils 0.34b0
	    Uninstalling opentelemetry-test-utils-0.34b0:
	      Successfully uninstalled opentelemetry-test-utils-0.34b0
	Successfully installed opentelemetry-api-1.13.0 opentelemetry-sdk-1.13.0 opentelemetry-semantic-conventions-0.34b0 opentelemetry-test-utils-0.34b0
	py39-opentelemetry-sdk run-test: commands[0] | pytest metrics
	INTERNALERROR> Traceback (most recent call last):
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/_pytest/main.py", line 266, in wrap_session
	INTERNALERROR>     config._do_configure()
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1037, in _do_configure
	INTERNALERROR>     self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pluggy/_hooks.py", line 277, in call_historic
	INTERNALERROR>     res = self._hookexec(self.name, self.get_hookimpls(), kwargs, False)
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
	INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
	INTERNALERROR>     return outcome.get_result()
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
	INTERNALERROR>     raise ex[1].with_traceback(ex[2])
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
	INTERNALERROR>     res = hook_impl.function(*args)
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pytest_benchmark/plugin.py", line 440, in pytest_configure
	INTERNALERROR>     bs = config._benchmarksession = BenchmarkSession(config)
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pytest_benchmark/session.py", line 38, in __init__
	INTERNALERROR>     self.logger = Logger(level, config=config)
	INTERNALERROR>   File "/home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/lib/python3.9/site-packages/pytest_benchmark/logger.py", line 20, in __init__
	INTERNALERROR>     self.term = py.io.TerminalWriter(file=sys.stderr)
	INTERNALERROR> AttributeError: module 'py' has no attribute 'io'
	ERROR: InvocationError for command /home/ocelotl/github/ocelotl/opentelemetry-python/.tox/py39-opentelemetry-sdk/bin/pytest metrics (exited with code 3)
	___________________________________ summary ____________________________________
	ERROR:   py39-opentelemetry-sdk: commands failed

When I do the same thing with pytest 7.1.3 tests pass.

Output of pip list when the error happens:

Package                            Version
---------------------------------- --------
asgiref                            3.5.2
attrs                              22.1.0
Deprecated                         1.2.13
exceptiongroup                     1.0.0rc9
flaky                              3.7.0
iniconfig                          1.1.1
opentelemetry-api                  1.13.0
opentelemetry-sdk                  1.13.0
opentelemetry-semantic-conventions 0.34b0
opentelemetry-test-utils           0.34b0
packaging                          21.3
pip                                22.3
pluggy                             1.0.0
py-cpuinfo                         8.0.0
pyparsing                          3.0.9
pytest                             7.2.0
pytest-benchmark                   3.4.1
setuptools                         65.5.0
tomli                              2.0.1
typing_extensions                  4.4.0
wheel                              0.37.1
wrapt                              1.14.1

Output of `pip list when the error does not happen:

Package                            Version
---------------------------------- -------
asgiref                            3.5.2
attrs                              22.1.0
Deprecated                         1.2.13
flaky                              3.7.0
iniconfig                          1.1.1
opentelemetry-api                  1.13.0
opentelemetry-sdk                  1.13.0
opentelemetry-semantic-conventions 0.34b0
opentelemetry-test-utils           0.34b0
packaging                          21.3
pip                                22.3
pluggy                             1.0.0
py                                 1.11.0
py-cpuinfo                         8.0.0
pyparsing                          3.0.9
pytest                             7.1.3
pytest-benchmark                   3.4.1
setuptools                         65.5.0
tomli                              2.0.1
typing_extensions                  4.4.0
wheel                              0.37.1
wrapt                              1.14.1

Output of lsb_release -a:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

Minimal example:

Clone opentelemetry-python, run tox -e py39-opentelemetry-sdk -- opentelemetry-sdk/tests/metrics/

By the way, thanks for all your work, Pytest is a cornerstone of our project!

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

[b3n3w]

Experiencing the same issue with the latest version of pytest 7.2.0 on different distributions.

[The-Compiler]

@ocelotl This is a bug in pytest-benchmark, due to it depending on py without declaring it as a dependency. The shim we introduced in pytest (#10396) intentionally only incudes py.path and py.error.

Looks like this has been fixed in pytest-benchmark already: ionelmc/pytest-benchmark#226 - so once a new release with that is out, it should work again.

@b3n3w Hard to say more without more details, but it's also very unlikely that you're seeing an issue in pytest. Check the traceback and the files involved to see the project responsible.

[The-Compiler]

Pinning this for a while for visibility, as we had a duplicate opened already...

[asottile]

note also you can work around by installing py separately