4-5x slowdown using pytest-cov

[jwineinger]

For some reason, I'm seeing a significant slowdown when using pytest-cov. The following test runs were literally one right after another.

FWIW, I'm running these in a docker-compose environment on docker for mac.

with coverage:

# py.test tests/ --durations=100 --junitxml=/tmp/xunit.xml --cov my_module --cov-report term-missing
=== test session starts ===
platform linux2 -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
Django settings: settings_override (from environment variable)
rootdir: /my/path, inifile:
plugins: xdist-1.16.0, mock-1.6.0, django-3.1.2, cov-2.5.1
collected 888 items 
<snip>
=== slowest 100 test durations ===
18.87s setup    ...
3.43s call     ...
3.20s call     ...
2.43s call     ...
<snip>
=== 870 passed, 17 skipped, 1 xfailed in 271.88 seconds ====

without coverage:

# py.test tests/ --durations=100 --junitxml=/tmp/xunit.xml
==== test session starts ===
platform linux2 -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
Django settings: settings_override (from environment variable)
rootdir: /opt/sps/identity-service, inifile:
plugins: xdist-1.16.0, mock-1.6.0, django-3.1.2, cov-2.5.1
collected 888 items 
<snip>
=== slowest 100 test durations ===
7.69s setup    ...
1.17s teardown ...
0.32s call     ...
0.30s call     ...
<snip>
=== 870 passed, 17 skipped, 1 xfailed in 58.74 seconds ===

Ideas on how to debug this?

[ionelmc]

Maybe you forgot to install the python-dev(el) package in your container (coverage would be compiled without the C extensions then). Tracing with pure python code is sloooooooow.

[jwineinger]

Looks like that was it. I made sure python-dev was installed in the container I build my wheels in and I'm now seeing similar times with and without coverage. Thanks!

[emjeyforta]

@jwineinger can you let me know how exactly did you do this? I run the tests within the docker container and the below command to install the same within the container does not seem to improve the speed. the tests without pytest-cov takes about 27min and with pytest-cov takes about 1h:15m which is incredibly high

RUN apt-get update && \
    apt-get install -y python3-dev build-essential && \
    rm -rf /var/lib/apt/lists/*

Looks like that was it. I made sure python-dev was installed in the container I build my wheels in and I'm now seeing similar times with and without coverage. Thanks!

[jwineinger]

@emjeyforta I use a multi-stage build. First stage builds wheels of my package and all dependencies. Later stages install them, and then we run tests in the resulting image. The first stage (now from python:3.12-slim) installs python3-dev (along with gcc and other non-python dev packages) like you have. I see nothing else relevant that pops out, sorry. 7 years is a long time to remember details.