Skip to content

Parallel mode disables usedevelop #1297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ionelmc opened this issue May 14, 2019 · 5 comments
Closed

Parallel mode disables usedevelop #1297

ionelmc opened this issue May 14, 2019 · 5 comments
Labels
bug:normal affects many people or has quite an impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.

Comments

@ionelmc
Copy link

ionelmc commented May 14, 2019

Instead it will install an sdist.

Reproducing example:

ionel@bionic:/tmp$ cat tox.ini
[tox]
envlist = py36-{a,b}

[toxenv]
usedevelop = true

ionel@bionic:/tmp$ cat setup.py
from setuptools import setup

setup(name='foobar')

ionel@bionic:/tmp$ tox -p 2 -vv
using tox.ini: /tmp/tox.ini (pid 68987)
  removing /tmp/.tox/log
using tox-3.10.0 from /usr/local/lib/python3.6/dist-packages/tox/__init__.py (pid 68987)
GLOB start: packaging
GLOB sdist-make: /tmp/setup.py
  removing /tmp/.tox/dist
[68991] /tmp$ /usr/bin/python3 setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/GLOB-0.log
running sdist
running egg_info
writing foobar.egg-info/PKG-INFO
writing dependency_links to foobar.egg-info/dependency_links.txt
writing top-level names to foobar.egg-info/top_level.txt
reading manifest file 'foobar.egg-info/SOURCES.txt'
writing manifest file 'foobar.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md

running check
warning: Check: missing required meta-data: url

warning: Check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

creating foobar-0.0.0
creating foobar-0.0.0/foobar.egg-info
copying files to foobar-0.0.0...
copying setup.py -> foobar-0.0.0
copying foobar.egg-info/PKG-INFO -> foobar-0.0.0/foobar.egg-info
copying foobar.egg-info/SOURCES.txt -> foobar-0.0.0/foobar.egg-info
copying foobar.egg-info/dependency_links.txt -> foobar-0.0.0/foobar.egg-info
copying foobar.egg-info/top_level.txt -> foobar-0.0.0/foobar.egg-info
Writing foobar-0.0.0/setup.cfg
creating '.tox/dist/foobar-0.0.0.zip' and adding 'foobar-0.0.0' to it
adding 'foobar-0.0.0/setup.cfg'
adding 'foobar-0.0.0/PKG-INFO'
adding 'foobar-0.0.0/setup.py'
adding 'foobar-0.0.0/foobar.egg-info/SOURCES.txt'
adding 'foobar-0.0.0/foobar.egg-info/dependency_links.txt'
adding 'foobar-0.0.0/foobar.egg-info/top_level.txt'
adding 'foobar-0.0.0/foobar.egg-info/PKG-INFO'
removing 'foobar-0.0.0' (and everything under it)

GLOB finish: packaging  after 0.49 seconds
copying new sdistfile to '/home/ionel/.tox/distshare/foobar-0.0.0.zip'
package .tmp/package/4/foobar-0.0.0.zip links to dist/foobar-0.0.0.zip (/tmp/.tox)
⠋ [0] py36-a start: parallel py36-a
py36-b start: parallel py36-b
[68999] /tmp$ /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/tox/__main__.py -p 2 -vv --installpkg .tox/.tmp/package/4/foobar-0.0.0.zip >.tox/py36-b/log/py36-b-12.log
[69000] /tmp$ /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/tox/__main__.py -p 2 -vv --installpkg .tox/.tmp/package/4/foobar-0.0.0.zip >.tox/py36-a/log/py36-a-13.log
⠴ [2] py36-a | py36-bpy36-b finish: parallel py36-b  after 1.58 seconds
✔ OK py36-b in 1.576 seconds
⠦ [1] py36-apy36-a finish: parallel py36-a  after 1.61 seconds
✔ OK py36-a in 1.606 seconds
_____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
  py36-a: commands succeeded
  py36-b: commands succeeded
  congratulations :)
cleanup /tmp/.tox/.tmp/package/4/foobar-0.0.0.zip
ionel@bionic:/tmp$ cat .tox/py36-b/log/py36-b-12.log
action: py36-b, msg: parallel py36-b
cwd: /tmp
cmd: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/tox/__main__.py -p 2 -vv --installpkg .tox/.tmp/package/4/foobar-0.0.0.zip
using tox.ini: /tmp/tox.ini (pid 68999)
using tox-3.10.0 from /usr/local/lib/python3.6/dist-packages/tox/__init__.py (pid 68999)
using package '/tmp/.tox/.tmp/package/4/foobar-0.0.0.zip', skipping 'sdist' activity
package 5/foobar-0.0.0.zip links to 4/foobar-0.0.0.zip (/tmp/.tox/.tmp/package)
py36-b start: getenv /tmp/.tox/py36-b
py36-b uses /usr/bin/python3
py36-b reusing: /tmp/.tox/py36-b
py36-b finish: getenv /tmp/.tox/py36-b after 0.08 seconds
py36-b start: installpkg /tmp/.tox/.tmp/package/5/foobar-0.0.0.zip
py36-b inst-nodeps: /tmp/.tox/.tmp/package/5/foobar-0.0.0.zip
setting PATH=/tmp/.tox/py36-b/bin:/home/ionel/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/lib:/home/ionel/.antigen/bundles/zsh-users/zsh-syntax-highlighting:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/dirpersist:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/command-not-found:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/colorize:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/python:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/git:/home/ionel/.antigen/bundles/rimraf/k:/home/ionel/.antigen/bundles/ionelmc/docker-zsh-completion:/home/ionel/.fzf/bin
[69011] /tmp$ /tmp/.tox/py36-b/bin/python -m pip install --no-deps -U .tox/.tmp/package/5/foobar-0.0.0.zip
Processing ./.tox/.tmp/package/5/foobar-0.0.0.zip
Building wheels for collected packages: foobar
  Building wheel for foobar (setup.py): started
  Building wheel for foobar (setup.py): finished with status 'done'
  Stored in directory: /home/ionel/.cache/pip/wheels/48/78/f8/2dcdc098ca50df4d6b8fee5d1f6ab13e05f3860b94c6703e7c
Successfully built foobar
Installing collected packages: foobar
  Found existing installation: foobar 0.0.0
    Not uninstalling foobar at /tmp, outside environment /tmp/.tox/py36-b
    Can't uninstall 'foobar'. No files were found to uninstall.
Successfully installed foobar-0.0.0
WARNING: You are using pip version 19.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
py36-b finish: installpkg /tmp/.tox/.tmp/package/5/foobar-0.0.0.zip after 0.91 seconds
py36-b start: envreport
setting PATH=/tmp/.tox/py36-b/bin:/home/ionel/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/lib:/home/ionel/.antigen/bundles/zsh-users/zsh-syntax-highlighting:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/dirpersist:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/command-not-found:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/colorize:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/python:/home/ionel/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/git:/home/ionel/.antigen/bundles/rimraf/k:/home/ionel/.antigen/bundles/ionelmc/docker-zsh-completion:/home/ionel/.fzf/bin
[69035] /tmp$ /tmp/.tox/py36-b/bin/python -m pip freeze >.tox/py36-b/log/py36-b-13.log
py36-b finish: envreport  after 0.32 seconds
py36-b installed: foobar==0.0.0
  removing /tmp/.tox/py36-b/tmp
py36-b start: run-test-pre
py36-b run-test-pre: PYTHONHASHSEED='3790122033'
py36-b finish: run-test-pre  after 0.00 seconds
py36-b start: run-test
py36-b finish: run-test  after 0.00 seconds
py36-b start: run-test-post
py36-b finish: run-test-post  after 0.00 seconds
cleanup /tmp/.tox/.tmp/package/5/foobar-0.0.0.zip
ionel@bionic:/tmp$ tox --version
3.10.0 imported from /usr/local/lib/python3.6/dist-packages/tox/__init__.py

I have found this problem while trying to figure out how to skip wheel building when not really necessary for my tox-wheel plugin. I have this feeling that whatever the fix will be it will break my plugin so lets discuss first I guess?
@ionelmc ionelmc added the bug:normal affects many people or has quite an impact label May 14, 2019
@gaborbernat gaborbernat added the help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted. label May 15, 2019
@gaborbernat
Copy link
Member

Seems like a big oversight on my part, sorry 🙅‍♂

@atugushev
Copy link

atugushev commented Nov 10, 2019 •
edited
Loading

@ionelmc looks like rename [toxenv] to [testenv] fixes the issue. See:

$ cat tox.ini
[tox]
envlist = py36-{a,b}

[testenv]
usedevelop = true

$ tox -p 2 -vv
using tox.ini: /private/var/folders/4x/rz_w89dx1z3bydfd6qpdjn_h0000gn/T/tmp.zkRutZYE/tox.ini (pid 76029)
using tox-3.14.0 from /usr/local/lib/python2.7/site-packages/tox/__init__.pyc (pid 76029)
skipping sdist step
⠋ [0] py36-a start: parallel py36-a
py36-b start: parallel py36-b
[76030] /private/var/folders/4x/rz_w89dx1z3bydfd6qpdjn_h0000gn/T/tmp.zkRutZYE$ /usr/local/opt/python@2/bin/python2.7 /usr/local/lib/python2.7/site-packages/tox/__main__.py -p 2 -vv >.tox/py36-a/log/py36-a-0.log
[76031] /private/var/folders/4x/rz_w89dx1z3bydfd6qpdjn_h0000gn/T/tmp.zkRutZYE$ /usr/local/opt/python@2/bin/python2.7 /usr/local/lib/python2.7/site-packages/tox/__main__.py -p 2 -vv >.tox/py36-b/log/py36-b-0.log
⠴ [2] py36-a | py36-bpy36-b finish: parallel py36-b  after 5.68 seconds
✔ OK py36-b in 5.678 seconds
py36-a finish: parallel py36-a  after 5.68 seconds
✔ OK py36-a in 5.679 seconds
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
  py36-a: commands succeeded
  py36-b: commands succeeded
  congratulations :)

Note the "skipping sdist step".

@asottile asottile mentioned this issue Mar 23, 2020
Closed
@gaborbernat gaborbernat reopened this Apr 19, 2020
@gaborbernat
Copy link
Member

Feel free to open a PR against master and fix it. My available efforts at the moment are aimed at fixing this as part of #1394, but that probably will take a while (ETA September).

@gaborbernat
Copy link
Member

This has now been fixed with the rewrite (version 4).

@mgedmin
Copy link
Contributor

mgedmin commented Dec 20, 2022

Huh, I just saw this today. I upgraded tox to version 4.0.15, ran tox p in check-python-versions and saw this:

flake8: OK ✔ in 0.48 seconds
mypy: OK ✔ in 0.48 seconds
isort: OK ✔ in 0.18 seconds
py310: OK ✔ in 7.63 seconds
py311: OK ✔ in 8.16 seconds
coverage: FAIL ✖ in 7.68 seconds
coverage: install_package> python -I -m pip install --force-reinstall --no-deps /home/mg/src/check-python-versions/.tox/.tmp/package/14/check_python_versions-0.20.1.dev0-py2.py3-none-any.whl
coverage: commands[0]> coverage run -m pytest tests
========================================== test session starts ==========================================
platform linux -- Python 3.10.7, pytest-7.2.0, pluggy-1.0.0
cachedir: .tox/coverage/.pytest_cache
rootdir: /home/mg/src/check-python-versions, configfile: pytest.ini
collected 355 items                                                                                     

tests/test___main__.py .                                                                          [  0%]
tests/test_cli.py .....................................................                           [ 15%]
tests/test_utils.py .............                                                                 [ 18%]
tests/test_versions.py ....................                                                       [ 24%]
tests/parsers/test_ini.py ........                                                                [ 26%]
tests/parsers/test_python.py ....................................                                 [ 36%]
tests/parsers/test_yaml.py ......................                                                 [ 43%]
tests/sources/test_appveyor.py ...........................                                        [ 50%]
tests/sources/test_github.py ....................                                                 [ 56%]
tests/sources/test_manylinux.py ....                                                              [ 57%]
tests/sources/test_setup_py.py .................................................................. [ 76%]
..........                                                                                        [ 78%]
tests/sources/test_tox.py ..............................................                          [ 91%]
tests/sources/test_travis.py .............................                                        [100%]

========================================== 355 passed in 3.98s ==========================================
coverage: commands[1]> coverage report -m --fail-under=100
Name                                                                                    Stmts   Miss Branch BrPart  Cover   Missing
-----------------------------------------------------------------------------------------------------------------------------------
.tox/coverage/lib/python3.10/site-packages/check_python_versions/__init__.py                2      0      0      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/__main__.py                1      0      0      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/cli.py                   190      0    119      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/__init__.py        0      0      0      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/appveyor.py       96     96     68      0     0%   25-181
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/ini.py            41      0     20      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/manylinux.py      36     36     18      0     0%   24-85
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/python.py        154      0     91      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/tox.py           139    139     87      0     0%   17-301
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/travis.py        170    170    122      0     0%   18-377
.tox/coverage/lib/python3.10/site-packages/check_python_versions/parsers/yaml.py          104      0     63      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/__init__.py        0      0      0      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/all.py             7      0      0      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/appveyor.py       98      0     68      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/base.py           22      0      0      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/github.py         79      0     57      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/manylinux.py      39      0     18      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/setup_py.py      198      0    131      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/tox.py           118      0     71      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/sources/travis.py         89      0     74      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/utils.py                  62      0     16      0   100%
.tox/coverage/lib/python3.10/site-packages/check_python_versions/versions.py               47      0     20      0   100%
-----------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                    1692    441   1043      0    73%
Coverage failure: total of 73 is less than fail-under=100
coverage: exit 2 (0.66 seconds) /home/mg/src/check-python-versions> coverage report -m --fail-under=100 pid=477239
py39: OK ✔ in 8.44 seconds
py38: OK ✔ in 8.57 seconds
py37: OK ✔ in 8.63 seconds
  py37: OK (8.63=setup[4.54]+cmd[4.09] seconds)
  py38: OK (8.57=setup[4.56]+cmd[4.02] seconds)
  py39: OK (8.44=setup[4.55]+cmd[3.89] seconds)
  py310: OK (7.63=setup[4.05]+cmd[3.57] seconds)
  py311: OK (8.16=setup[4.44]+cmd[3.72] seconds)
  pypy3: OK (11.82=setup[8.22]+cmd[3.60] seconds)
  flake8: OK (0.47=setup[0.03]+cmd[0.45] seconds)
  mypy: OK (0.48=setup[0.05]+cmd[0.43] seconds)
  isort: OK (0.18=setup[0.01]+cmd[0.17] seconds)
  coverage: FAIL code 2 (7.68=setup[2.23]+cmd[4.79,0.66] seconds)
  evaluation failed :( (11.91 seconds)

Coverage showing the paths as .tox/coverage/lib/python3.10/site-packages/check_python_versions/ indicates that usedevelop = true in my tox.ini was ignored.

Running tox -e covergage afterwards showed the correct 100% coverage with the paths in the coverage report showing src/check_python_versions/.

Since life is short and sporadic errors like this are annoying I modified check-python-versions to no longer use usedevelop in the next commit, but I thought you might want to know that the bug is not fixed yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug:normal affects many people or has quite an impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ionelmc @mgedmin @gaborbernat @atugushev