Tox fails to install package under test when it comes with virtualenv

[jaraco]

Now that setuptools includes six and because virtualenv upgrades setuptools, virtualenvs get installed with six. As a result, when using tox to invoke the six test suite before the version has been bumped, the code under test doesn't get installed, probably because pip considers the installed version to satisfy the requirement. Then, tests run against the released code. The issue doesn't occur if usedevelop is indicated or if the version under test has a different metadata version than the version installed.

Probably, tox should install with --upgrade --force-reinstall (note that --force-reinstall doesn't work unless --upgrade is indicated) when installing the package under test.

[obestwalter]

Hi @jaraco I am sure you know about this already, but I better add it for reference: it is already possible to change the install command specifically for the project.

I am not sure wether it is a good idea to change the default install command.

[gaborbernat]

In general, I think the --upgrade --force-reinstall flag is not needed and would significantly slow down the install operation as it would force reinstall all packages. Given for edge cases the user can provide their own install_command I would close this as won't fix. Unless anyone objects that is.

[jaraco]

Yes, @gaborbernat, you're right - such a default behavior would slow down things dramatically and would be prohibitively undesirable. So I withdraw my original suggestion.

Perhaps instead tox should --force-reinstall --upgrade after installing the package and its requirements (in case pip failed to install the package under test). e.g.:

pip install .
pip install --force-reinstall --upgrade .

Still, that's not great because it does cause the satisfactory packages to be checked/upgraded.

What we really need is another way to signal to pip to definitely install the package I asked you to install.

In many ways, this issue isn't a tox issue. Perhaps it's a six issue - where six should use something like setuptools_scm or setuptools build tags to designate a distinct version number from the released version.... or just bump the version in the repo immediately after release.

Or perhaps it's a pip issue - that pip should provide a means to install . unconditionally. Maybe even late versions do this. I'll double-check.

[gaborbernat]

@jaraco any progress?

[jaraco]

I've confirmed that pip 10+ will unconditionally install . even if the version info matches:

six master $ .tox/python/bin/pip install .
Processing /Users/jaraco/code/public/six
Installing collected packages: six
  Found existing installation: six 1.11.0
    Uninstalling six-1.11.0:
      Successfully uninstalled six-1.11.0
  Running setup.py install for six ... done
Successfully installed six-1.11.0
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

And pip 9 has the reported behavior:

six master $ .tox/python/bin/pip install .
Processing /Users/jaraco/code/public/six
  Requirement already satisfied (use --upgrade to upgrade): six==1.11.0 from file:///Users/jaraco/code/public/six in ./.tox/python/lib/python3.7/site-packages
You are using pip version 9.0.3, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Therefore, this issue is fixed upstream.