tox does not expand {opts} {packages} from inside install_command

[ssbarnea]

[testenv]
install_command = pip install -c{env:UPPER_CONSTRAINTS_FILE:/dev/null} {opts} {packages}

[testenv:foo]
commands =
   pip check
   {[testenv]install_command}
   pip check

This simplified example is supposed to test python environment for conflicts before and after installing the package but it fails with an exception like:

Invalid requirement: '{opts}'
Traceback (most recent call last):
File "/Users/ssbarnea/os/cinder/.tox/reqs/lib/python2.7/site-packages/pip/req/req_install.py", line 82, in __init__
req = Requirement(req)
File "/Users/ssbarnea/os/cinder/.tox/reqs/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py", line 96, in __init__
requirement_string[e.loc:e.loc + 8]))
InvalidRequirement: Invalid requirement, parse error at "'{opts}'"

The current workaround is not to use expansion, duplicate the code and lose the ability to use {opts} and {packages}.

Please note that is impossible to use install_command on this environment because we need to run some commands before we install the package.

[obestwalter]

Hi @ssbarnea thanks for the issue - this feels very familiar though. Did you check if there is already an issue like that open? I seem to remember that this substitution is simply not applied to all keys. Might be easy to fix. Want to have a look?

[obestwalter]

Digging through the code atm to properly understand another issue and look what I found:

https://github.com/tox-dev/tox/blob/2.8.0/tox/config.py#L1092

    # special case: opts and packages. Leave {opts} and
    # {packages} intact, they are replaced manually in
    # _venv.VirtualEnv.run_install_command.
    if sub_value in ('opts', 'packages'):
        return '{%s}' % sub_value

So this is why. Dunno if and when we could/should change that behaviour.

[gaborbernat]

I believe we should manually fix up the commands section here.

[jayvdb]

@ssbarnea e4d0d60 doesnt solve this, but it will probably change the error that occurs.

I did toy with implementing {packages} and {opts} as valid substitutions. It still could be done, but one complication is that the venv API has a VirtualEnv.run_install_command that takes a custom value for {packages} and {opts}, so those need to be replaced outside of the substitution engine if that API is to be retained.

However they could also have valid substitutions inside the substitution engine that are used if install_command is fully resolved, which could be triggered by mentioning it from another variable like your example. opts is a bit difficult as it is a composition of many bits of config, and its importance may have diminishing returns as power users are often now using poetry or some other voodoo as their install_command and dropping {opts} as it is very pip specific.

Implementing only {packages} seems to be worthwhile and feasible. And getting {opts} or {opts:} to be empty when not used in the context of VirtualEnv.run_install_command would also be very easy. So, do you need {opts} for your example to work for your needs?

[ssbarnea]

That is a very old ticket and I am not aware of needing a change anymore. I do not even remember where I needed it so better to close it.