How to pass arguments to pip install?

[alegonz]

Hi there, thanks for this cool tool :)

Is there any way to pass additional arguments to the pip install that installs the requirements files? Specifically, my workflow involves dependencies that are in a private repository so I need to pass --extra-index-url.

Looking at the source it seems it is not possible at moment. Perhaps something like this could do the job?

REQUIREMENTS_TXT?=requirements.txt  # Multiple paths are supported (space separated)
PIP_INSTALL_ARGS?=
...
ifneq ($(_REQUIREMENTS),)
	$(VENV)/pip install $(foreach path,$(_REQUIREMENTS),-r $(path)) $(PIP_INSTALL_ARGS)
endif
...

If the above is sensible I can throw a PR :)

[sio]

You're correct that there is no such functionality in Makefile.venv and your changes look like they'd work. But I'm not sure we should add this feature to our tool.

Is there some reason you don't want to provide --extra-index-url within your requirements.txt or via PIP_EXTRA_INDEX_URL environment variable? I think that would feel more natural to most Python developers.

There is nothing inherently wrong with the approach you suggested, but I think it would encourage people to create flakier workflows that require others to relearn how familiar things are done for no good reason.

[alegonz]

Is there some reason you don't want to provide --extra-index-url within your requirements.txt or via PIP_EXTRA_INDEX_URL environment variable? I think that would feel more natural to most Python developers.

I cannot add the URL to the requirements.txt file since the URL contains the authentication token and the requirements file is managed with git. I wasn't aware of PIP_EXTRA_INDEX_URL, though, thanks for the pointer (I'm not sure whether everyone is familiar with this environment variable since pip's documentation is rather obscure on this regard).

There is nothing inherently wrong with the approach you suggested, but I think it would encourage people to create flakier workflows that require others to relearn how familiar things are done for no good reason.

How could it become flakier and unfamiliar? It is following the same pattern as REQUIREMENTS_TXT.

[sio]

By flakiness I meant that it would be another intermediate step that would do nothing but could potentially break something, however unlikely.

As for the familiarity, I do not consider my tool as essential to Python developers as pip. Therefore I estimate there are a lot more people familiar with pip than there are those familiar with Makefile.venv. It would be impolite to burden the majority of potential users with learning documentation for a feature that does nothing but obfuscates PIP_EXTRA_INDEX_URL into a different variable.

Could you test if PIP_EXTRA_INDEX_URL works for you? I'm not at my computer right now.

If it works I think there is no need to modify Makefile.venv. We should probably add a hint to README to compensate for pip documentation (hopefully temporary) deficiency.

[alegonz]

I see, thanks for clarifying.
I'll give the environment variable a try and let you know (later as I'm AFK at the moment). If it does, a hint in the README is a good idea 👍

[alegonz]

Just tried the environment variable, it works fine. Thank you 👍

(While in my use case it is fine as I only need one extra-index URL, it seems the environment variable does not support specifying multiple URLs (according to this article, haven't tried it myself)).

[sio]

Thank you for confirming that it works! I'll add a note about PIP_* variables to README.

I think multiple values could be supplied via one environment variable, space separated: PIP_EXTRA_INDEX_URL="https://url1 https://url2". I have not tested that though, and official docs are unfortunately lacking in that regard. If you can test that please leave a comment here.