Add a --save option to install ala npm's save option

[wraithan]

What this would do is add the package to the requirements file specified along with the version number of it.

$ cat requirements.txt
old-package==1.0.0

$ pip install new-package --save requirements.txt

# Install logs, installs version 1.5.3 of new-package

$ cat requirements.txt
old-package==1.0.0
new-package==1.5.3

I think it would also be great if it included the dependencies but maybe put a comment in there about them, or maybe that would be another option.

[dstufft]

I agree with this feature, though I don't want it to include dependencies because I want to introduce proper lockfile support at some point :)

[wraithan]

Maybe an additional option to include deps then?

pip install new-package --save requirements.txt --save-deps

or something like that?

[dstufft]

That could make sense, then when we get lockfile support we can deprecate --save-deps.

[wraithan]

I'm 👍 on that.

[kennethreitz]

Is managing environments within the scope of pip? I was always under the impression that the pip team officially decided that pip was only for modifying environments, and something else would need to be built for managing them in a more conclusive fashion.

[dstufft]

I dunno :] Depends on the feature itself I guess. e.g. I wouldn't want pip generating virtualenvs but dealing with the current environment seems reasonable to me. Maybe the other devs feel differently.

[Ivoz]

I've seen at least a few features now, that could make sense with a command extension infrastructure for pip?

E.G install pip-locking and it could give you an extra command to use in pip to do this work. Same with pip [un]zip, and one other I can't quite remember...

[wraithan]

Is managing environments within the scope of pip? I was always under the impression that the pip team officially decided that pip was only for modifying environments, and something else would need to be built for managing them in a more conclusive fashion

pip is the only one that deals in requirements.txt, and can generate them with the pip freeze command, so I don't feel like it is outside of its scope to also append to or update them.

[karan]

This one's a much needed feature! npm has spoiled me for packages.

[techdragon]

Ok, I'm fairly sure this is something where we need a more complex feature than the issue requests.
Saving only into requirements would be the worst way to use a hypothetical save flag i can think of.
I was thinking about this recently due to a blog post ( https://medium.com/@alonisser/things-i-wish-pip-learned-from-npm-f712fa26f5bc ) and I think there are two things a save option must be capable of.

• Specifying how we are saving dependencies, setup.py or requirements.txt
• Specifying dependency groups. Like these https://pythonhosted.org/setuptools/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies ( I am not sure if its possible to do this with requirements.txt files other than making 'requirements-.txt' files )

Having a --save flag that can handle these would pretty much give us parity with the NPM --save option.
If a pull request for all this shows up, is it going to have trouble getting accepted? I have concerns that having pip edit setup.py may be viewed as something we shouldn't be doing because setup.py is python code we execute. But any solution that doesn't use setup.py would fail to solve the problem correctly. NPM's --save option updates the 'package' configuration file. Setup.py is the equivalent file for a python package.

-- Update --
I actually would prefer to not update the requirements.txt file because that would help perpetuate the bad practice of 'install my library by running pip install -r requirements.txt' and doesn't help the packaging story as much as adding this feature for 'only setup.py' would do.

[Ivoz]

The fact that setup.py is executable code makes it rather unsuited for the above wont, I think. An unfortunate fact of history. One thing that makes package.json great is that it is simply static metadata, in... json!

A great aide to reasoning about the ideas in this issue might therefore reside in considering it with requirements 2.0 in mind. Coincidentally it also describes a purely-metadata json format, and greatly expands on a next-generation requirements.txt's abilities.

[Glavin001]

+1 I am also a Node.js developing using npm and I definitely want and expect this in pip. Hopefully this is coming in the future.

[dman777]

+1 on this also

[Glavin001]

Are you accepting Pull Requests for features such as this one? I just finished my exams and hopefully will have some time over the summer to contribute to more open source projects. If there is interest -- I know I sure am interested in this feature -- and the maintainers of pip are willing to accept a pull request, then I am open for developing this out. We should discuss ideal implementation, of course, before I begin any coding on this.

@Ivoz mentions requirements 2.0 which looks great, and I am very excited for it.

@techdragon:

I actually would prefer to not update the requirements.txt file because that would help perpetuate the bad practice of 'install my library by running pip install -r requirements.txt' and doesn't help the packaging story as much as adding this feature for 'only setup.py' would do.

I am not an experienced Python developer, however I see a custom file such as setup.py being less preferrable given that it is Python executible code and therefore cannot be easily manipulated by helpful tools like pip. Ideally, we should have a file like requirements 2.0 with a purely-metadata json format. However, this is not the case and does not exist, yet. I look forward to that day when it does. Until then, I do prefer something along the lines of pip install -r requirements.txt or more preferably pip install ( see #2488 ) and hope to bring that into reality.

@wraithan had a good idea with pip install new-package --save requirements.txt --save-deps that addressed your two requirements:

I think there are two things a save option must be capable of.

• Specifying how we are saving dependencies, setup.py or requirements.txt
• Specifying dependency groups. Like these https://pythonhosted.org/setuptools/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies ( I am not sure if its possible to do this with requirements.txt files other than making 'requirements-.txt' files )

Instead of having the simplicity that npm has with npm install --save <package> we can come close and in addition specify the path to the requirements.txt file.

I see pip install <package> --save requirements.txt being sufficient. The argument would be --save <requirements.txt file path> and by specifying different requirement.txt files you could have both normal and dev-dependencies (think --save requirements.txt and --save requirements-dev.txt, for example).

Once something like requirements 2.0 exists, this will all become deprecated and we will have the ideal solution. For now, what do you all think of simply --save <requirements file path> as the new argument?

[p2mbot]

+ 1 for --save

[bekhzod91]

+1 for --save

[19940624]

+1 for --save

[epicserve]

Just my two cents, but I think it would be awesome if --save could use the setup.cfg file. That way you wouldn't have to specify where to save the requirements. I would also make the default destination for --save, CWD/requirements.txt and the default for --save-dev, CWD/requirements-dev.txt

Example setup.cfg:

[pip]
base = config/requirements/base.txt  ; location for --save
dev = config/requirements/dev.txt    ; location for --save-dev

[NickWoodhams]

+1 for --save

[santiagobasulto]

+1 for --save and also --save-dev if possible. (packages=[] and tests_require=[])

[nottrobin]

+1 this would be so helpful

[harobed]

+1

[noelvo]

+1

[pablomaurer]

+1

[yassineAlouini]

+1 as well :)

[noelvo]

This is the PR #2988. There are only --save and --save-dest for now.

[bitzip]

+1

[ghost]

+1 do it

[tino]

How about adding a flag with which pip only outputs the installed packages in requirements format? That would enable one to do:

pip install --output-as-save new-package other-package >> requirements.txt

and saves the copy pasting of the output, and then replacing the "-" with "==" :).
You'd still have to manage upgrades, but that's easy with sorting and git-gutter stuff in the editor.

(Obviously there are probably better candidates for the name output-as-save, just the first thing that came to mind)

[xiemeilong]

+1

[westurner]

• Pipfile, Pipfile.lock
  • | PyPI: https://pypi.python.org/pypi/pipenv
    • | PyPI: https://pypi.python.org/pypi/requirements-parser
    • | PyPI: https://pypi.python.org/pypi/pipfile
  • | Src: https://github.com/kennethreitz/pipenv
  • These save to the Pipfile:
    • pipenv install pkgname
    • pipenv install pkgname --dev
  • https://github.com/kennethreitz/pipenv/blob/master/pipenv/utils.py
    • pip reqs.txt <--> Pipfile

[pradyunsg]

@dstufft @pfmoore Do you think that this should wait until pipfile matures into something or should this be added before that?

[williamboman]

FWIW; I use the following bash function for saving to ./requirements.txt

pip-save ()
{
    for pkg in $@;
    do
        pip install "$pkg" && {
            name="$(pip show "$pkg" | grep Name: | awk '{print $2}')";
            version="$(pip show "$pkg" | grep Version: | awk '{print $2}')";
            echo "${name}==${version}" >> requirements.txt
        };
    done
}

[dstufft]

It should probably wait for pipfile yea.

[jnoortheen]

I created this package pipm exactly for this reason. It can be used only during development and use pip in CI/CD. It just wraps around pip and not another standard.

[mehulmpt]

2018 and a so useful feature still not available! Why?

[RonnyPfannschmidt]

@mehulmpt nobody with time and skill available to make a good one - pip is volunteer driven and on very low manpower

[matkoniecz]

@mehulmpt If you are neither writing it nor hiring other people to write it then you should not complain.

[mehulmpt]

@matkoniecz I was asking, not complaining, it was a genuine question because I wanted to know if there's any specific reason why this is still not there. And I got my answer, no need to get offended.

[williamboman]

2018 and you can't figure out why a feature is not implemented in FOSS! Why?

[bigusef]

I want to make tow requirement file one for development package and another for production
and help and I don't want to use and the package like pipenv or pipm

[westurner]

https://pypi.org/project/requirementslib/ can read and write Pipfile and requirements.txt; but pip does not (yet?) depend upon requirementslib at this time.

https://mail.python.org/archives/list/[email protected]/message/IQVZVVWX2BLEP6D4WQMKNXZHBF2NZINU/

[westurner]

IDK what requirementslib currently does with comments and blank lines?

• ENH: requirementslib: nondestructive editing?
• REF,BLD: pip: depend upon requirementslib
• ENH: pip: add --save and --dev and --save-deps options to pip install and pip upgrade
  • ENH: ~def add_or_update_requirement(req, reqfile)

[westurner]

def add_or_update_requirement(req, reqfile,
    save_hash=True,
    hash=sha256):

?

Should there be a --no-save-hash CLI arg?

[AdamGold]

I have created a small module to automatically insert package versions to setup.py, without me needing to check pip freeze and explictly write the numbers:
https://github.com/AdamGold/pypkgfreeze

Currently working on inserting new packages to setup.py so it will be much more comfortable.

[westurner]

• Are there advantages to having versions (and package hashes) in external requirements files?

• Is there a good way to modify the setup.py AST and write it back out with comments and newlines preserved?

[westurner]

• Setup.py install_requires (currently?) doesn't support at least:
  • package hashes
  • editable packages
  • URLs

[westurner]

Additionally, it’s best practice to indicate any known lower or upper bounds.

It is not considered best practice to use install_requires to pin dependencies to specific versions, or to specify sub-dependencies (i.e. dependencies of your dependencies). This is overly-restrictive, and prevents the user from gaining the benefit of dependency upgrades

https://packaging.python.org/discussions/install-requires-vs-requirements/

[AdamGold]

• Is there a good way to modify the setup.py AST and write it back out with comments and newlines preserved?

Don't know of any way to do that.

• Setup.py install_requires (currently?) doesn't support at least:

  • package hashes
  • editable packages
  • URLs

A possible solution would be for me to make a change in requirements.txt in addition to changing setup.py for supporting a broader variety of projects.

[pradyunsg]

I do not support the idea of modifying setup.py files to do this; none the less, I think this should be answered:

Is there a good way to modify the setup.py AST and write it back out with comments and newlines preserved?

I know of: https://github.com/PyCQA/baron

[jnoortheen]

I wrote https://github.com/jnoortheen/pipm sometime back using setup.cfg as the metadata provider. It works for simple cases, but writing data back to setup.cfg using configparser wasn't stable . With the acceptance of

• https://peps.python.org/pep-0631/
• https://peps.python.org/pep-0621/

pip can manage dependencies too while using requirements*.txt as the lockfile format.