"--package-cache"

[qwcode]

this was discussed in broad terms at pycon with @dholth and @jezdez

here's the text from a gist I posted months ago. reposting it as a real issue to get discussion at some point. not sure when this would land in an actual milestone to be worked on.

---

"--package-cache"

• a new "pip install" config option that offers name-based (i.e. not url-based) caching.
• the "--download-cache" option would stay as it is, and not be altered
• the package-cache dir would just contain packages (e.g. "nose-1.2.tar.gz", or lxml-2.5[..].whl), anything you've installed while using it.
• the package-cache would be turned off by default, since the user assumes a certain amount of risk/responsibility in using it (no hash checking with what pypi lists)
• the package-cache (if turned on) would be checked prior to using the PackageFinder (which searches pypi and findlinks)
  (if the req was "nose", any nose version in the package-cache would "win", unless --upgrade is specified)
• package-cache offers an alternative to this workflow:
  • pip install -d DIR PACKAGES
  • pip install --no-index --find-links=DIR PACKAGES
• "--package-cache" could be used with "--download-cache", but to no advantage most of the time. the only case I can imagine it would help:
  • you've never used "--package-cache" before, but you have a download-cache
  • the first attempt at using package-cache would benefit from an existing download-cache.
• a "pip install --build-wheel" option could be created that "pip wheel"'s installed packages to the package cache if set.

[qwcode]

a good mailing list thread on the --download-cache a while back with comments from me and carl
https://groups.google.com/d/msg/python-virtualenv/J92cKpftSjA/i9zSae8zV3AJ

[ptone]

+1, I think this becomes even more useful with wheel support.

I think it would be useful when using download cache to have packages get moved (with new name scheme) from download-cache -> package-cache, but if both are enabled, only store new downloads in package-cache.

Also would be good to have a "prune-cache" command that can go through and delete older versions - of course you may want to keep older versions, but the prune option can just cover the common case.

[harobed]

a good mailing list thread on the --download-cache a while back with comments from me and carl
https://groups.google.com/d/msg/python-virtualenv/J92cKpftSjA/i9zSae8zV3AJ

In this thread, I'm agreed with Marcus Smith proposal.

[harobed]

the package-cache dir would just contain packages (e.g. "nose-1.2.tar.gz", or lxml-2.5[..].whl), anything you've installed while using it.

Will a wheel archive be created for all installed package ?

[qwcode]

Will a wheel archive be created for all installed package ?

not the way this was written up originally (this was pre-wheel thinking).
but I just added another bullet to the description for that idea.

[harobed]

Have you already started to work on implementation of this feature ?

[harobed]

@qwcode ping

[qwcode]

no

[harobed]

Ok, I'll may be work on this feature…

[ngrilly]

This feature --package-cache, especially combined with --build-wheel, would be a very useful addition to pip, and ease a lot of deployment scenario.

[Tinche]

A thousand times yes! This will also make multi-project handling easier.

How about a way of installing a package from the file system into the cache? For example, after building a wheel with 'python setup.py bdist_wheel' I'd like to be able to add the generated wheel to the cache, either automatically (controlled via feature) or manually (custom pip command?). I guess 'pip install -e . --build-wheel' could be used to accomplish this using the last bullet point.

[qwcode]

@Tinche see #1572 "Internal Wheel cache", which replaces this idea for the most part.

[dstufft]

Going to close this, as it's now done(ish)!