fix editable URL parsing / support (hg) URL Schemes: urllib

[westurner]

Defect

Currently, pip fails to parse URLs to editables which contain multiple + signs.

$ pip install -e hg+bb+ssh://pypy/pypy@tip#egg=pypy
Obtaining pypy from hg+bb+ssh://pypy/pypy@tip#egg=pypy
  Cloning hg bb+ssh://pypy/pypy (to revision tip#egg=pypy) to ./test/src/pypy

It would be great if the output would be:

$ pip install -e hg+bb+ssh://pypy/pypy@tip#egg=pypy
Obtaining pypy from hg+bb+ssh://pypy/pypy@tip#egg=pypy
  Cloning hg bb+ssh://pypy/pypy (to revision tip) to ./test/src/pypy

Use Case

As a package developer, I would like to install packages from Mercurial repositories that are addressed with Mercurial "schemes" for brevity and consistency.

The Schemes Extension for Mercurial supports custom URL schemes. [1]

For example, with a scheme defined in .hgrc (hg showconfig | grep '^scheme')

[schemes]
bb+ssh = ssh://hg@bitbucket.org/
example = https://{1}.example.org/hg/

This will clone ssh://hg@bitbucket.org/pypy/pypy

hg clone bb+ssh://pypy/pypy

[1] http://mercurial.selenic.com/wiki/SchemesExtension

Relevant

• Support installing from Git URLs #539 : Support Installing from Git URLs
• url.<base>.insteadOf: http://git-scm.com/docs/git-config
• Add support for hg-git (e.g. pip install -e hg+git+ssh://git@....) #224 : Add support for hg git, which was deemed unworthy due to project scoping?
• Downloads Hashing Command Line and Requirements File #468 , support hash checks for url reqs with hash fragment #735 : URL Fragment Hashes
• https://github.com/qwcode/pip/commit/4377b0fb64f66b3157e78ebcc75d3919f54716aa

Sources

• pip.req.parse_requirements [1]
• tests.test_requirements [2]
• tests.test_install_requirements [3]
• docs/requirements.txt [4]

Are there test cases that demonstrate why Python 2urlparse [5] / Python 3 urllib.parse should or should not be utilized here?

The docstrings for urlparse [6] list a handful of RFCs that may or may not be relevant to describing editable python packages as URLs with

• schemes containing multiple plus signs
  • hg+https://
  • hg+bb+ssh://
  • git+https://
  • git+git://
• fragments containing
  • egg names: #egg=
  • file hashes: #md5=

[1] https://github.com/pypa/pip/blob/develop/pip/req.py#L1291
[2] https://github.com/pypa/pip/blob/develop/tests/test_requirements.py
[3] https://github.com/pypa/pip/blob/develop/tests/test_install_requirement.py
[4] https://github.com/pypa/pip/blob/develop/docs/requirements.txt#L45
[5] http://docs.python.org/2/library/urlparse.html
[6] http://hg.python.org/cpython/file/2.7/Lib/urlparse.py

[westurner]

https://gist.github.com/4347838

[dstufft]

I'm going to close this issue. While it's true that we don't support this, I think it's unlikely to be a good idea to support it. Instructions or files created using this mercurial feature are not going to work on any other developer machines without extra configuration. Further more, in 5 years there has been a single request for it, so I do not believe that the demand is high enough to warrant inclusion.