Can't use twine unless it is on PATH

[brettcannon]

I execute Twine from an unactivated virtualenv (or at least I used to until this change occurred). Probably no explicit reason to change and I can just change my habit, but thought I should put it here in case anyone else has issues.

> ../py34_venv/bin/twine upload *
Traceback (most recent call last):
  File "../py34_venv/bin/twine", line 11, in <module>
    sys.exit(main())
  File "/Users/bcannon/Repositories/caniusepython3/py34_venv/lib/python3.4/site-packages/twine/__main__.py", line 24, in main
    return dispatch(sys.argv[1:])
  File "/Users/bcannon/Repositories/caniusepython3/py34_venv/lib/python3.4/site-packages/twine/cli.py", line 41, in dispatch
    p = subprocess.Popen(["twine-{0}".format(args.command)] + args.args)
  File "/Users/bcannon/Repositories/cpython/default/Lib/subprocess.py", line 850, in __init__
    restore_signals, start_new_session)
  File "/Users/bcannon/Repositories/cpython/default/Lib/subprocess.py", line 1448, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'twine-upload'

[dstufft]

Hm, I don't think twine would have ever worked like that, it uses git style dispatching to sub commands.

You can just do ../py34_venv/bin/twine-upload * though

[merwok]

Out of curiosity, why git-style many-programs distpatch instead of argparse subparsers?

[dstufft]

At the time I had this idea that twine would turn into this cool little utility for doing all sorts of packaging things and that people would want to make their own twine plugins. So I used the git-style dispatch to make that easier. Turns out that was probably a dumb idea in general and I don't see it getting more than maybe one other command for register.

[merwok]

:‑) I like the idea of separate tools: distutils to build stuff, distutils and wheel to create distributions, tox to test them, twine to upload, pip to download and install.

[dstufft]

tbh I've been thinking of moving twine into pip heh, Feels like it'd be easier to just do pip publish to push a package to PyPI.

[merwok]

distutils register and upload are not going away though, and I have bugs to fix and things to add to them. It may be better to avoid too many ways to do the same thing.

Vaguely related: I wonder if upload_docs should be added to distutils.

[dstufft]

distutils register/upload need to be strongly discouraged if not outright removed/broken. There is a massive corpus of them out there that don't even use HTTPS even without validation, they are dangerous.

[merwok]

Well it’s as bad as any other security bug in Python / stdlib. It’s handled the same way: bug reports, patches, new releases.

[dstufft]

No, it's worse because of the nature of it.

• Distutils does not let you upload a file that you've already created, so if you support older versions of Python that no longer get security updates from upstream you're forced to send your password plaintext if you use distutils.
• It's impossible to fix in 2.x without backporting ssl verification and the cert loading APIs (or bundling certificates itself) which python-dev has said they won't do. So 2.x versions will always be vulnerable.
• Unlike the typical security vulnerability which affects your own machines, this one is trivial to daisy chain to compromising the entire Python ecosystem which uses PyPI if you get moderately lucky.

[techtonik]

@merwok - I wish PSF took security issues more seriously and paid bonuses to people who discover the stuff (or at least crediting them) - http://bugs.python.org/issue12226

I like pip publish idea.

[merwok]

PSF is not responsible for Python development.

[techtonik]

@merwok then why it owns it?

[merwok]

PSF owns the legal right as a representative of the authors and users of Python, but it does not decide on development matters.

[mmerickel]

I tried out twine because I saw @dstufft recommending its use, but it's completely broken when used outside of the path (even just env/bin/twine upload -h fails horribly). It took me a while to track down the issue as well. Are there plans to fix this or is this tabled until some sort of pip publish is added?

On a side-note my https://github.com/mmerickel/subparse package supports adding subcommands via entry points instead of path-based dispatch and just uses argparse.

[techtonik]

Seems like this commit is guilty c2b43ce. While git style dispatching to sub commands suxx (because it is impossible to exchange data with plugins this way), entrypoints are also awkward (because use too many hacks to get what can be better known as lookup registry or shared lookup database), it is still possible to preserve both ways - and get back some parts from the aforementioned commit.

I'd avoid complications at all and just list all files in commands package as subcommands.

[brettcannon]

If I wrote a PR to use argparse or something to handle the argument dispatch so that executing twine didn't require a specific $PATH would it be accepted? Or would you rather see the work go towards making pip publish happen (I personally would love to have pip become the "CLI of PyPI")?

[dstufft]

I would accept a PR to use argparse, I think it's safe to say the git style thing failed for twine. I've been waiting on a chance to define an "upload 2.0" API before I created a pip publish command (though that's just my personal opinion, I think the old upload API isn't particular good).

[brettcannon]

@dstufft OK, I'll see if I can free up some time to get you a PR.

[sigmavirus24]

Alternatively, we could use entry-points to keep the extensibility of the tool while fixing this use-case. Thoughts? If there are no objections, I'll tag this for 1.4 and try to work on it this week if no one beats me to it.

[dstufft]

I'm perfectly happy with entry-points as well.

[brettcannon]

Fine by me and I won't be you to it, @sigmavirus24 =)