Description of how to set a project description doesn't mention that you have to use "twine"

[rspeer]

I looked at the page for ConceptNet on the new PyPI, and saw that it didn't have much of a description.

I went to the "manage" page and saw this message: https://imgur.com/a/pSlvUl6

I read the documentation it linked, and read about long_description and long_description_content_type, so I put those in setup.py, incremented the version number, and ran python setup.py sdist upload, like I always have.

Whoops. It turns out that you can't use long_description_content_type unless you're using Twine and have a bunch of other dependencies up to date, so the formatting is still garbage: https://pypi.org/project/ConceptNet/

I eventually learned that Twine is required by Googling a lot and finding it several paragraphs down on someone's blog post: https://dustingram.com/articles/2018/03/16/markdown-descriptions-on-pypi . Yes, the packaging documentation now says to use Twine, but very far from the place that PyPI linked me to -- and I had no reason to know that the command had changed from python setup.py sdist upload or that there was imperfect compatibility with the old command. I don't just re-read the entire packaging guide every week for fun.

The message about how to set your project description should mention Twine, or ideally, PyPI should be able to fix releases created with python setup.py sdist upload.

[rspeer]

Also, this is at odds with the page in the Python 3.6 official documentation about PyPI, called "The Python Package Index", which mentions python setup.py sdist upload and doesn't mention twine:

https://docs.python.org/3.6/distutils/packageindex.html

[brainwane]

@jonparrott could you share your thoughts on this?

[di]

Ultimately the problem here is that distutils doesn't currently support Metadata 2.1. There is a relevant bug report here: https://bugs.python.org/issue33388

There is also some discussion at pypa/packaging-problems#127 which explains that distutils is not likely to gain support for newer Metadata specifications.

In the short term, adding some explanations of the limits of using python setup.py sdist upload to the distutils docs might suffice. In the long term, addressing that packaging-problem in some way is likely the best path towards resolving this confusion.

Since this is not something that can be resolved within the Warehouse project (and since this is sufficiently documented in those other issue trackers) I think we can probably close this issue as a duplicate.

[theacodes]

I wonder if it's worthwhile to duplicate the warning that's on the index for distutils across all of its pages, notably, this part:

This guide only covers the basic tools for building and distributing extensions that are provided as part of this version of Python. Third party tools offer easier to use and more secure alternatives. Refer to the quick recommendations section in the Python Packaging User Guide for more information.

Having that on the index is great, but all too often users find subpages via organic search and never even realize they're reading something that is literally rigged for failure.

[rspeer]

That is addressing the distutils documentation issue, but I think there's still something to be done for the UX of setting a project description on PyPI.

I'm talking about the box that says, for example:

Project description and sidebar

To set the 'ConceptNet' description, author, links, classifiers, and other details for your next release, use the setup() arguments in your setup.py file. Updating these fields will not change the metadata for past releases. See the Python Packaging User Guide for more help.

...where "Python Packaging User Guide" is a deep link into the middle of the documentation that explains the arguments. It makes it sound like you can just add some arguments and upload a new package the way you usually do.

The user is not instructed to re-read the entire Python Packaging User Guide including the parts they think they know, and expecting the user to do so would be unreasonable anyway.

This text box should contain a summary of the actually recommended steps for setting a project description, including uploading with the twine command.

[merwok]

It is accepted practice to link from the Python docs to third-party projects.
In addition to Thea’s idea of including the disclaimer on all pages, a ticket could be opened on bugs.python.org to request adding a link to twine from distutils upload docs.

[rspeer]

I'm starting to think I shouldn't have even mentioned the Python documentation, because the issue I'm reporting is not with the Python documentation. It is with the interface text in Warehouse, which does not provide the user with the information they need to set their project description correctly.

"The user should have read the documentation and the documentation should have been right" is not the solution to this.

[theacodes]

For what it's worth, the notice is already in the Python docs as of 3.6: https://docs.python.org/3.6/distutils/index.html.

I agree with @rspeer here. Our little notice in Warehouse for project description should mention that Twine is basically necessary for full description support. I'll send a PR for that momentarily.

[theacodes]

@rspeer thanks for filing this and helping us figure it out. Let us know if there's more we can do.