Support Markdown for readmes

[ewdurbin]

Originally reported by: Hickford (Bitbucket: hickford, GitHub: hickford)

---

If you write your readme in Restructured Text, PyPI will render it on your project's page (eg. https://pypi.python.org/pypi/requests ). It should be possible to write your readme in Markdown too. Markdown is a pervasive format used on GitHub and StackOverflow. It shouldn't be necessary to learn a new markup language or rewrite your project's documentation to publish a Python package.

This is common frustration. The Python community suffers because it's a barrier to publishing packages and writing documentation . Please consider supporting Markdown!

• http://stackoverflow.com/questions/10718767/have-the-same-readme-both-in-markdown-and-restructuredtext
• http://i.imgur.com/7NYlimN.png
• https://coderwall.com/p/qawuyq

---

Update 2014: Here's a pull request! https://bitbucket.org/pypa/pypi/pull-request/59/support-markdown-for-readmes-fixes-148/diff

---

• Bitbucket: https://bitbucket.org/pypa/pypi/issue/148

[ewdurbin]

Original comment by Will McKenzie (Bitbucket: kmd_willmckenzie, GitHub: Unknown):

---

I'll second this! Much prefer Markdown syntax and it's a pain having to convert before uploading a package.

[ewdurbin]

Original comment by Donald Stufft (Bitbucket: dstufft, GitHub: dstufft):

---

This is unlikely to happen in the near term. Warehouse (PyPI 2.0) is organized so that eventually we can support multiple markdown selections. The primary problem is we don't have any way to know if we should be rendering in markdown or rST. One way around that would be to render Markdown if rST fails, however some markdown is valid rST but it renders incorrectly in that case and looks really bad. It's also not a very particularly elegant solution especially if people want to do something else like asciidoc or some other formatting method. Likely the metadata 2.0 stuff will (eventually maybe?) support saying what markup your description is in and then we can render it using the appropriate renderer. It will also enable us to actually reject and upload if a package doesn't render correctly instead of just falling back to plaintext, because we'll know if it's supposed to be able to render or not.

[ewdurbin]

Original comment by Hickford (Bitbucket: hickford, GitHub: hickford):

---

Really? I understand PyPI's current algorithm to be

If there's a readme.rst, readme.txt or readme (no extension) file, render that as restructured text. Otherwise, nothing.

Here's a backwards compatible algorithm that would elate markdown authors:

If there's a readme.rst, readme.txt or readme (no extension) file render that as restructured text. Else if there's a readme.md or readme.markdown file render that as Markdown. Otherwise, nothing.

If you're concerned about projects with a readme.txt written in Markdown, they are already broken. This change wouldn't fix that, but it would help the majority of Markdown authors who use the .md or .markdown extensions. They do that because GitHub renders files with those extensions.

[ewdurbin]

Original comment by Donald Stufft (Bitbucket: dstufft, GitHub: dstufft):

---

No, PyPI doesn't exactly do that. PyPI renders a long_description, if a long_description is not provided it will attempt to discover a README file and will insert that as the long_description.

[ewdurbin]

Original comment by Will McKenzie (Bitbucket: kmd_willmckenzie, GitHub: Unknown):

---

A potential interim solution (until the metadata stuff that will let a package maintainer specify the format) could be to have a developer leave the long_description field blank and have a README.(md|markdown) file to be rendered in it's stead? I know with my packages I'm basically just reading the README.txt file and assigning it to the long description field (I didn't know how the algorithm worked).

[ewdurbin]

Original comment by Hickford (Bitbucket: hickford, GitHub: hickford):

---

I had a go! Here's a pull request https://bitbucket.org/pypa/pypi/pull-request/33/support-markdown-for-readmes/diff

[ewdurbin]

Original comment by Marc Abramowitz (Bitbucket: msabramo, GitHub: msabramo):

---

I like Will's idea. Many, many Python projects have boilerplate in setup.py to set long_description to the contents of their README. That's because people want to have the best chance of their description getting picked up but almost everyone prefers editing a description in a separate file over something embedded in setup.py, plus the file extension tells you what format it's in.

I think it would be great if PyPI/warehouse scanned for a README.rst, README.md, README.markdown and used that as the priority for the description to show. Perhaps setuptools could be modified to warn you in the unlikely event that your long_description has different data than your README (@jaraco: What do you think?).

I think in the next version of the metadata standard I would drop long_description and instead offer something like long_description_filename.

Maybe not a bad idea to add long_description_filename to setuptools now. Then package maintainers could start setting that and PyPI/Warehouse could use that in preference to long_description. That would let the server instantly know the format language from the extension and package maintainers could set it and remove lines from their setup.py to populate long_description from a file.

[ewdurbin]

Original comment by Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco):

---

@msabramo I'm not sure your suggestion makes sense in the general case. What you and Will are suggesting is to create a new field in the metadata that references content outside the metadata. In particular, long_description_filename references a file that exists in one of the uploaded distribution packages, but which one? What if the distribution package is a wheel or egg or Windows installer? What if there are multiple distributions (Python version specific)? What if the metadata has been uploaded (e.g. through the register command) but no distribution has been uploaded? I see value in having the long description inline with the metadata in all cases, because that means that all metadata is represented in one entity and doesn't require steps to assemble it (and the complication that entails).

[ewdurbin]

Original comment by Krisztian Fekete (Bitbucket: krisztianfekete, GitHub: Unknown):

---

What about a new field: long_description_format defaulting to reST for backward compatibility, but enabling us to have markdown formatted descriptions?

Lifting a README file into the manifest is a different feature, which can be solved with tooling as demonstrated by pbr.

[ewdurbin]

Original comment by Marc Abramowitz (Bitbucket: msabramo, GitHub: msabramo):

---

@jaraco: Those are good points - I hadn't considered that there could be multiple distributions or even no distribution if the user registered but did not upload. Maybe the idea from @krisztianfekete is a better one?

[ewdurbin]

Original comment by Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco):

---

I have no objection to supporting markdown or even moving toward preferring markdown. This poses a larger question of what should the metadata reflect to support this ability? Also, does the tool chain that renders documentation support markdown (namely Sphinx or Warehouse)? I'd recommend getting some feedback from pypa-dev on a change of this scope.

[ewdurbin]

Original comment by Marc Abramowitz (Bitbucket: msabramo, GitHub: msabramo):

---

@hickford's patch would add Markdown to legacy PyPI. I'm thinking it could be tweaked a bit to look for a long_description_format property in the metadata and a similar change could be made to Warehouse (GitHub).

Then we could make a setuptools extension to add that property to package metadata (use an extension so we don't have to modify setuptools unless it proves to be wildly popular).

[ewdurbin]

Original comment by illume NA (Bitbucket: illume, GitHub: illume):

---

Added an issue to devpi https://bitbucket.org/hpk42/devpi/issue/193/support-markdown-for-readmes

[ewdurbin]

Original comment by Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco):

---

I'm slighly -1 on adding another metadata field to specify the content type of an existing metadata field (i.e. long_description/long_description_format). If we did adopt that, I would recommend using MIME types (so something like "text/x-markdown; charset=utf-8" for markdown) to encourage alignment with other standards.

Rather than storing the metadata for the long description in another field, perhaps the format could be inferred by a heuristic and specified for cases where the heuristic fails using out-of-band features of the format (i.e. comments). For example, the following two snippets could represent a way to specify the content type of the document:

.. Content-Type: text/x-rst

[//]: # Content-Type: text/x-markdown

In this way, the long_description field becomes content-agnostic, but the tools now can infer the content type and users would have a means to force a content type.

This approach would not require any changes to tooling except for those that render package metadata.

[ewdurbin]

Original comment by Marc Abramowitz (Bitbucket: msabramo, GitHub: msabramo):

---

@jaraco: Very interesting ideas to use MIME types and to attempt to deduce the format without adding metadata. @hickford: Want to update your PR?