Markdown parser doesn't ignore leading whitespace in list items

[stevengj]

julia> Markdown.parse("""
       * A simple list
         split over
         three lines
       """)
    •  A simple list   split over   three lines

As I understand it, indentation matching the initial list item should be ignored, rather than converted into multiple spaces. e.g. here is how Github renders the same list:

• A simple list
  split over
  three lines

In fact, Github seems to ignore any amount of leading whitespace in the list items. On the other hand, Jupyter renders a code block if you indent by an extra four spaces beyond the W+1 spaces for a list marker of width W plus one space:

* A simple list
  split over
  three lines

* A simple list
      with code

is rendered as:

As I understand the commonmark spec, Jupyter's behavior seems the more correct one:

[stevengj]

(I noticed this in #13780.)

[hayd]

Part of the issue is that html ignores multiple whitespaces. IMO we should do the same when rendering markdown in the repl.

Note: You can compare markdown implementations with babelmark2.

Indentation immediately after a list never becomes code in commonmark., so potentially that's a Jupyter bug.

[stevengj]

@hayd, I agree that when rendering non-code cells, we should ignore multiple whitespace. That would fix this issue.

[hayd]

I think this can be patched by regex replacing multiple whitespace here:

function terminline(io::IO, md::AbstractString)
    print(io, replace(md, r"[\s\t\n]+", " "))
end

This feels a little hacky, so let's ping @one-more-minute :)

[hayd]

Pushed a PR to fix as above.

One weird thing (unrelated to my PR, but similar to the example above) is that quote has different behaviour:

julia> Markdown.parse("- a\n b")
    •  a b

julia> Markdown.parse("> a\n b")
  |  a

  b

i.e. the next line doesn't count as part of the quote (it should).

[hayd]

I was looking at the quote part the other day, it seems that quote parse needs to know which characters should start a fresh block... I can't see a way around hardcoding what line starts can escape quote.

Related: I think we ought not allow • instead of - to construct a list, it's not markdown.