Native syntax highlighting in dcc.Markdown #562

wbrgss · 2019-06-07T02:10:07Z

This PR removes dcc.SyntaxHighlighter in favor of Markdown-flavoured code syntax.

Example usage (EDIT: import syntax has changed; see below for discussion):

import dash_core_components as dcc

...

    dcc.Markdown(dedent(
    '''
    # Markdown title

    ## Markdown subtitle — highlighted syntax below

    ``` python
    # Describe the layout, or the UI, of the app
    app.layout = html.Div([
        html.H3('Hello World'),
        dcc.Dropdown(
            id='my-dropdown',
            options=[
                {'label': 'Coke', 'value': 'COKE'},
                {'label': 'Tesla', 'value': 'TSLA'},
                {'label': 'Apple', 'value': 'AAPL'}
            ],
            value='COKE'
        ),
        dcc.Graph(id='my-graph')
    ])
    ```
    '''
   # use the dark theme and add styles around the Markdown block
    ), highlight_config={'dark': True}, style={'border-left': '1px solid lightgray'}),

wbrgss · 2019-06-07T02:16:59Z

dash_core_components/_imports_.py

@@ -36,14 +35,13 @@
    "Loading",
    "Location",
    "LogoutButton",
-    "Markdown",
+    "markdown",


@alexcjohnson This isn't very robust, because it must be manually edited after a build. But I'm not sure how to change the auto-generated _imports_.py. I figured it might require a change to dash-generate-components. However, it may not be needed. The dependency, highlight.js, is much smaller than the now-unneeded react-syntax-highlighter.

Hmm yeah that's not great... also as it is, markdown will always be imported anyway so we haven't saved its bytes (aside from what we've already saved by swapping highlight.js in for react-syntax-highlighter). __init__.py isn't auto-generated, right? Seems like we can:

leave _imports_.py and Markdown.py where they go automatically

in __init__.py, filter out Markdown by changing the imports like:

import ._imports_ as _imports __all__ = [] for _name in dir(_imports): if _name != 'Markdown' and not _name.startswith('_'): locals()[_name] = _imports[_name] __all__.append(_name)

add a markdown.py that looks exactly like your markdown.__init__.py

Then, to use markdown.py and include its dist files, you have to do from dash_core_components.markdown import Markdown

One problem with this is if people make a mistake and import dash_core_components.Markdown it won't throw an error in python, it'll just fail to render. I suppose we could at least make the if (!window.hljs) block throw a clear error telling you how to import the component correctly? (same for react-markdown if we manage to pull that out too)

Similarly if people do dcc.markdown.Markdown('''...'''') after the app's start (e.g. inside a callback), then the dist won't be included right?

Right, that wouldn't work either. OK, so maybe it's premature to try and break this stuff out, too many ways to break it and I don't see a clean way around it. Let's file this under "should be solved by lazy loading" and put Markdown back on par with everything else - ie a regular top-level import whose js deps are always available.

OK, I'll do this in a more conventional way and include react-markdown and highlight.js in the dcc _js_dist. If I do this, it will only add about 20kB. Before, with SyntaxHighlighter, it added 188kB from react-syntax-highlight, and 740kB (!) from highlight.js. Installing highlight.js via NPM is ~533kB 🙀.

The default build of highlight.js is 50kB with support for "24 commonly used languages". My custom build is only 12kB; IIRC I'm only including Python, CSS, Javascript and Markdown support. You can easily create a customized highlight.js here.

I'll suggest a highlight.js with the following languages supported (20kB minified):

CSS

HTTP

JavaScript

Python

Bash

JSON

Markdown

HTML, XML

SQL (?)

"Shell Session" (``` shell ... ```)

Obviously we can always add more later without introducing any breaking changes. Any languages I'm missing?

Oh that's awesome! I'd just add TypeScript, and yes let's keep SQL.

At some point we can make some utilities to pick from a few bundle options for both highlight.js and plotly.js - including perhaps an option to turn both of these off entirely and let you put your own in assets.

And in fact, if react-markdown is a single thing and isn't expected to be configurable, I'd leave it in the main dcc bundle, keep it off window

wbrgss · 2019-06-07T02:17:36Z

src/components/Markdown.react.js

+    /**
+     * Config options for syntax highlighting.
+     */
+    highlight_config: PropTypes.object,


This could be more specific — so far it accepts just one parameter.

Good call to make this an object we can extend later, but yeah, let's make it exact and enumerate its contents. Also:

rather than dark: false | true let's do theme: 'light' | 'dark' so we can add more themes later if we choose.

give it a defaultProps entry of {}, that'll simplify the logic a bit. (You're probably aware of this, but don't put nested values like {theme: 'light'} in a default, that leads to unpleasant consequences when the user overrides part but not all of the prop)

wbrgss · 2019-06-07T02:20:33Z

Closes #559

alexcjohnson · 2019-06-07T03:48:14Z

dash_core_components/markdown/__init__.py

+    {
+        'relative_package_path': 'highlight.pack.js',
+        'namespace': 'dash_core_components'
+    }


~~Can we get react-markdown into a separate run-from-window file too?~~
Edit: no, leave it, see #562 (comment), didn't realize that one was so small!

alexcjohnson · 2019-06-10T14:09:28Z

src/components/Markdown.react.js

+            // skip highlighting if highlight.js isn't found
+            return;
+        }
+        const nodes = document.querySelectorAll('pre code');


ooh that's too greedy (and potentially slow) - can we restrict it to this component?

Yeah, agreed. querySelectorAll is really slow. I changed it to use a React ref callback combined with getElementsByTagName.

alexcjohnson · 2019-06-10T14:15:45Z

src/components/Markdown.react.js

+
+    /**
+     * User-defined inline styles for the rendered Markdown
+     * Legacy from SyntaxHighlighter prop `customStyles`


Not sure exactly what we used customStyle for previously, but this one will behave a bit differently as it's applied to the container <div>, not the <pre>. Anyway seems reasonable to include it, but let's restrict the docstring to describing what the prop does now, and put a description of its relation to SyntaxHighlighter in our migration guide in the docs.

let's restrict the docstring to describing what the prop does now

OK, done. One thing to keep in mind is that customStyle was one level deeper before; i.e. it was applied to the <Markdown> child of the parent <div>, not the parent <div> itself, as it is now. The thing is, Markdown isn't really a container, more of a mutator. I think there are docs examples with e.g. style={'border-left': 'grey'}.

react-markdown will already natively wrap code blocks in `<code>` and add a language CSS class github style, e.g. with ``` python So to enable syntax highlighting, we simply iterate through the code blocks and tokenize them with highlight.js

Usage: ``` from dash_core_components.markdown import Markdown ... Markdown([ ... ]) ```

This package is >1MB minified (!) https://bundlephobia.com/[email protected]

alexcjohnson

Beauty 💃

alexcjohnson · 2019-06-13T01:44:01Z

not sure why Percy didn't get any snapshots but whatever... merged.

Luvideria · 2019-09-11T18:26:58Z

I was using Syntax Highlighter before, I had

dcc.SyntaxHighlighter(id='click-data1', language='C++',showLineNumbers='true',theme='dark'),

That I cannot really replicate anymore.
I now use

dcc.Markdown(id='click-data1',highlight_config={'showLineNumbers':'true','theme':'dark','customStyles':{'showLineNumbers':'true'}})

and I wrap my text between back quotes but it does not recognize c++, it works with python though.
Also the line numbers don't show up.

In short:
Should I open another issue?
How do I highlight C++ syntax?
How do I display line numbers?

byronz · 2019-09-11T18:45:41Z

How do I highlight C++ syntax?

if you scroll back enough, I think it's not in the initial 20-ish list @wbrgss customized, but we can add C++ syntax if it makes sense.

How do I display line numbers?

I think it's simply not supported by highlightjs https://highlightjs.readthedocs.io/en/latest/line-numbers.html

alexcjohnson · 2019-09-11T18:56:08Z

Thanks @Luvideria - I wasn't sure this got much use outside of our own docs :)

Sounds like we need 2 new issues in fact:

Additional languages: are there others we should add by default? With the caveat that we don't want to include all available languages, that would be too big. So we should also document the process to make & use your own hljs build (https://highlightjs.org/download/#cdns) - also requires that we implement something like Support replacing resources dash#655
Re-implement line numbers as an option.

Are there any other features SyntaxHighlighter had that we should work to reintroduce with Markdown?

damienrj · 2019-10-15T09:05:19Z

Is there a full example available? I am trying to figure out why

app.layout = html.Div([
    cyto.Cytoscape(
        id='graph',
        layout={'name': 'dagre','rankDir': 'LR'},
        elements=elements,
        stylesheet=stylesheet,
        style={'width': '100%', 'height': '450px'}
    ),
    dcc.Markdown(id='tapNodeData-json',highlight_config={'theme':'dark'})
])


@app.callback(Output('tapNodeData-json', 'children'),
              [Input('graph', 'tapNodeData')])
def displayTapNodeData(data):
    

    return """
#### Dash and Markdown
``` python
x = 1
if x == 1:
    # indented four spaces
    print("x is 1.")

isn't working. Everything looks great except no syntax highlighting on the code block, and the dark theme doesn't seem to be doing anything. Other options like adding a border box did work. This is getting rendered in a Juypter notebook which might be the issue.

wbrgss · 2019-10-15T18:31:51Z

@damienrj Your code appears to work for me. You can run a demo online here, which shows that the dark mode, syntax highlighting, and callback works. Are you closing the triple quotes (""") and backticks (```)? Indeed, Jupyter could be the problem as well; I haven't tested it.

Code for the above demo (click arrow to expand)

import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_cytoscape as cyto
from dash.dependencies import Input, Output

import json

app = dash.Dash(__name__)
server = app.server
app.config.suppress_callback_exceptions = True

app.layout = html.Div([
    dcc.Markdown(id='tapNodeData-json',highlight_config={'theme':'dark'}),
    cyto.Cytoscape(
        id='graph',
        elements=[
            {'data': {'id': 'one', 'label': 'Node 1'}, 'position': {'x': 50, 'y': 50}},
            {'data': {'id': 'two', 'label': 'Node 2'}, 'position': {'x': 200, 'y': 200}},
            {'data': {'source': 'one', 'target': 'two','label': 'Node 1 to 2'}}
        ],
    ),
])


@app.callback(Output('tapNodeData-json', 'children'),
              [Input('graph', 'tapNodeData')])
def displayTapNodeData(data):
    return ["""
#### Dash and Markdown
``` python
x = 1
if x == 1:
    # indented four spaces
    print("x is 1.")
` ` ` # spaces to escape backticks for GitHub
# Node Data:
## Click on a node to display the relevant metadata
""",
"""
``` python
{}
` ` `
""".format(json.dumps(data))]

if __name__ == '__main__':
    app.run_server(host='0.0.0.0', debug=True)

Just FYI, in the future, if you wish to get help with your syntax or usage of Dash, please open a new topic in the Community Forum. If you think you've discovered a bug, please open a new issue in this repo. These are more suitable places to discuss this kind of problem than a merged PR.

damienrj · 2019-10-15T20:59:25Z

Thanks, I will do that in the future. I mistakenly posted in here to see if there was more documentation tied to the PR. I will probably go ahead and make a PR to include an example to show the syntax highlighting for general use. Will work on chasing down the issue separately from dash now, thanks again!

wbrgss commented Jun 7, 2019

View reviewed changes

wbrgss changed the title ~~Markdown highlight~~ Native syntax highlighting in dcc.Markdown Jun 7, 2019

alexcjohnson reviewed Jun 7, 2019

View reviewed changes

alexcjohnson reviewed Jun 10, 2019

View reviewed changes

wbrgss added 23 commits June 12, 2019 17:47

Add highlight.js dependencies to module

f5f1e34

Enable syntax highlighting in Markdown

5bcc25b

react-markdown will already natively wrap code blocks in `<code>` and add a language CSS class github style, e.g. with ``` python So to enable syntax highlighting, we simply iterate through the code blocks and tokenize them with highlight.js

Add markdown as submodule

215822f

Usage: ``` from dash_core_components.markdown import Markdown ... Markdown([ ... ]) ```

Initialize _css_dist list in dash_core_components __init__.py

6dbf5f4

Manually specify /markdown module import

de23aaa

🔪 SyntaxHighlighter

a633a20

🔪 Markdown py build

a651971

Add new Markdown py build

84cc7d0

Remove refs to SyntaxHighlighter

00efb08

Modify build step to mv Markdown.py to subdir

4c4f1bb

Fix conditional classes

92f462c

Replace SyntaxHighlighter tests with Markdown + highlight tests

9815007

black

ec943c9

Prettier + eslint const

7bd0390

rm markdown module subdir and repack highlight.js with more lang support

fa30ab7

Don't account for removed subdir in build step

25d097f

Back to normal (generated) imports

91008c3

highlight.pack.js in root package _js_dist

04c36e1

highlight.css in webpack bundle

a0c9511

🔪 react-syntax-highlighter

d45600f

This package is >1MB minified (!) https://bundlephobia.com/[email protected]

highlight.css in vc

ff50895

ref.querySelectorAll -> getElementsByTagName to get <code> nodes

a8ecbde

Remove comment reference to legacy API

c008baa

highlight_config.dark bool -> highlight_config.theme

a3b73e5

wbrgss force-pushed the markdown-highlight branch from dc499ce to a3b73e5 Compare June 12, 2019 21:50

Bad rebase on test_integration

2cb7c9a

alexcjohnson approved these changes Jun 13, 2019

View reviewed changes

alexcjohnson merged commit 1c28303 into master Jun 13, 2019

alexcjohnson deleted the markdown-highlight branch June 13, 2019 01:44

wbrgss mentioned this pull request Jun 13, 2019

Merge Markdown and SyntaxHighlighter, and import separately #559

Closed

alexcjohnson mentioned this pull request Jun 14, 2019

Markdown dedent, and remove containerProps #569

Merged

This was referenced Sep 12, 2019

Additional languages code Markdown #643

Open

Line numbers code Markdown #644

Open

Uh oh!

Native syntax highlighting in dcc.Markdown #562

Native syntax highlighting in dcc.Markdown #562

Uh oh!

Conversation

wbrgss commented Jun 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chriddyp Jun 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wbrgss Jun 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wbrgss commented Jun 7, 2019

Uh oh!

alexcjohnson Jun 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wbrgss Jun 12, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexcjohnson left a comment

Choose a reason for hiding this comment

Uh oh!

alexcjohnson commented Jun 13, 2019

Uh oh!

Luvideria commented Sep 11, 2019

Uh oh!

byronz commented Sep 11, 2019 • edited by alexcjohnson Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcjohnson commented Sep 11, 2019

Uh oh!

damienrj commented Oct 15, 2019

Uh oh!

wbrgss commented Oct 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

damienrj commented Oct 15, 2019

Uh oh!

Uh oh!

wbrgss commented Jun 7, 2019 •

edited

Loading

chriddyp Jun 7, 2019 •

edited

Loading

wbrgss Jun 7, 2019 •

edited

Loading

alexcjohnson Jun 7, 2019 •

edited

Loading

wbrgss Jun 12, 2019 •

edited

Loading

byronz commented Sep 11, 2019 •

edited by alexcjohnson

Loading

wbrgss commented Oct 15, 2019 •

edited

Loading