Dash Dev Tools

[chriddyp]

In https://github.com/orgs/plotly/projects/3, many of our issues involve Dash's front-end: Front-end error handling, front-end live-DAG, front-end hot-reload.

I'd like to open this issue to discuss an architecture for handling these requirements.

Some framing:

JavaScript Error Handling

• We should expose Dash's front-end errors to the users. Currently, they are only available in the browser's dev tools, which is too hidden for our community.

For example, consider Atom's plugins:

Python Error Handling
Could we use the same architecture that we use for JavaScript errors to display python errors? That way, the user never needs to look at the terminal for errors, they only need to focus on their app.

For example, consider create-react-app:

or werkzeug:

Hot Reloading
If we had hot-reloading, could we build a nice little "control bar" that users could use to configure hot reloading? For example, it could:

• Tell the user if hot-reloading is on or off
• Users could turn hot-reloading on and off
• It could provide build times or "last updated" times

Displaying the DAG
If we wanted to display the DAG to the user, could we place this in the Dash Dev Tools container as well?

---

cc @plotly/dash

[rmarren1]

React 16 introduced Error Boundaries

Error boundaries are React components that catch JavaScript errors anywhere in their child component tree, log those errors, and display a fallback UI instead of the component tree that crashed. Error boundaries catch errors during rendering, in lifecycle methods, and in constructors of the whole tree below them.

If we upgrade to react 16 we can

1. Change the AppProvider to a class definition
2. Add the new componentDidCatch to set state.hasError = true and state.error = error when an error is thrown in the app
3. In the render method of AppProvider, either only render the state.error if state.hasError, else render

<Provider store={store}>
        <AppContainer/>
    </Provider>

Or even render the app with a small error message bar like here or even design something nice like the atom plugin error message above.

I'll see if I can get a proof of concept / see if it really is that easy...

[nicolaskruchten]

+1 for error-boundaries! it makes React development a lot easier and makes working with sometimes-buggy apps a lot nicer: just parts of the app become unusable instead of all of them :)

[valentijnnieman]

Error-boundaries sound great. Would be very interesting to see that in a proof of concept! I'm not sure about the control bar idea - but's that's just my general preference. Don't really like development UI to be on my web application! But what about maybe a browser plugin, like React devTools has?

[bpostlethwaite]

Ditto, I like the error boundary idea.
I share @valentijnnieman concern about the control bar concept. It may add more confusion and maintenance than it is worth and it would have to be done really well --- there is lower hanging fruit to tackle first.

[rmarren1]

Error boundaries work pretty well. I wrapped the <AppProvider /> with a simple <ErrorHandler /> which catches any errors thrown in the app and displays it to the user.

What may pose a difficulty is that React refuses to render any component which is throwing an exception, meaning components would need to be perfect and not throw any errors. Previously, Dash components could be buggy (e.g. throw some errors to console) and stay rendered on the app (e.g. in this example, you can keep clicking the button and it would stay while throwing an error each click). With this kind of update we would either need to completely unmount the app and display an error message, or put error handlers lower down in the component tree.

I think the logic behind not rendering any components which are throwing an error in a lifecycle method is solid, explained here

[rmarren1]

Looking into this further, it looks like Error Boundaries are not perfect since they only catch errors in lifecycle methods, constructors, and render methods. Front-end errors will work fine, but server side 500 errors will not be caught since they originate from event handlers.

There are two workarounds I have found to get both error types in the same place, which are a little hacky but might be worth it.

1. We could have both front-end and back-end dispatch a new error action to the redux store, and a top level component could then listen for these and conditionally display errors or the app. In the front-end, we can catch errors in a top level error boundary and then dispatch the action in the componentDidCatch lifecycle method, and for back-end errors we could chain a catch method to this promise and dispatch form there. Someone did this and has a minimal example.
2. It seems like setProps is always in the stack trace of 500 errors,
   

perhaps we could catch errors here with something like

export function setPropsWrapper (newProps) {
  setProps(newProps).catch(e => this.setState(() => { throw e }));
}

which would push them up into the react error boundary.

[T4rk1n]

The werkzreug one is sent by flask when debug=True but since it's an ajax calls it won't show, maybe we could have the renderer show it when it receives a 500 code from a callback and debug=True ?

@rmarren1 setProps is always in the stack of error 500 since it's the one that initiate the callback request. But it could also happen in /_dash-layout and the other path the renderer sends ajax requests to.

[rmarren1]

True @T4rk1n, I forgot about those other endpoints. I think then that catching errors in the following three fetch calls then dispatching an error can work for the back-end:

1. https://github.com/plotly/dash-renderer/blob/25e4c566afb9d79685ba065e0a99ff4a00195d7a/src/actions/api.js#L7-L15
2. https://github.com/plotly/dash-renderer/blob/25e4c566afb9d79685ba065e0a99ff4a00195d7a/src/actions/api.js#L19-L28
3. https://github.com/plotly/dash-renderer/blob/25e4c566afb9d79685ba065e0a99ff4a00195d7a/src/actions/index.js#L472-L735

In fact the string fetch( doesn't appear anywhere else in dash-renderer, so I think that is all the network communication to the server-side?

[T4rk1n]

@rmarren1 The first two are wrappers, find the usage of those.

https://github.com/plotly/dash-renderer/blob/25e4c566afb9d79685ba065e0a99ff4a00195d7a/src/actions/index.js#L537-L541

Could dispatch the error from there, the response content type should be text/html and if debug=True in dash the error will have a formatted stacktrace, otherwise it be the generic error 500 message. Maybe dash should send a json error instead and the renderer show the same error component for both front-end and back-end errors ?

[chriddyp]

(semi-related to the immediate discussion above. Here's another interesting solution for displaying logs in the browser, it could be a separate endpoint and read from a file: https://github.com/mozilla-frontend-infra/react-lazylog)

[rmarren1]

^^ I like that, looks like CircleCI output

I've been looking into handling things on the Flask side now. We need a way to catch errors in debug mode and do something with them that makes it available to the front end.

The LazyLog component looks great for this (haven't tried this part, but it looks like with the 'stream' attribute set to true we can continuously listen to an error file), then we would only need to serve an error file from Flask and then write to that file when an exception occurs.

This is what is usually done, but this custom handler is only fired when debug=False, the exception is reraised before the custom handler is called in debug mode.

@app.errorhandler(500)
def error(e):

The flask signals module seems like it provides just what we need. (http://flask.pocoo.org/docs/0.12/api/#signals)

Specifically the got_request_exception signal.

This signal is sent when an exception happens during request processing. It is sent before the standard exception handling kicks in and even in debug mode, where no exception handling happens. The exception itself is passed to the subscriber as exception.

I tested this module in Dash and it works nicely 😄
adding this bit:

        @got_request_exception.connect_via(self.server)
        def when_dash_gets_exception(sender, exception, **extra):
            with open('500.log', 'w') as f:
                f.write(traceback.format_exc())

to dash.py makes Dash print all 500's to a '500.log' text file.

I think this is way easier than catching all the exceptions in the front end! I'll see if I can get it working end-to-end, E.g., server-side errors read by LazyLog which triggers some high order component in react to render an error message. Then we just need to feed front-end exceptions caught with the React Error Boundaries into the same rendering mechanism and I think it would work

[rmarren1]

I have a work in progress going in these two pull requests

plotly/dash-renderer#64
#307

Still trying to figure some small quirks out, but it is at a point where it does what it should locally.

Here's how it looks:

[chriddyp]

This is very very cool @rmarren1 !

A few comments:

• Could we try hooking in the werkzeug debugger? We'd probably just write the response data from werkzeug into a div with dangerouslySetInnerHTML similar to https://stackoverflow.com/a/26905426/4142536.
• I'm interested in exploring some different UIs for this. In particular:
  • Styling the tracebacks nicely like create-react-app
  • Maybe moving things to a popup so that it doesn't replace the contents of the screen. Perhaps a way to expand those errors to take up more spaces
  • We have a few designers on contract, I'll check in with them about exploring some different designs
• I think we should turn these on and off through a new set of config arguments rather than using flask's debug for everything. That's a separate topic, I'll elaborate on this here: Dash's config system #312
• Some ideas for the front-end errors:
  • When we start serving un-minified bundles in dev mode, these tracebacks could get pretty awesome Serve unminified JS bundles in dev mode #313
  • Maybe we could include some other information to help folks debug their app. For example, an option to display the store.getState().layout?
  • The front-end errors are usually related to bad props. So once Validation on component properties #264 is solved, we should see a lot less front-end errors. However, it will be really nice for component authors to be able to throw error messages that will actually get propagated to our users.

Finally, re browser extension vs rendering the dev tools within the app: I'm a pretty strong proponent of shipping this dev tools UI as part of the app for a few reasons:

• It's immediately useful and discoverable. If it's a browser extension, our community members have to go and download it, and many or most just won't
• It's in your face! If we have a browser extension, our community members have to open up their browser console. We can't assume that our community members are comfortable or aware of their browser's dev tools
• I'm assuming that writing a dev tools UI would be more work, as we'd have to write it for different browsers. Some users in corporate environments might not even be able to install it on their corporate machine
• With Serve unminified JS bundles in dev mode #313, we could even ship a different version of dash-renderer in "prod mode" that doesn't have the dev mode UI
• If it gets in the way, users can turn it off if they want to

Of course, if it's part of the app, it needs to be really ergonomic. So, let's keep experimenting with different UIs. I'll loop back on this issue about getting some design talent to help us out as well.

[rmarren1]

One point on the UI is that if we are dealing with a front-end error (originating from a React lifecycle method of come component in the app) then the app will unmount by default as of React 16, so in the current solution we couldn't render the app and the error message.

We could include an error boundary around every component in the tree, then figure out a way to bubble caught errors up to a higher-order error handler which will display the message along with the app (excluding the component in the app which caused the error). We might even be able to put a red-box around the space where the buggy component used to be, which would make it super apparent where the error comes from. I'll see if I can do this in the WIP I have.

[chriddyp]

We might even be able to put a red-box around the space where the buggy component used to be, which would make it super apparent where the error comes from. I'll see if I can do this in the WIP I have.

Wow, that would be really sweet

[rmarren1]

Werkzueg works. Actually this may be a better solution than pushing errors from Flask over a socket, and you can overlay it onto a running app. I'll see what more I can do with the front-end errors.

commits for this in https://github.com/rmarren1/dash-renderer/tree/dash-dev-tools

[chriddyp]

Wow! does the werkzeug console work too?

[rmarren1]

I don't think so, it says "If you enable JavaScript you can also use additional features such as code execution (if the evalex feature is enabled), automatic pasting of the exceptions and much more.", probably because I am using dangerouslySetInnerHtml

[rmarren1]

That red box thing works pretty well. We can replace the component with a red-background div or span that inherits the id and className of the component that faulted (so hopefully it will render a component of the same size and in the same position) with debug information it.

[chriddyp]

"If you enable JavaScript you can also use additional features such as code execution (if the evalex feature is enabled), automatic pasting of the exceptions and much more.", probably because I am using dangerouslySetInnerHtml

Yeah, sounds right. It looks like browser's don't support script tags that are inserted through innerHtml (facebook/react#8838) and I suspect that the error response has some <script/> tags embedded in it. Maybe we do something like this instead: facebook/react#8838 (comment)

[T4rk1n]

Maybe embed the html in an <iframe srcdoc="werkzeug error"> ?

[rmarren1]

Played around with a bunch of stuff.

1. iframes do not seem to work. I was getting cross-origin errors so I tried to set the <base> attribute href and parent to be localhost:8050 and _parent, but nothing seemed to work as it seems werkzueg generates the form action URLs itself
2. innerHTML doesn't work because it strips the script tag
3. This mess:

  <img
          style={{"display": "none"}}
          src="http://placehold.it/1x1"
          onLoad={(
            function() {
              document.open();
              document.write(error.errorPage);
              document.close();
            })()}
  />

surprisingly works perfectly, Werkzueg console and all. The only bad part is it takes over the entire screen so we can't embed it over the app, but we might be able to fix that in the future.

[T4rk1n]

I don't mind it taking the whole screen, an error still has to be fixed and it is in your face so you can't ignore it.

[mkhorton]

@chriddyp how did you generate the DAG graphic in screenshot shown above? Seems super useful

[nicolaskruchten]

@mkhorton it came from https://github.com/nicolaskruchten/dash_callback_chain although that repo contains just a proof-of-concept and hasn't been tested with newer versions of Dash :)

[rmarren1]

Made another pre-release at dash-renderer==0.14.0-rc3
plotly/dash-renderer#64

[nite]

Related redux devtools for inspiration: https://github.com/reduxjs/redux-devtools

[chriddyp]

Here’s a mock up of what “dash dev tools” could look like. Slides 2-6 are just the werkzeug debugger themed with CSS. Slides 7-16 are the front-end error messages and the general container for other dev-tools things that we might add (like the DAG graph, hot reloading options one day maybe, etc). would love any and all feedback.
https://www.figma.com/proto/icviajRVLhC69Ud0OW7dKe/dev-tools?node-id=162%3A282&redirected=1&scaling=min-zoom

[chriddyp]

Front-end error messages

Werkzeug debugger

Container for other stuff

Like the callback graph

More mocks here: https://www.figma.com/proto/icviajRVLhC69Ud0OW7dKe/dev-tools?node-id=162%3A2135&redirected=1&scaling=min-zoom

[brylie]

Woah, the callback graph would make it a lot easier to reason about complex Dash apps!

[chriddyp]

Mostly finished in plotly/dash-renderer#100. The only exception is the Python debugger. Still a good idea, but we should discuss in a separate issue if we get the chance to go forward with it.