WIP: Migrate to jupyter-js-services

[jhpedemonte]

Based on 4.x branch.

This works for simple notebooks which don't contain widgets.

Ref #105

[parente]

Reminder for later: read Urth options instead of hardcoding. Also, need to mimic spawn call for container that thebe does ahead of requesting a kernel. (jupyter-js-services assumes the kernel provider container is pre-running, i.e., the notebook server.)

[jhpedemonte]

OK, was finally able to test against both old and new tmpnb impls.

Note the added TODO? block. I did not add support (yet) for checking for existing kernels. Not sure if we want to go down that route or simply always kill the kernel when leaving page (even on reload). @parente, what do you think?

[parente]

I worry that killing when leaving the page is too hard to get right. Can you ensure an XHR is sent and processed with confidence on page unload? The state of XHR available at that point is always questionable in my experience.

Thebe also has some fancy logic for writing a cookie to reuse a spawned container if it still exists, to avoid eating up the pool of containers available for other users. I'm open to ideas different from the the Thebe approach or to just mimicking what Thebe does.

But we need to do something for lifecycle management for sure.

[jhpedemonte]

While we can do page unload actions, not sure about XHR. That might be too much to do at that point, but we could try. But this won't take care of issues where page/browser crashes or immediately quits/closes and doesn't allow unload actions.

Yes, the Thebe cookie logic is what I wrote in the TODO? block in the code.

---

Here's my proposal:

1. Implement something similar to Thebe's cookie logic. The client code will check to see if last used kernel exists and if so, reuse it.
2. Have a server process which periodically looks for "stale" kernels (those whose websockets connections are no longer connected to a browser) and kill those.

I think that combination would work well for (1) cases where user refreshes page (kernel is reused); and (2) user leaves page for extended period of time (kernel is killed; new one started next time user returns).

[jhpedemonte]

@parente: Regarding previous comment, we implemented proposal item 1 (storing container URL if one is created and re-using when reloading page). But we're wondering about how this should work with different notebooks/dashboards.

Let's say we have 2 different dashboards, both using tmpnb.org. We start first dashboard, notices there isn't a stored container URL so it spawns a new container, before starting a new kernel (the container URL is stored).

Later, we start the 2nd dashboard. It finds a stored container URL and therefore uses that one, so it does not spawn before creating a new kernel.

Is this OK, to have 2 different dashboards using same container (but different kernels)?

Thebe recently added a check to see if the set URL is same as stored URL. This won't really work with tmpnb.org, since it redirects to a different subdomain (i.e. https://tmp35.tmpnb.org/user/...) -- seems like that check will always fail and so it will always call spawn.

[parente]

Is this OK, to have 2 different dashboards using same container (but different kernels)?

For now, yes, since we're just trying to implement a lighter-weight Thebe, compatible with tmpnb.

But down the line, each dashboard might have different requirements on the container environment (e..g, libs are available, max memory is allocated, ...) That's where the proposal like jupyter/enhancement-proposals#5 come into play for standardizing an API for requesting kernel environments that a variety of providers can implement.

[parente]

I think we can safely close this. jupyter-incubator/dashboards_nodejs_app has a working implementation tracking bleeding edge jupyter-js-services, jupyter-js-widgets, and so on ...