Add Documentation

[blink1073]

We should use RTD for our narrative and API docs.

[vidartf]

I can put in some cycles here, but don't want to carry it myself. Who else? :)

[blink1073]

I did some experimentation today with sphinx_js.
Here are the issues I faced:

• We have to manually create all of the entries for classes, functions, etc.
• Because of the monorepo, we would need a top level package.json and tsconfig.json that worked for all of the packages. Example output:

Running Sphinx v2.2.1

Using TypeScript 3.7.2 from /Users/steve/miniconda/lib/node_modules/typedoc/node_modules/typescript/lib
Error: Cannot find global value 'Promise'.
Error: /Users/steve/workspace/jupyter/lumino/packages/algorithm/src/array.ts(15)
 Cannot use imports, exports, or module augmentations when '--module' is 'none'.
Error: /Users/steve/workspace/jupyter/lumino/packages/algorithm/src/sort.ts(43)
 Cannot find name 'Set'. Do you need to change your target library? Try changing the `lib` compiler option to es2015 or later.
Error: /Users/steve/workspace/jupyter/lumino/packages/algorithm/src/sort.ts(44)
Error: /Users/steve/workspace/jupyter/lumino/packages/algorithm/src/sort.ts(44)
 Cannot find name 'Map'. Do you need to change your target library? Try changing the `lib` compiler option to es2015 or later.
Error: /Users/steve/workspace/jupyter/lumino/packages/algorithm/tests/src/array.spec.ts(11)
 Cannot find module 'chai'.

I also tried making a self-contained sphinx doc build in one of the packages, but ran into similar issues.

My inclination is that we stick with Typedoc and come up with a workflow (potentially using GitHub actions) that updates a jupyterlab.github.io repo with both lumino docs and JupyterLab docs. I'll bring this up for discussion at the dev meeting on Wednesday.

[jasongrout]

One frustration I have with typedoc is that it doesn't seem to support crosslinks between the packages, at least with our current setup. I would love for the docs of one module that referenced interfaces defined in another module to link to that other module's documentation, for example.

Is there a way to get typedoc to document multiple modules at once to provide this crosslinking? I investigated this for a bit last week, but didn't find a way forward.

[vidartf]

Api-extractor might solve some or all of these issues?

[blink1073]

Thanks @vidartf, I think that is correct. api-extractor can take a folder that contains separately created api.json files and create a cohesive set of .md files, which we can then reference from sphinx.

https://api-extractor.com/pages/setup/generating_docs/

[vidartf]

Some open questions though:

• Will the default markdown render suffice (API docs have internal inter-linking, but no RST refs from outside into it)?
• Can we write some plugin for the markdown renderer to add some recommonmark magic to allow refs to entries from the RST code?
• Should we create an RST documentation writer?

I'm proposing we start with the former, and simple RST links. Then we consider https://api-extractor.com/pages/setup/custom_docs/ as needed.

[blink1073]

I agree it is enough to start with internal inter-linking.

[kinow]

Never used Phosphor, or lumino. But we are evaluating it for our app. So I'll be watching this issue/repo and can help reviewing docs, testing examples, and trying to suggest one thing or two 👍 thanks for keeping phosphor alive.

[vidartf]

Some parts definitely worth considering:

• Project overview/intro page
• Getting started guides (one for widgets, one for application?): How to get a simple example webpage using widgets / an application with plugins up and running ("boiler-plate guide").
• High-level overview of widgets package concepts.
  • Major components (widgets, panels, layouts, and the split in responsibilities between them).
  • What is the role of widgets, what is it not: Not a full controls library (compared to e.g. Qt that does both windows/frames/app/etc. but also comboboxes, checkboxes etc.).
  • How does widgets get added to the DOM?
  • Tie in secondary concepts such as signals, messages, disposables etc, linking to more in-depth docs for those.
  • What are the steps involved in an initial layout or a layout update? (messages, how messages propagate in the DOM tree, who is responsible for the propagation, and what are ways to modify/interfere with it)?
  • Which bits of the logic do a user need to care about if creating a "leaf" widget, vs if they want to create a new panel (i.e. layout widget).
• High-level overview of application package concepts.
  • What problem is it meant to solve?
  • Why does it do it the way it does?
  • Which questions are left open for the application implementer to resolve (e.g. how to get/manage a collection of plugins)?
• Write a "package documentation" for each package. This can maybe be done with a docstring in index.ts files:

  /**
   * @packageDocumentation
   * Lorem ipsum...
   */

  Alternatively, it might be written directly as markdown files.

[bollwyvl]

I had looked at two solutions still in the typedoc camp for another project...

However, if api-generator looks like like the way to go, and has output based on a schema, I've no complaints!

• https://github.com/marcj/typedoc-plugin-lerna-packages
  • the top-level links include the namespace, e.g. @lumino/algorithm
• https://github.com/unstubbable/typedoc-plugin-monorepo
  • top level links are "cleaned", e.g. algorithm

...both of them will do a single-cli-invocation generation of the same structure as the current API, but with an overarching index.html. However, per @jasongrout's gripe, neither of them will link up things like Widget implements IObservableDisposable, even though they're all in scope during the build, which is indeed frustrating. I'm not sure how much deeper into the pipe it's worth going.

I had some earlier success with getting the output of the lerna-based one to work with sphinx_js on another project, but it's quite sensitive to the output of jsdoc/typedoc's JSON output, which both seem to change fairly casually between minor versions, which is hard to stay on top of.

[bollwyvl]

Had a look at the output of api-documentor: it looks like eponymous class/namespace pairs get clobbered, such that widgets.widget.md only contains the namespace contents and not the class information, while things like the Widget constructor expect to be linking to the class. Haven't looked into what it would take to untangle them...

[bollwyvl]

Blunt-force adding the kind to the the filename in MarkdownDocumenter.ts did the trick:

private _getFilenameForApiItem(apiItem: ApiItem): string {
  ...
  return baseName + "." + apiItem.kind.toLowerCase() + ".md";
}

(this yields widgets.widget.class.md and widgets.widget.namespace.md, and back-links work as expected)

It's also possible (though required bringing in a fair amount of machinery from YamlDocumenter) to get links in next to signatures:

    private _writeApiItemPage(apiItem: ApiItem): void {
        ...
        output.appendNode(
          new DocFencedCode({
            configuration,
            code: apiItem.getExcerptWithModifiers(),
            language: "typescript"
          })
        );

        const signatureLinks: DocLinkTag[] = [];

        for (const token of apiItem.excerptTokens) {
          if (token.kind == ExcerptTokenKind.Reference) {
            const specUid: string | undefined =
              token.canonicalReference && token.canonicalReference.toString();
            if (specUid != null) {
              const linkedItem = this._apiItemsByCanonicalReference.get( // copied this from yaml
                specUid
              );
              if (linkedItem != null) {
                signatureLinks.push(
                  new DocLinkTag({
                    configuration,
                    tagName: "@link",
                    linkText: token.text,
                    urlDestination: this._getLinkFilenameForApiItem(linkedItem)
                  })
                );
              }
            }
          }
        }
        if (signatureLinks.length) {
          output.appendNode(
            new DocParagraph({ configuration }, signatureLinks)
          );
        }
       ...

Yielding:

#```typescript
export declare class Widget implements IMessageHandler, IObservableDisposable 
#```
[IMessageHandler](./messaging.imessagehandler.interface.md)[IObservableDisposable](./disposable.iobservabledisposable.interface.md)

It would take some doing (if even possible in markdown) to get syntax-highlighted, linked signatures.

On the whole, it's a pretty impressive piece of work, and it seems like a light fork (or set of PRs) would provide the features we're seeking. An RstDocumenter.ts would certainly create a lot of benefit, specifically for labextensions, but would also be a large amount of work, especially if things needed to exist that weren't already covered by tsdoc.

[kinow]

We are using Vue.js with Lumino, which brings some challenges as Vue.js components work with the Virtual-DOM, but Lumino widgets interact directly with the DOM. We got an example that can be used by other Vue.js devs as reference for one way to integrate Vue.js and Lumino: https://github.com/kinow/vue-lumino

Let me know in case there's any place in the documentation with links for integrations like this 👍

Thanks
Bruno

[blink1073]

Hi @kinow, nice work! I think for now it would be appropriate to add an External Examples section to the README and start with a pointer to your repository.