docdict and IDEs

[hoechenberger]

While the docdict simplifies our lives and makes documentation more consistent by reducing redundancy, there's also an unfortunate side effect; namely, that IDEs may be unaware of our custom approach to docstrings. I'm often confronted with information like this:

which, really, is just as unhelpful as it can be if I'm not sure about the parameters the function expects.

I thought maybe we could provide some stubs of the documentation with those placeholders expanded; but apparently stubs are merely intended for type information.

Now I was thinking that maybe when preparing a release, we could expand those strings? So that at least the MNE version that most users will be using – a released "stable" version – would have "proper" docstrings?

Or… are there any ways to kind of inject the rendered HTML docs from our website or something? Anyone know about magic of that kind? @drammock maybe?

[drammock]

maybe when preparing a release, we could expand those strings?

That seems like it would fix the issue you're having. I think it would have to be an automated thing that expanded all the docdicts and made a commit, made a release, and then did a revert commit right after the tagged release to put the docdict back in place. Note that this would not fix your issue if you're using latest master in your IDE.

are there any ways to kind of inject the rendered HTML docs from our website or something?

Seems like that's really an issue for the IDE devs, not for MNE-Python devs. We can't control where they're looking for docs, and it appears they're pulling straight from the source code files.

I suppose if there were a more widespread standard for dealing with docstring redundancy, and that standard were already supported by various IDEs, then I'd consider switching from docdict to whatever that mystery other standard is. But off the top of my head I don't know of one.

[agramfort]

ipython does not have this pb. I think it's an IDE fix that is needed here

…

[hoechenberger]

ipython does not have this pb.

Wow, you're right! Great, this gives me something to work with.

[hoechenberger]

I've opened a feature request upstream at Pylance:
microsoft/pylance-release#346

[drammock]

Closing because there's nothing to be done on our end, it seems. @hoechenberger feel free to reopen if you disagree.

[hoechenberger]

Just wanted to let you know that this issue still has not been addressed upstream. If you have a minute, it would be great if you could add a 👍 to microsoft/pylance-release#346 and leave a comment there, to show this issue is of relevance.

[hoechenberger]

I took a step forward here since I don't think we'll ever see a fix in the VS Code Python extension / Pylance. The problem is that Pylance and other language servers only perform static analysis without actually running or importing the code. This makes it impossible for these tools to get access to "expanded" (dynamically generated) docstrings like the ones we're using.

It's gotten so bad that for most MNE functionality, the docstrings are now completely useless to me within the IDE. I have to keep a browser window or an IPython window open to view the docs.

I therefore took the initiative to create a script that generates type stubs for all public functions and classes in MNE and injects (expanded) docstrings into the stubs. These docstrings will then be picked up and displayed correctly by Pylance in VS Code (I didn't test any other tools so far, but @cbrnr told me that it seems to be working nicely in PyCharm as well).

This is what MNE-Python main looks like for me out of the box:

And here with the stubs:

The stubs are created fully automatically. I packaged them such that they can be installed via pip. Hence, no modifications to MNE are required. There is no interference with any MNE-Python code at runtime.

Currently, default parameter values are still missing. That's next on my to-do list.

You can find the stubs and installation instructions at https://github.com/hoechenberger/mne-python-stubs

cc @larsoner @mscheltienne @britta-wstnr @wmvanvliet

[larsoner]

Rather than a separate project if it's fully automated we could add a script to tools/dev/* that we run manually before each release to generate the stubs. (We could do things in a more automated way, too, with a bit more work.) Then there's no need for a separate package. We already have a step like this with generate_codemeta.py.

We could add a little .yml to at least ensure the script runs on every PR (or run it on a schedule). It would be really neat to create an action that runs to recreate the stubs and pushes any changes to an open PR. That would be more work but could be a neat Sunday project for someone who is motivated.

[cbrnr]

This is great! I agree that it should probably be integrated into MNE, since it runs automatically anyway.

I would not run it on every PR though, because it takes a non-trivial amount of time (and resources).

[britta-wstnr]

This is amazing @hoechenberger - and I agree with the other posters that it should be integrated into MNE for (at least) releases. I assume many users use VSCode or PyCharm so this is an important enhancement.

[hoechenberger]

Hello all,

thanks for the positive feedback on the type stubs!

For those who didn't follow the discussion:
I've created "stub files" (.pyi) that contain the "expanded" docstrings for all MNE-Python functions, classes, and methods.

The stubs do not contain any type hints that are not already present in MNE-Python, except for a few that are plainly obvious to stubgen (the tool I used for creating the stubs) and were added automatically.

The goal was to improve the user experience when working in an IDE.

With these type stubs, the VS Code experience changes from this unusable mess:

to this (markup still subject to change):

I am happy that you like it!

I have worked and talked with @cbrnr for many hours in the past couple of days to evaluate what could be a viable move forward from here. I will try to summarize our thoughts below and make a concrete proposal on how to proceed.

Background information

• The stubs are currently optimized for the best experience in VS Code.
• VS Code (or, rather, Pylance / Pyright, which are used by VS Code) cannot handle Numpydoc-style docstrings and basically has no support for reST-based markup.
• Hence, I inject Markdown into the stubs in many places, and remove reST markup in others, in order to make things look better.
• However, PyCharm doesn't support Markdown.
• These stubs would therefore actually degrade the UX for PyCharm users!
• At the same time, PyCharm does support Numpydoc-style docstrings pretty nicely out of the box.
• So there's a conflict: what works well in VS Code doesn't work well in PyCharm, and vice versa.

Issues we're trying to address

• But the biggest issue we set out to solve is, of course, that of the "un-expanded" doccer-based docstrings (e.g., %(verbose) etc.)
• Second-biggest issue: Sphinx tags / blocks often simply disappear in both VS Code and PyCharm. For example, blocks like .. warning::, .. versionchanged:: etc. just don't show up at all!
• The third issue is aesthetics when it comes to Sphinx-based references: reading ~mne.io.Raw is irritating with that leading tilde.

Proposal

• We create type stubs with expanded docstrings from which we remove Sphinx tags / directives that yield hidden blocks and ugly markup (e.g., :class:, ~, .. warning:: etc). These stubs will automatically look good in PyCharm, and provide more useful documentation in VS Code (albeit it won't be rendered nicely in the latter IDE).
• These type stubs live in a package types-mne that is separate from MNE-Python. This allows us to release this package independently of MNE-Python releases. This package should live under the mne-tools org. Having a package separate from mne also avoids unintended interference with the existing .pyi files when automatically creating the stubs.
• While normally such stub packages depend on the package they're providing the stubs for, we could instead make mne depend on this package, so it gets installed automatically when you install MNE-Python.
• VS Code-specific alterations to the docstrings should live in a separate package types-mne-vscode. This is where all the Markdown and emoji fun can happen.

Additional tasks

• We'd also need to evaluate behavior of Spyder and Jupyter Notebook.
• @cbrnr and I will reach out to pyright and inquire about the possibility of supporting Numpydoc (or if a contribution to add support would be appreciated).

Please share your thoughts! 👍

[mscheltienne]

These type stubs live in a package types-mne that is separate from MNE-Python. This allows us to release this package independently of MNE-Python releases. This package should live under the mne-tools org. Having a package separate from mne also avoids unintended interference with the existing .pyi files when automatically creating the stubs.

I'm not entirely sold on the separate package, the only.pyi files are the init for lazy loading right?
And I don't see the need to release separately from MNE, on the contrary it might create some conflicts if you include dev changes?

Besides that, amazing! (and apologies for the formatting, github on phone is not my preferred interface)