DOC: Unable to successfully build docs: AttributeError: module 'pymc' has no attribute ...

[paw-lu]

Issue with current documentation:

I'm trying to build the documentation.

What I do

1. Clone PyMC
2. Check out the commit tagged with v5.0.1
3. Install deps:

    $ python -m pip install --requirement requirements-dev.txt
    $ python -m pip install numpyro
    $ python -m pip install --editable .

4. Run the make doc commands:

   $ make clean
   $ make html

This is all ran via GH actions, and the past setup has worked perfectly for previous versions of PyMC,
it's the upgrade to 5.0.0+ that has given me trouble.

What I get

Problem seems to happen when dimensionality.ipynb is executed.

I get a lot of ModuleNotFoundError errors like this for a lot of different modules:

/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/vi.rst:23: WARNING: autosummary: failed to import NormalizingFlow.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.NormalizingFlow'
* AttributeError: module 'pymc' has no attribute 'NormalizingFlow'
* ModuleNotFoundError: No module named 'NormalizingFlow'
* KeyError: 'NormalizingFlow'
* ValueError: not enough values to unpack (expected 2, got 1)

Full logs are below—but it seem like these attributes are missing, some examples:

• pymc.align_minibatches
• pymc.SymbolicRandomVariable
• pymc.SimulatorRV
• pymc.KullbackLiebler
• pymc.AR1
• pymc.ordered_univariate

Full run logs

Run nox --tags build
nox > Running session clone
nox > gh repo clone pymc-devs/pymc
Cloning into 'pymc'...
nox > cd pymc
nox > gh api '--header=Accept: application/vnd.github+json' /repos/pymc-devs/pymc/releases/latest --jq=.tag_name
nox > git checkout tags/v5.0.1 -b v5.0.1
Switched to a new branch 'v5.0.1'
nox > Session clone was successful.
nox > Running session docs
nox > Creating virtual environment (virtualenv) using python3.10 in .nox/docs
nox > cd pymc
nox > python -m pip install --requirement requirements-dev.txt
nox > python -m pip install numpyro
nox > python -m pip install --editable .
nox > make clean
rm -rf docs/_build/*
rm -rf docs/source/api/generated
rm -rf docs/source/api/**/generated
rm -rf docs/source/api/**/classmethods
rm -rf docs/source/contributing/private_api/generated
rm -rf docs/source/contributing/private_api/**/generated
rm -rf docs/source/contributing/private_api/**/classmethods
rm -rf docs/jupyter_execute
nox > make html
sphinx-build docs/source docs/_build -b html
Running Sphinx v5.3.0
WARNING: Invalid configuration value found: 'language = None'. Update your configuration to a valid language code. Falling back to 'en' (English).
making output directory... done
[autosummary] generating autosummary for: 404.md, PyMC_and_PyTensor.rst, api.rst, api/backends.rst, api/data.rst, api/distributions.rst, api/distributions/censored.rst, api/distributions/continuous.rst, api/distributions/discrete.rst, api/distributions/mixture.rst, ..., learn/consulting.md, learn/core_notebooks/GLM_linear.ipynb, learn/core_notebooks/dimensionality.ipynb, learn/core_notebooks/index.md, learn/core_notebooks/model_comparison.ipynb, learn/core_notebooks/posterior_predictive.ipynb, learn/core_notebooks/pymc_overview.ipynb, learn/core_notebooks/pymc_pytensor.ipynb, learn/usage_overview.rst, learn/videos_and_podcasts.md
WARNING: [autosummary] failed to import pymc.AR1.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'AR1'
* ImportError: 
* ModuleNotFoundError: No module named 'pymc.AR1'
WARNING: [autosummary] failed to import pymc.KullbackLiebler.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.KullbackLiebler'
* AttributeError: module 'pymc' has no attribute 'KullbackLiebler'
* ImportError: 
WARNING: [autosummary] failed to import pymc.NFVI.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'NFVI'
* ModuleNotFoundError: No module named 'pymc.NFVI'
* ImportError: 
WARNING: [autosummary] failed to import pymc.NormalizingFlow.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.NormalizingFlow'
* AttributeError: module 'pymc' has no attribute 'NormalizingFlow'
* ImportError: 
WARNING: [autosummary] failed to import pymc.SimulatorRV.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.SimulatorRV'
* AttributeError: module 'pymc' has no attribute 'SimulatorRV'
* ImportError: 
WARNING: [autosummary] failed to import pymc.align_minibatches.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'align_minibatches'
* ModuleNotFoundError: No module named 'pymc.align_minibatches'
* ImportError: 
WARNING: [autosummary] failed to import pymc.at_rng.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.at_rng'
* ImportError: 
* AttributeError: module 'pymc' has no attribute 'at_rng'
WARNING: [autosummary] failed to import pymc.distributions.transforms.ordered_multivariate.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.distributions.transforms.ordered_multivariate'; 'pymc.distributions.transforms' is not a package
* ImportError: 
* AttributeError: module 'pymc.distributions.transforms' has no attribute 'ordered_multivariate'
WARNING: [autosummary] failed to import pymc.distributions.transforms.ordered_univariate.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.distributions.transforms.ordered_univariate'; 'pymc.distributions.transforms' is not a package
* ImportError: 
* AttributeError: module 'pymc.distributions.transforms' has no attribute 'ordered_univariate'
WARNING: [autosummary] failed to import pymc.math.abs_.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.math.abs_'; 'pymc.math' is not a package
* AttributeError: module 'pymc.math' has no attribute 'abs_'
* ImportError: 
WARNING: [autosummary] failed to import pymc.pandas_to_array.
Possible hints:
* ImportError: 
* ModuleNotFoundError: No module named 'pymc.pandas_to_array'
* AttributeError: module 'pymc' has no attribute 'pandas_to_array'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/pymc/sampling/jax.py:39: UserWarning: This module is experimental.
  warnings.warn("This module is experimental.")
WARNING: [autosummary] failed to import pymc.set_at_rng.
* ModuleNotFoundError: No module named 'set_at_rng'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/pytensorf.rst:6: WARNING: autosummary: failed to import at_rng.
Possible hints:
* ModuleNotFoundError: No module named 'at_rng'
* AttributeError: module 'pymc' has no attribute 'at_rng'
* ValueError: not enough values to unpack (expected 2, got 1)
* KeyError: 'at_rng'
* ModuleNotFoundError: No module named 'pymc.at_rng'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/pytensorf.rst:6: WARNING: autosummary: failed to import pandas_to_array.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.pandas_to_array'
* ValueError: not enough values to unpack (expected 2, got 1)
* ModuleNotFoundError: No module named 'pandas_to_array'
* KeyError: 'pandas_to_array'
* AttributeError: module 'pymc' has no attribute 'pandas_to_array'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/vi.rst:7: WARNING: autosummary: failed to import NFVI.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'NFVI'
* ValueError: not enough values to unpack (expected 2, got 1)
* ModuleNotFoundError: No module named 'NFVI'
* ModuleNotFoundError: No module named 'pymc.NFVI'
* KeyError: 'NFVI'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/vi.rst:23: WARNING: autosummary: failed to import NormalizingFlow.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.NormalizingFlow'
* AttributeError: module 'pymc' has no attribute 'NormalizingFlow'
* ModuleNotFoundError: No module named 'NormalizingFlow'
* KeyError: 'NormalizingFlow'
* ValueError: not enough values to unpack (expected 2, got 1)
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/glossary.md:18: WARNING: glossary term must be preceded by empty line
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/glossary.md:2: WARNING: duplicate label glossary, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/glossary.md
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/guides/Gaussian_Processes.rst:8: WARNING: duplicate label gaussian processes, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/gp.rst
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/installation.md:2: WARNING: duplicate label installation, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/installation.md
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/books.md:2: WARNING: duplicate label books, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/books.md
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/consulting.md:2: WARNING: duplicate label consulting, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/consulting.md

Exception occurred:
  File "/home/runner/work/pymc-dash-docset/pymc-dash-docset/.nox/docs/lib/python3.10/site-packages/myst_nb/core/execute/direct.py", line 50, in start_client
    raise ExecutionError(str(self.path)) from result.err
myst_nb.core.execute.base.ExecutionError: /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/core_notebooks/dimensionality.ipynb
The full traceback has been saved in /tmp/sphinx-err-48k_zbw6.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
nox > Command make html failed with exit code 2
nox > Session docs failed.
nox > Ran multiple sessions:
nox > * clone: success
nox > * docs: failed
make: *** [Makefile:64: html] Error 2
Error: Process completed with exit code 1.

Context

I maintain the Dash docset for PyMC, and try to keep up with the new releases!

Idea or request for content:

No response

[OriolAbril]

None of the ModuleNotFound errors are the reason for the failed doc build. They are logger errors, not actual exceptions, they also happen in our readthedocs build (and have been for quite a while, these are tracked in #5282). From the full logs, it looks like the error is that the dimensionality notebook can't be executed.

We re-execute the notebooks in https://www.pymc.io/projects/docs/en/stable/learn/core_notebooks/index.html with every single commit to main, but we recently modified the sphinx config so that execution errors in notebooks fail the sphinx build, before that there would be errors laying around in the rendered docs which wasn't ideal. See dim notebook for 4.4 release for example (also see how in 5 and 5.0.1 the whole notebook runs without errors though).

My guess therefore is that there is an installation issue when using pip on gh-actions (we use conda on both gh actions and readthedocs) which prevents the notebook from running and it now fails the build.

Potential fixes that come to mind would be

• using conda for the pymc install (you can take a look at https://github.com/pymc-devs/pymc/blob/main/.github/workflows/tests.yml for the config)
• or override the notebook execution error -> sphinx build failure config variable. You could change the nb_execution_allow_errors but it might be a better idea if you do that to directly skip executing notebooks to build faster and to avoid the errors making their way into the built docs. I think you should use the D flag like this: sphinx-build ... -D nb_execution_mode=off. Ref: https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-D

[OriolAbril]

@paw-lu #6781 will update the doc building process. Now, notebooks will only be executed when READTHEDOCS env variable exists. In case any of the above did not fix the issue this should help. It will also update the "build the docs locally" page