Merge pull request #411 from westford14/westford14/codespell-nb

Author: drbenvincent

.github/workflows/ci.yml

@@ -25,8 +25,14 @@ jobs:
         run: pip install -e .[test]
       - name: Run doctests
         run: pytest --doctest-modules --ignore=causalpy/tests/ causalpy/
+      - name: Run extra tests
+        run: pytest docs/source/.codespell/test_notebook_to_markdown.py
       - name: Run tests
         run: pytest --cov-report=xml --no-cov-on-fail
+      - name: Check codespell for notebooks
+        run: |
+          python ./docs/source/.codespell/notebook_to_markdown.py --tempdir tmp_markdown
+          codespell
       - name: Upload coverage to Codecov
         uses: codecov/codecov-action@v4
         with:

docs/source/.codespell/notebook_to_markdown.py

@@ -0,0 +1,71 @@
+#   Copyright 2024 The PyMC Labs Developers
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+"""
+This is a simple script that converts the jupyter notebooks into markdown
+for easier (and cleaner) parsing for the codespell check. Whitelisted words
+are maintained within this directory in the `codespeel-whitelist.txt`. For
+more information on this pre-commit hook please visit the github homepage
+for the project: https://github.com/codespell-project/codespell.
+"""
+
+import argparse
+import os
+from glob import glob
+
+import nbformat
+from nbconvert import MarkdownExporter
+
+
+def notebook_to_markdown(pattern: str, output_dir: str) -> None:
+    """
+    Utility to convert jupyter notebook to markdown files.
+
+    :param pattern:
+        str that is a glob appropriate pattern to search
+    :param output_dir:
+        str directory to save the markdown files to
+    """
+    for f_name in glob(pattern, recursive=True):
+        with open(f_name, "r", encoding="utf-8") as f:
+            nb = nbformat.read(f, as_version=4)
+
+        markdown_exporter = MarkdownExporter()
+        (body, _) = markdown_exporter.from_notebook_node(nb)
+
+        os.makedirs(output_dir, exist_ok=True)
+
+        output_file = os.path.join(
+            output_dir, os.path.splitext(os.path.basename(f_name))[0] + ".md"
+        )
+
+        with open(output_file, "w", encoding="utf-8") as f:
+            f.write(body)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "-p",
+        "--pattern",
+        help="the glob appropriate pattern to search for jupyter notebooks",
+        default="docs/**/*.ipynb",
+    )
+    parser.add_argument(
+        "-t",
+        "--tempdir",
+        help="temporary directory to save the converted notebooks",
+        default="tmp_markdown",
+    )
+    args = parser.parse_args()
+    notebook_to_markdown(args.pattern, args.tempdir)

docs/source/.codespell/test_data/test_notebook.ipynb

@@ -0,0 +1,31 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(f\"{os.__file}__\")\n",
+    "\n",
+    "# Speling mistake."
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/source/.codespell/test_notebook_to_markdown.py

@@ -0,0 +1,43 @@
+#   Copyright 2024 The PyMC Labs Developers
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+"""Notebook to markdown tests."""
+
+import os
+from tempfile import TemporaryDirectory
+
+import pytest
+from notebook_to_markdown import notebook_to_markdown
+
+
+@pytest.fixture
+def data_dir() -> str:
+    """Get current directory."""
+    return os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data")
+
+
+def test_notebook_to_markdown_empty_pattern(data_dir: str) -> None:
+    """Test basic functionality of notebook_to_markdown with empty pattern."""
+    with TemporaryDirectory() as tmp_dir:
+        pattern = "*.missing"
+        notebook_to_markdown(f"{data_dir}/{pattern}", tmp_dir)
+        assert len(os.listdir(tmp_dir)) == 0
+
+
+def test_notebook_to_markdown(data_dir: str) -> None:
+    """Test basic functionality of notebook_to_markdown with a correct pattern."""
+    with TemporaryDirectory() as tmp_dir:
+        pattern = "*.ipynb"
+        notebook_to_markdown(f"{data_dir}/{pattern}", tmp_dir)
+        assert len(os.listdir(tmp_dir)) == 1
+        assert "test_notebook.md" in os.listdir(tmp_dir)

docs/source/knowledgebase/quasi_dags.ipynb

@@ -104,7 +104,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "This leads us to Randomized Controlled Trials (RCTs) which are considered the gold standard for estimating causal effects. One reason for this is that we (as experimenters) intervene in the system by assigning units to treatment by {term}`random assignment`. Because of this intervention, any causal influence of the confounders upon the treatment $\\mathbf{X} \\rightarrow Z$ is broken - treamtent is now soley determined by the randomisation process, $R \\rightarrow T$. The following causal DAG illustrates the structure of an RCT."
+    "This leads us to Randomized Controlled Trials (RCTs) which are considered the gold standard for estimating causal effects. One reason for this is that we (as experimenters) intervene in the system by assigning units to treatment by {term}`random assignment`. Because of this intervention, any causal influence of the confounders upon the treatment $\\mathbf{X} \\rightarrow Z$ is broken - treamtent is now solely determined by the randomisation process, $R \\rightarrow T$. The following causal DAG illustrates the structure of an RCT."
    ]
   },
   {

docs/source/notebooks/ancova_pymc.ipynb

@@ -222,7 +222,7 @@
     "## Run the analysis\n",
     "\n",
     ":::{note}\n",
-    "The `random_seed` keyword argument for the PyMC sampler is not neccessary. We use it here so that the results are reproducible.\n",
+    "The `random_seed` keyword argument for the PyMC sampler is not necessary. We use it here so that the results are reproducible.\n",
     ":::"
    ]
   },

docs/source/notebooks/did_pymc.ipynb

@@ -148,7 +148,7 @@
     "## Run the analysis\n",
     "\n",
     ":::{note}\n",
-    "The `random_seed` keyword argument for the PyMC sampler is not neccessary. We use it here so that the results are reproducible.\n",
+    "The `random_seed` keyword argument for the PyMC sampler is not necessary. We use it here so that the results are reproducible.\n",
     ":::"
    ]
   },

docs/source/notebooks/did_pymc_banks.ipynb

@@ -329,7 +329,7 @@
     "* $\\mu_i$ is the expected value of the outcome (number of banks in business) for the $i^{th}$ observation.\n",
     "* $\\beta_0$ is an intercept term to capture the basiline number of banks in business of the control group, in the pre-intervention period.\n",
     "* `district` is a dummy variable, so $\\beta_{d}$ will represent a main effect of district, that is any offset of the treatment group relative to the control group.\n",
-    "* `post_treatment` is also a dummy variable which captures any shift in the outcome after the treatment time, regardless of the recieving treatment or not.\n",
+    "* `post_treatment` is also a dummy variable which captures any shift in the outcome after the treatment time, regardless of the receiving treatment or not.\n",
     "* the interaction of the two dummary variables `district:post_treatment` will only take on values of 1 for the treatment group after the intervention. Therefore $\\beta_{\\Delta}$ will represent our estimated causal effect."
    ]
   },
@@ -515,7 +515,7 @@
    "source": [
     "## Analysis 2 - DiD with multiple pre/post observations\n",
     "\n",
-    "Now we'll do a difference in differences analysis of the full dataset. This approach has similarities to {term}`CITS` (Comparative Interrupted Time-Series) with a single control over time. Although slightly abitrary, we distinguish between the two techniques on whether there is enough time series data for CITS to capture the time series patterns."
+    "Now we'll do a difference in differences analysis of the full dataset. This approach has similarities to {term}`CITS` (Comparative Interrupted Time-Series) with a single control over time. Although slightly arbitrary, we distinguish between the two techniques on whether there is enough time series data for CITS to capture the time series patterns."
    ]
   },
   {

docs/source/notebooks/geolift1.ipynb

@@ -269,7 +269,7 @@
     "We can use `CausalPy`'s API to run this procedure, but using Bayesian inference methods as follows:\n",
     "\n",
     ":::{note}\n",
-    "The `random_seed` keyword argument for the PyMC sampler is not neccessary. We use it here so that the results are reproducible.\n",
+    "The `random_seed` keyword argument for the PyMC sampler is not necessary. We use it here so that the results are reproducible.\n",
     ":::"
    ]
   },

docs/source/notebooks/inv_prop_pymc.ipynb

@@ -22,9 +22,9 @@
                 "\n",
                 "In this notebook we will briefly demonstrate how to use propensity score weighting schemes to recover treatment effects in the analysis of observational data. We will first showcase the method with a simulated data example drawn from Lucy D’Agostino McGowan's [excellent blog](https://livefreeordichotomize.com/posts/2019-01-17-understanding-propensity-score-weighting/) on inverse propensity score weighting. Then we shall apply the same techniques to NHEFS data set discussed in Miguel Hernan and Robins' _Causal Inference: What if_ [book](https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/). This data set measures the effect of quitting smoking between the period of 1971 and 1982. At each of these two points in time the participant's weight was recorded, and we seek to estimate the effect of quitting in the intervening years on the weight recorded in 1982.\n",
                 "\n",
-                "We will use inverse propensity score weighting techniques to estimate the average treatment effect. There are a range of weighting techniques available: we have implemented `raw`, `robust`, `doubly robust` and `overlap` weighting schemes all of which aim to estimate the average treatment effect. The idea of a propensity score (very broadly) is to derive a one-number summary of individual's probability of adopting a particular treatment. This score is typically calculated by fitting a predictive logit model on all an individual's observed attributes predicting whether or not the those attributes drive the indivdual towards the treatment status. In the case of the NHEFS data we want a model to measure the propensity for each individual to quit smoking. \n",
+                "We will use inverse propensity score weighting techniques to estimate the average treatment effect. There are a range of weighting techniques available: we have implemented `raw`, `robust`, `doubly robust` and `overlap` weighting schemes all of which aim to estimate the average treatment effect. The idea of a propensity score (very broadly) is to derive a one-number summary of individual's probability of adopting a particular treatment. This score is typically calculated by fitting a predictive logit model on all an individual's observed attributes predicting whether or not the those attributes drive the individual towards the treatment status. In the case of the NHEFS data we want a model to measure the propensity for each individual to quit smoking. \n",
                 "\n",
-                "The reason we want this propensity score is because with observed data we often have a kind of imbalance in our covariate profiles across treatment groups. Meaning our data might be unrepresentative in some crucial aspect. This prevents us cleanly reading off treatment effects by looking at simple group differences. These \"imbalances\" can be driven by selection effects into the treatment status so that if we want to estimate the average treatment effect in the population as a whole we need to be wary that our sample might not give us generalisable insight into the treatment differences. Using propensity scores as a measure of the prevalance to adopt the treatment status in the population, we can cleverly weight the observed data to privilege observations of \"rare\" occurence in each group. For example, if smoking is the treatment status and regular running is generally not common among the group of smokers, then on the occasion we see a smoker marathon runner we should heavily weight their outcome measure to overcome their low prevalence in the treated group but real presence in the unmeasured population. Inverse propensity weighting tries to define weighting schemes are inversely proportional to an individual's propensity score so as to better recover an estimate which mitigates (somewhat) the risk of selection effect bias. For more details and illustration of these themes see the PyMC examples [write up](https://www.pymc.io/projects/examples/en/latest/causal_inference/bayesian_nonparametric_causal.html) on Non-Parametric Bayesian methods. {cite:p}`forde2024nonparam`\n"
+                "The reason we want this propensity score is because with observed data we often have a kind of imbalance in our covariate profiles across treatment groups. Meaning our data might be unrepresentative in some crucial aspect. This prevents us cleanly reading off treatment effects by looking at simple group differences. These \"imbalances\" can be driven by selection effects into the treatment status so that if we want to estimate the average treatment effect in the population as a whole we need to be wary that our sample might not give us generalisable insight into the treatment differences. Using propensity scores as a measure of the prevalance to adopt the treatment status in the population, we can cleverly weight the observed data to privilege observations of \"rare\" occurrence in each group. For example, if smoking is the treatment status and regular running is generally not common among the group of smokers, then on the occasion we see a smoker marathon runner we should heavily weight their outcome measure to overcome their low prevalence in the treated group but real presence in the unmeasured population. Inverse propensity weighting tries to define weighting schemes are inversely proportional to an individual's propensity score so as to better recover an estimate which mitigates (somewhat) the risk of selection effect bias. For more details and illustration of these themes see the PyMC examples [write up](https://www.pymc.io/projects/examples/en/latest/causal_inference/bayesian_nonparametric_causal.html) on Non-Parametric Bayesian methods. {cite:p}`forde2024nonparam`\n"
             ]
         },
         {
@@ -832,7 +832,7 @@
             "cell_type": "markdown",
             "metadata": {},
             "source": [
-                "We see here how the particular weighting scheme was able to recover the true treatment effect by defining a contrast in a different pseudo population. This is a useful reminder in that, while propensity score weighting methods are aids to inference in observational data, not all weighting schemes are created equal and we need to be careful in our assessment of when each is applied appropriately. Fundamentally the weighting scheme of choice should be tied to the question of what are you trying to estimate. Aronow and Miller's _Foundations of Agnostic Statistics_ {cite:p}`aronowFoundations` has a good explantion of the differences between the `raw`, `robust` and `doubly robust` weighting schemes. In some sense these offer an escalating series of refined estimators each trying to improve the variance in the ATE estimate. The `doubly robust` approach also tries to offer some guarantees against model misspecification. The `overlap` estimator represents an attempt to calculate the ATE among the population with the overlapping propensity scores. This can be used to guard against poor inference in cases where propensity score distributions have large non-overlapping regions."
+                "We see here how the particular weighting scheme was able to recover the true treatment effect by defining a contrast in a different pseudo population. This is a useful reminder in that, while propensity score weighting methods are aids to inference in observational data, not all weighting schemes are created equal and we need to be careful in our assessment of when each is applied appropriately. Fundamentally the weighting scheme of choice should be tied to the question of what are you trying to estimate. Aronow and Miller's _Foundations of Agnostic Statistics_ {cite:p}`aronowFoundations` has a good explanation of the differences between the `raw`, `robust` and `doubly robust` weighting schemes. In some sense these offer an escalating series of refined estimators each trying to improve the variance in the ATE estimate. The `doubly robust` approach also tries to offer some guarantees against model misspecification. The `overlap` estimator represents an attempt to calculate the ATE among the population with the overlapping propensity scores. This can be used to guard against poor inference in cases where propensity score distributions have large non-overlapping regions."
             ]
         },
         {