Skip to content
Open
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
54f23e3
Add download button for Jupyter notebooks in tutorial pages
Sep 18, 2025
ff55d5a
Refactor Quarto rendering and remove download notebook script
Sep 22, 2025
8dd8fbf
Merge branch 'main' into add_notebook_dl
AoifeHughes Sep 22, 2025
091e1a1
Update to Turing 0.40
mhauru Aug 13, 2025
fc4df94
Update version in _quarto.yml
mhauru Aug 13, 2025
df1f9cd
Fix DPPL 0.37 run_ad change
mhauru Aug 13, 2025
14973ad
Fix VI tutorial
mhauru Aug 20, 2025
06bb33d
Fix model-manual's use of contexts
mhauru Aug 21, 2025
e7519c9
Fix references to __context__
mhauru Aug 21, 2025
4530247
Fix use of addlogprob for log prior
mhauru Aug 21, 2025
57c3162
Fix typo
mhauru Sep 16, 2025
2065a14
Regenerate manifest
mhauru Sep 16, 2025
2babb2d
Remove version pin of DelayDiffEq and update Manifest
mhauru Sep 18, 2025
b41a4cb
Fix call to evaluate
mhauru Sep 18, 2025
425539d
Add note about contexts tutorial being out of date
mhauru Sep 19, 2025
dde0ed3
Apply suggestions from code review
mhauru Sep 22, 2025
a32ec1b
Add ipynb format support
Sep 24, 2025
1e7f351
maybe added links to dl?
Sep 24, 2025
e455881
Merge branch 'add_notebook_dl' of https://github.com/TuringLang/docs …
Sep 24, 2025
d1bacac
reset mainifest
Sep 24, 2025
a78fac4
dont execute ipynb
Sep 24, 2025
98bf999
bump local changes
Sep 25, 2025
60a07a8
Merge branch 'main' into add_notebook_dl
AoifeHughes Sep 26, 2025
af7ceb4
add post-render script for converting .quarto_ipynb files to .ipynb
Sep 26, 2025
4459016
Merge branch 'add_notebook_dl' of https://github.com/TuringLang/docs …
Sep 26, 2025
7f00453
Refactor notebook generation process and clean up tutorial metadata
Sep 29, 2025
708d22b
Merge branch 'main' into add_notebook_dl
AoifeHughes Sep 29, 2025
64c62a4
please work :(
Sep 29, 2025
1c6fad9
Merge branch 'add_notebook_dl' of github.com:TuringLang/docs into add…
Sep 29, 2025
26cdf37
freeze?
Sep 30, 2025
01e6954
last try
Sep 30, 2025
f55de5c
forgot to add links...
Sep 30, 2025
848dc87
grep...
Sep 30, 2025
fdbee40
fixed edit loc
Sep 30, 2025
2a56d63
placement
Sep 30, 2025
9ed76f1
added a script to do notebooks justice
Sep 30, 2025
27b9d8b
posix
Sep 30, 2025
e06a333
Merge branch 'main' into add_notebook_dl
AoifeHughes Oct 1, 2025
2a396e4
patched to insert pkg local usage over global.
Oct 6, 2025
3a1691e
removed specifying versions
Oct 6, 2025
e7979d9
Merge branch 'main' into add_notebook_dl
AoifeHughes Oct 6, 2025
d07ab29
made variable thingy in script.
Oct 6, 2025
c00b84d
Add environment variable for Colab path prefix in notebook link scripts
Oct 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ jobs:
- name: Render Quarto site
run: quarto render

- name: Generate Jupyter notebooks
run: sh assets/scripts/generate_notebooks.sh

- name: Add notebook download links to HTML
run: sh assets/scripts/add_notebook_links.sh

- name: Save _freeze folder
id: cache-save
if: ${{ !cancelled() }}
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,12 @@ jobs:
- name: Render Quarto site
run: quarto render

- name: Generate Jupyter notebooks
run: sh assets/scripts/generate_notebooks.sh

- name: Add notebook download links to HTML
run: sh assets/scripts/add_notebook_links.sh

- name: Rename original search index
run: mv _site/search.json _site/search_original.json

Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,5 @@ venv
site_libs
.DS_Store
index_files
digest.txt
digest.txt
**/*.quarto_ipynb
27 changes: 27 additions & 0 deletions assets/scripts/add_notebook_links.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/bash
# Add Jupyter notebook download links to rendered HTML files
# This adds a download link to the toc-actions section (next to "Edit this page" and "Report an issue")

set -e

echo "Adding notebook download links to HTML pages..."

# Find all HTML files that have corresponding .ipynb files
find _site/tutorials _site/usage _site/developers -name "index.html" 2>/dev/null | while read html_file; do
dir=$(dirname "$html_file")
ipynb_file="${dir}/index.ipynb"

# Check if the corresponding .ipynb file exists
if [ -f "$ipynb_file" ]; then
# Check if link is already present
if ! grep -q 'Download notebook' "$html_file"; then
# Insert the notebook link AFTER the "Report an issue" link
# This ensures it goes in the right place in the sidebar toc-actions
# The download="index.ipynb" attribute forces browser to download instead of navigate
perl -i -pe 's/(<a href="[^"]*issues\/new"[^>]*><i class="bi[^"]*"><\/i>Report an issue<\/a><\/li>)/$1<li><a href="index.ipynb" class="toc-action" download="index.ipynb"><i class="bi bi-journal-code"><\/i>Download notebook<\/a><\/li>/g' "$html_file"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make the string 'Download notebook' a variable?

echo " ✓ Added notebook link to $html_file"
fi
fi
done

echo "Notebook links added successfully!"
32 changes: 32 additions & 0 deletions assets/scripts/generate_notebooks.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main other question is: if the qmd-to-ipynb script is already in Python, why not write the Bash scripts in Python as well?

I don't mind the use of Python (it's way better for quick scripts than Julia is), but I think reducing the number of moving parts that need to work together will make for easier maintenance in the future. Especially because all of these scripts serve a common purpose and you're unlikely to want to run one without running the others.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh, personally I find any bash stuff gets harder to read the more complex it gets, I can do that though, not a problem. Let me do the current changes and verify it works then I can explore this fully.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yes bash is definitely hard to read but I meant moving the bash to python - not the other way round!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aha, gotchu. tbh, I originally wrote the bash as a quick fix with the CI, it was only because the default convert didn't work that I did the python too.

Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash
# Generate Jupyter notebooks from .qmd files without re-executing code
# This script converts .qmd files to .ipynb format with proper cell structure

set -e

echo "Generating Jupyter notebooks from .qmd files..."

# Find all .qmd files in tutorials, usage, and developers directories
find tutorials usage developers -name "index.qmd" | while read qmd_file; do
Copy link
Preview

Copilot AI Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using 'while read' with 'find' can fail with filenames containing spaces or special characters. Use 'find ... -print0 | while IFS= read -r -d "" qmd_file; do' for safer file handling.

Suggested change
find tutorials usage developers -name "index.qmd" | while read qmd_file; do
find tutorials usage developers -name "index.qmd" -print0 | while IFS= read -r -d '' qmd_file; do

Copilot uses AI. Check for mistakes.

dir=$(dirname "$qmd_file")
ipynb_file="${dir}/index.ipynb"

echo "Converting $qmd_file to $ipynb_file"

# Convert qmd to ipynb using our custom Python script
# Use relative path from repo root (assets/scripts/qmd_to_ipynb.py)
python3 assets/scripts/qmd_to_ipynb.py "$qmd_file" "$ipynb_file"

# Check if conversion was successful
if [ -f "$ipynb_file" ]; then
# Move the notebook to the _site directory
mkdir -p "_site/${dir}"
cp "$ipynb_file" "_site/${ipynb_file}"
echo " ✓ Generated _site/${ipynb_file}"
else
echo " ✗ Failed to generate $ipynb_file"
fi
done

echo "Notebook generation complete!"
echo "Generated notebooks are in _site/ directory alongside HTML files"
181 changes: 181 additions & 0 deletions assets/scripts/qmd_to_ipynb.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
#!/usr/bin/env python3
"""
Convert Quarto .qmd files to Jupyter .ipynb notebooks with proper cell structure.
Each code block becomes a code cell, and markdown content becomes markdown cells.
"""

import sys
import json
import re
from pathlib import Path
from typing import List, Dict, Any, Optional


class QmdToIpynb:
def __init__(self, qmd_path: str):
self.qmd_path = Path(qmd_path)
self.cells: List[Dict[str, Any]] = []
self.kernel_name = "julia-1.11" # Default kernel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated notebooks don't work in Google Colab, or am I doing something wrong? I think the defined kernel might be the issue since Colab only supports the lts version of Julia by default. Also, we'll need to change this hardcoded version every time we update the docs to a new version. Maybe we should add a note in the README or change it to use the version from an env file or some variable that we can use everywhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think latest push should fix this. Lemme verify and confirm once it builds


def parse(self) -> None:
"""Parse the .qmd file and extract cells."""
with open(self.qmd_path, 'r', encoding='utf-8') as f:
content = f.read()

lines = content.split('\n')
i = 0

# Skip YAML frontmatter
if lines[0].strip() == '---':
i = 1
while i < len(lines) and lines[i].strip() != '---':
# Check for engine specification
if lines[i].strip().startswith('engine:'):
engine = lines[i].split(':', 1)[1].strip()
if engine == 'julia':
self.kernel_name = "julia-1.11"
elif engine == 'python':
self.kernel_name = "python3"
i += 1
i += 1 # Skip the closing ---

# Parse the rest of the document
current_markdown = []

while i < len(lines):
line = lines[i]

# Check for code block start
code_block_match = re.match(r'^```\{(\w+)\}', line)
if code_block_match:
# Save any accumulated markdown
if current_markdown:
self._add_markdown_cell(current_markdown)
current_markdown = []

# Extract code block
lang = code_block_match.group(1)
i += 1
code_lines = []
cell_options = []

# Collect code and options
while i < len(lines) and not lines[i].startswith('```'):
if lines[i].startswith('#|'):
cell_options.append(lines[i])
else:
code_lines.append(lines[i])
i += 1

# Add code cell (with options as comments at the top)
full_code = cell_options + code_lines
self._add_code_cell(full_code, lang)

i += 1 # Skip closing ```
else:
# Accumulate markdown
current_markdown.append(line)
i += 1

# Add any remaining markdown
if current_markdown:
self._add_markdown_cell(current_markdown)

def _add_markdown_cell(self, lines: List[str]) -> None:
"""Add a markdown cell, stripping leading/trailing empty lines."""
# Strip leading empty lines
while lines and not lines[0].strip():
lines.pop(0)

# Strip trailing empty lines
while lines and not lines[-1].strip():
lines.pop()

if not lines:
return

content = '\n'.join(lines)
cell = {
"cell_type": "markdown",
"metadata": {},
"source": content
}
self.cells.append(cell)

def _add_code_cell(self, lines: List[str], lang: str) -> None:
"""Add a code cell."""
content = '\n'.join(lines)

# For non-Julia code blocks (like dot/graphviz), add as markdown with code formatting
# since Jupyter notebooks typically use Julia kernel for these docs
if lang != 'julia' and lang != 'python':
# Convert to markdown with code fence
markdown_content = f"```{lang}\n{content}\n```"
cell = {
"cell_type": "markdown",
"metadata": {},
"source": markdown_content
}
else:
cell = {
"cell_type": "code",
"execution_count": None,
"metadata": {},
"outputs": [],
"source": content
}

self.cells.append(cell)

def to_notebook(self) -> Dict[str, Any]:
"""Convert parsed cells to Jupyter notebook format."""
notebook = {
"cells": self.cells,
"metadata": {
"kernelspec": {
"display_name": "Julia 1.11",
"language": "julia",
"name": self.kernel_name
Copy link
Preview

Copilot AI Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded display name 'Julia 1.11' doesn't match the dynamic kernel name when using Python. Consider making the display name dynamic based on the kernel.

Copilot uses AI. Check for mistakes.

},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.11.0"
}
Comment on lines 161 to 165
Copy link
Preview

Copilot AI Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The language_info is hardcoded for Julia even when the kernel is Python. This should be conditional based on the detected engine/kernel type.

Copilot uses AI. Check for mistakes.

},
"nbformat": 4,
"nbformat_minor": 5
}
return notebook

def write(self, output_path: str) -> None:
"""Write the notebook to a file."""
notebook = self.to_notebook()
with open(output_path, 'w', encoding='utf-8') as f:
json.dump(notebook, f, indent=2, ensure_ascii=False)


def main():
if len(sys.argv) < 2:
print("Usage: qmd_to_ipynb.py <input.qmd> [output.ipynb]")
sys.exit(1)

qmd_path = sys.argv[1]

# Determine output path
if len(sys.argv) >= 3:
ipynb_path = sys.argv[2]
else:
ipynb_path = Path(qmd_path).with_suffix('.ipynb')

# Convert
converter = QmdToIpynb(qmd_path)
converter.parse()
converter.write(ipynb_path)

print(f"Converted {qmd_path} -> {ipynb_path}")


if __name__ == "__main__":
main()
2 changes: 1 addition & 1 deletion tutorials/coin-flipping/index.qmd
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Introduction: Coin Flipping"
engine: julia
aliases:
aliases:
- ../00-introduction/index.html
- ../00-introduction/
---
Expand Down