Add support for macro expansion in rustdoc source code pages #137229

GuillaumeGomez · 2025-02-18T17:10:39Z

This is what it looks like:

You can test it here. In this case, I also enabled the --generate-link-to-definition to show that both options work well together.

Note: There is a bug currently in firefox where the line numbers are not displayed correctly if they're inside the "macro expansion" span: https://bugzilla.mozilla.org/show_bug.cgi?id=1949948 ~~Found a workaround around this bug.~~

r? @notriddle

rustbot · 2025-02-18T17:10:50Z

Some changes occurred in HTML/CSS/JS.

cc @GuillaumeGomez, @jsha

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

notriddle · 2025-02-18T17:18:21Z

src/librustdoc/html/highlight.rs

+                "{closing}\
+<span class=expansion>\
+    <input id={id} type=checkbox>\
+    <label for={id}>↕</label>{reopening}",


Why is this done differently than the normal expand/collapse system, but in code and in presentation?

Because it would force to duplicate all the common code. For example in:

func(arg, macro!(blabla), arg3);

Only macro!(blabla) will be different, so it's not interesting to duplicate all the rest. Instead, we put the original code in a span, the expanded one in another and then we can simply switch them based on the status of the checked property of the input. I'm also not a big fan of our current details approach as it requires more JS to be working fully as expected but it's a debate for another time.

src/librustdoc/html/render/span_map.rs

src/librustdoc/html/static/css/rustdoc.css

rustbot · 2025-02-21T09:40:00Z

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

GuillaumeGomez · 2025-02-22T13:02:20Z

I opened the firefox bug report: https://bugzilla.mozilla.org/show_bug.cgi?id=1949948

bors · 2025-02-24T14:44:59Z

☔ The latest upstream changes (presumably #137511) made this pull request unmergeable. Please resolve the merge conflicts.

GuillaumeGomez · 2025-02-26T09:49:41Z

PR is ready for review.

bors · 2025-05-06T17:29:55Z

☔ The latest upstream changes (presumably #140702) made this pull request unmergeable. Please resolve the merge conflicts.

GuillaumeGomez · 2025-05-06T17:34:34Z

Fixed merge conflict.

GuillaumeGomez · 2025-05-12T12:08:24Z

Went around firefox bug, so now it's always displayed as expected.

GuillaumeGomez · 2025-07-03T12:53:58Z

Fixed merge conflict. Diff for CSS is now much smaller. :)

bors · 2025-07-17T09:25:17Z

☔ The latest upstream changes (presumably #144044) made this pull request unmergeable. Please resolve the merge conflicts.

GuillaumeGomez · 2025-07-17T11:21:03Z

Fixed merge conflict.

…ns open inside expansion spans

GuillaumeGomez · 2025-08-01T14:58:20Z

@fmease mentioned that patterns and statements could get macro expansion as well so I added support for both cases.

fmease

Oops forgot to submit after our chat

fmease · 2025-08-01T12:25:02Z

src/librustdoc/html/macro_expansion.rs

+            walk_expr(self, expr);
+        }
+    }
+


Maybe also visit_pat. For things like match () { expand!() => {} }.

Maybe also visit_stmt. E.g., a macro expanding to let _ = …; (this is not an expr).

GuillaumeGomez · 2025-08-01T18:28:08Z

Too late, already did it. :p

lolbinarycat · 2025-08-21T18:33:27Z

src/librustdoc/html/highlight.rs

@@ -271,6 +298,104 @@ fn empty_line_number(out: &mut impl Write, _: u32, extra: &'static str) {
    out.write_str(extra).unwrap();
 }

+fn get_next_expansion<'a>(
+    expanded_codes: Option<&'a Vec<ExpandedCode>>,


I feel like this could easily just be &[ExpandedCode]? and also this lifetime could be omitted?

I don't see any reason we would need to distinguish between an empty vec and None, as this function would return None in either case.

lolbinarycat · 2025-08-21T18:57:46Z

src/librustdoc/html/macro_expansion.rs

+struct ExpandedCodeInfo {
+    /// Callsite of the macro.
+    span: Span,
+    /// Expanded macro source code.


Suggested change

/// Expanded macro source code.

/// Expanded macro source code (html escaped)

lolbinarycat · 2025-08-21T18:58:16Z

src/librustdoc/html/highlight.rs

+    } else if let Some(Class::Expansion) = klass {
+        // Nothing to escape here so we get the text to write it directly.
+        out.write_str(text.0).unwrap();


Suggested change

} else if let Some(Class::Expansion) = klass {

// Nothing to escape here so we get the text to write it directly.

out.write_str(text.0).unwrap();

} else if let Some(Class::Expansion) = klass {

// This has already been escaped so we get the text to write it directly.

out.write_str(text.0).unwrap();

lolbinarycat · 2025-08-21T19:03:53Z

src/librustdoc/html/macro_expansion.rs

+    /// Expanded span
+    expanded_span: Span,


Suggested change

/// Expanded span

expanded_span: Span,

/// Span of macro-generated code.

expanded_span: Span,

lolbinarycat · 2025-08-21T19:05:01Z

src/librustdoc/html/macro_expansion.rs

+/// HIR visitor which retrieves expanded macro.
+///
+/// Once done, the `expanded_codes` will be transformed into a vec of [`ExpandedCode`]
+/// which contains more information needed when running the source code highlighter.


Suggested change

/// which contains more information needed when running the source code highlighter.

/// which contains the information needed when running the source code highlighter.

nit: neither ExpandedCode nor ExpandedCodeInfo contains strictly more info

lolbinarycat · 2025-08-21T19:35:14Z

src/librustdoc/html/macro_expansion.rs

+
+    fn compute_expanded(mut self) -> FxHashMap<BytePos, Vec<ExpandedCode>> {
+        self.expanded_codes.sort_unstable_by(|item1, item2| item1.span.cmp(&item2.span));
+        let mut expanded: FxHashMap<BytePos, Vec<ExpandedCode>> = FxHashMap::default();


Suggested change

let mut expanded: FxHashMap<BytePos, Vec<ExpandedCode>> = FxHashMap::default();

let mut expanded: FxHashMap<BytePos, Vec<ExpandedCode>> = FxHashMap::with_capacity(self.expaned_codes.len());

I think that having multiple macro expansions with the same byte position should be rare enough that this is a good estimation of the hashmap size?

lolbinarycat · 2025-08-21T19:40:07Z

src/librustdoc/html/mod.rs

 // used by the error-index generator, so it needs to be public
+pub(crate) mod macro_expansion;
 pub mod markdown;


Suggested change

// used by the error-index generator, so it needs to be public

pub(crate) mod macro_expansion;

pub mod markdown;

pub(crate) mod macro_expansion;

// used by the error-index generator, so it needs to be public

pub mod markdown;

lolbinarycat · 2025-08-21T19:45:38Z

src/librustdoc/lib.rs

+                    run_renderer(
+                        krate,
+                        render_opts,
+                        cache,
+                        tcx,
+                        |krate, render_opts, cache, tcx| {
+                            json::JsonRenderer::init(krate, render_opts, cache, tcx)
+                        },
+                    )


Suggested change

run_renderer(

krate,

render_opts,

cache,

tcx,

|krate, render_opts, cache, tcx| {

json::JsonRenderer::init(krate, render_opts, cache, tcx)

},

)

run_renderer(

krate,

render_opts,

cache,

tcx,

json::JsonRenderer::init,

)

Does this not work? only reason I can think it wouldn't is if there's some implicit reborrowing or deref coercion that needs to happen.

lolbinarycat · 2025-08-21T19:47:56Z

src/librustdoc/lib.rs

+        opt(
+            Unstable,
+            Flag,
+            "",
+            "generate-macro-expansion",
+            "Add possibility to expand macros in the HTML source code pages",
+            "",
+        ),


All of the flags at the end of the list are deprecated or removed. This flag should be moved before the "deprecated / removed options" comment.

lolbinarycat · 2025-08-21T19:49:29Z

tests/rustdoc-gui/src/macro_expansion/lib.rs

@@ -0,0 +1,51 @@
+// Test crate used to check the `--generate-macro-expansion` option.
+//@ compile-flags: -Zunstable-options --generate-macro-expansion --generate-link-to-definition
+


Please make this test a multi-file crate, so we can ensure those are handled properly.

rustbot assigned notriddle Feb 18, 2025

notriddle reviewed Feb 18, 2025

View reviewed changes