`-Zrustdoc-scrape-examples` not working if there is no workspace

[GuillaumeGomez]

Problem

In killercup/test-rustdoc-example-scraping-across-workspace-on-docrs#1 and in rust-lang/docs.rs#2305, we were able to find out that if you re-export non-workspace members, examples would not be scraped for them

Steps

$ git clone https://github.com/killercup/test-rustdoc-example-scraping-across-workspace-on-docrs.git
$ cargo +nightly doc -Zunstable-options -Zrustdoc-scrape-examples
$ ... open in browser and see no examples
$ git checkout HEAD~
$ cargo +nightly doc -Zunstable-options -Zrustdoc-scrape-examples
$ ... open in browser and see examples

Possible Solution(s)

When scraping examples from a package, cargo only asks for examples that use APIs from specific crates, presumably for performance reasons

        // Only scrape example for items from crates in the workspace, to reduce generated file size
        for pkg in cx.bcx.ws.members() {
            let names = pkg
                .targets()
                .iter()
                .map(|target| target.crate_name())
                .collect::<HashSet<_>>();
            for name in names {
                rustdoc.arg("--scrape-examples-target-crate").arg(name);
            }
        }

This is filtered to workspace members. Really what is needed is filtering for public dependencies which has recently been revived (rust-lang/rfcs#3516).

If the performance hit is not noticeable enough, we likely can remove this completely.

If the performance his is noticeable enough, the path forward:

• Long term: switch this to filtering on public normal dependencies (recursively)
• Short term: decide whether to
  • keep filtering by workspace members
  • take the perf hit and do no filtering
  • filter for all normal dependencies, recursively

Notes

No response

Version

No response

[epage]

@willcrichton Wanted to double check on your motivation for doing the workspace member filtering. Happen to remember if this was benchmarked?

[epage]

If we filter by public dependency, because of the previous public dependency RFC, most of the infrastructure is in place. You'd just need to check if the unstable feature is active to make the behavior conditioned on the short term vs long term solution and then check the public dependencies in the long term solution.

The one piece of plumbing that might be missing is the taking the public dependency information at the package/dependency level and using it at the Unit level. I'm assuming its in place though.

Regardless of the short term solution, I'd recommend we include the long term solution of public dependencies because that is on a path for being stable for 2024 edition.

[willcrichton]

I don't remember if I benchmarked it. To double-check, I did a quick experiment. I cloned Bevy and ran with latest Cargo:

cargo doc -Z unstable-options -Z rustdoc-scrape-examples
cat target/debug/deps/*.examples | wc -c

Then I modified the relevant for-loop to be:

for pkg in cx.bcx.packages.packages() {

And re-ran again. With workspace member filtering, the total size of the files was 643k. Without, the total size was 694k.

So from that one data point, there's no big space issue.

[GuillaumeGomez]

Ok, sending the patch then. Thanks for taking a look @willcrichton !