Save x.py's help text for saving output time

[Shunpoco]

Fix #141903

Currently x.py help (--help) builds bootstrap binary everytime, so it takes some seconds to print help.
This PR does:

• Saves current help text into a file (x.py run generate-help)
• Changes bootstrap.py to print the help in the saved file and to exit without touching bootstrap binary
• Modifies x.py test bootstrap to check if the help file is up-to-date

[rustbot]

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[jieyouxu]

I'm on the fence on this. I'm slightly leaning towards "I don't feel like the extra complexity and having to remember another point of potential outdatedness pulls its weight". But if another bootstrap reviewer feels okay about this, that is fine with me too.

[Kobzol]

I also wasn't a super big fan of this, tbh. But while testing this PR, I realized that this approach actually cannot work in the way it was designated in the original issue, or at least not generally. Because it assumes that bootstrap options and flags are unified, but that is no longer the case. Since we switched to clap, some subcommands actually have their own arguments, and thus also different help.

For example, here is the output of x perf --help:

Perform profiling and benchmarking of the compiler using `rustc-perf`

Usage: x.py perf [OPTIONS] [PATHS]... [-- <ARGS>...] <COMMAND>

Commands:
  eprintln    Run `profile_local eprintln`. This executes the compiler on the given benchmarks and stores its stderr output
  samply      Run `profile_local samply` This executes the compiler on the given benchmarks and profiles it with `samply`. You need to install `samply`, e.g. using `cargo install samply`
  cachegrind  Run `profile_local cachegrind`. This executes the compiler on the given benchmarks under `Cachegrind`
  benchmark   Run compile benchmarks with a locally built compiler
  compare     Compare the results of two previously executed benchmark runs

Arguments:
  [PATHS]...
          paths for the subcommand

  [ARGS]...
          arguments passed to subcommands

Options:
  -v, --verbose...
          use verbose output (-vv for very verbose)

  -i, --incremental
          use incremental compilation

      --config <FILE>
          TOML configuration file for build

      --build-dir <DIR>
          Build directory, overrides `build.build-dir` in `bootstrap.toml`

... many shared bootstrap flags

But with this PR, when I do x perf --help, it will only print the saved help of x --help, which is wrong.

So for this to work generically, the Python wrapper would have to:

1. Recognize which bootstrap command is being executed (at least for the commands that have some custom arguments/options/subcommands)
2. Save the help output for each such command specifically

Even 1) is annoying, as the Python wrapper would have to reimplement CLI parsing logic, to recognize e.g. things like ./x --stage 1 build. But in any case, I don't think we should have that kind of complexity in the Python layer.

Sorry for not catching that earlier, but I think that this is not going to work. CC @jyn514 if you have other opinions.

[Shunpoco]

Thanks, I didn't catch x <subcommand> --help also goes to the saved file.

And

Recognize which bootstrap command is being executed (at least for the commands that have some custom arguments/options/subcommands)

I agree, this is inevitable if all subcommands' helps are also loaded from files. I don't think this is a good way.

I think we can make only root help (x help|--help|-h) use a saved file by changing help_triggered in bootstrap.py. In that case, only the root speeds up to print the help but other subcommands still use helps from the binary.
Of course I need to confirm if that is enough valuable to bring the complexity, though.

[Mark-Simulacrum]

Per above I think we can probably close this?

One thought is we could plausibly generate e.g. the man pages that clap supports and check those in, but it all seems fairly complicated for not as much value.

[jyn514]

I think if we only do this for the very top-level x.py --help that's fine. We can still delegate to the rust code for subcommand help (anything other than the exact string -h or --help), that doesn't require the python to be smart.

[Shunpoco]

@jyn514 Thanks for the confirmation,
fb5b849 makes only root help use a saved file (help for subcommands go to the binary)

[Mark-Simulacrum]

Probably not relevant to this PR, but I wonder if we should really be showing all of this in help... e.g., the PGO options don't feel relevant to 99% of x.py users, and there's enough options here that we end up scrolling off screen for the actually-useful collection of subcommands test/doc/check/etc.

It might mean that we want a verbose and non-verbose help for the top level?

[Shunpoco]

It makes sense to me for both > not relevant to this PR and > want a verbose and non-verbose help for the top level

Although I don't know if Clap supports such detailed/brief help, can I make an issue ticket for it (Check Clap's features, then modify the top-level help if we can)

[Shunpoco]

@rustbot review

Note: I will clean up the commit history after the review

[Mark-Simulacrum]

The second part of this seems like it's not quite right, it needs (sys.argv[1] in ["-h", "--help"] and len(sys.argv) == 2), right? Otherwise x.py --help check wouldn't do the right thing?

[Shunpoco]

Currently x.py --help check prints the top level help, not check's help. So I think we don't need len(sys.argv) == 2 for it.
Or, do you think x.py --help check should be identical with x.py check --help?

[Shunpoco]

Or, do you think~

I guess if we dispatch x.py --help check to bootstrap binary. Clap returns the top level help for it unless we modify the behaviour.