Remove default config from bootstrap

[Shourya742]

This PR removes the default config initialization from parse_inner, as it introduced many assumptions during config setup. Instead, each variable is now manually initialized to eliminate certain invariants in parse_inner and streamline the process.

The PR is still a WIP and has been opened for an initial review.

r? @Kobzol

[Kobzol]

Heh, I had feedback for you, but I see that Clippy did it for me in one of the commits xD Let me know once it's ready and I'll check the changes locally, but in general looks great.

[bors]

☔ The latest upstream changes (presumably #145407) made this pull request unmergeable. Please resolve the merge conflicts.

[Kobzol]

Let me know if it's ready for another round of review :)

[Kobzol]

Great work! Left a bunch of comments.

In general, I'm not happy about DownloadContext. Even with the macro I proposed, it's still too easy to use values before they are initialized, for example rust_info is used in download context before it is initialized :( And that's just one thing I noticed, possibly there may be more.

Before we can remove more I/O from parse_inner, I would suggest creating more granular versions of DownloadContext. For example, download_beta_toolchain requires exec_ctx, stage0_metadata, host_target, out, patch_binaries_for_nix, bootstrap_cache_path, llvm_assertions and is_running_on_ci, so approx. 50% of DownloadContext. If we created a separate ctx for this function, we could reduce the likelihood of similar errors.

[Kobzol]

Could you move

targets: flags_target
                .map(|TargetSelectionList(targets)| targets)
                .or_else(|| {
                    build_target.map(|t| t.iter().map(|t| TargetSelection::from_user(t)).collect())
                })
                .unwrap_or_else(|| hosts.clone())

below initialization of hosts? These three things should be configured next to one another.

In general, I would put complex initialization out of the Config { } literal and into a local variable. So e.g.

let skip = flags_skip
                .into_iter()
                .chain(flags_exclude)
                .chain(build_exclude.unwrap_or_default())
                .map(|p| {
                    // Never return top-level path here as it would break `--skip`
                    // logic on rustc's internal test framework which is utilized by compiletest.
                    #[cfg(windows)]
                    {
                        PathBuf::from(p.to_string_lossy().replace('/', "\\"))
                    }
                    #[cfg(not(windows))]
                    {
                        p
                    }
                })
                .collect();

...

Config { ..., skip, ... }

E.g. gdb: build_gdb.map(PathBuf::from) is fine to have in the Config constructor, but more complex and multiline initialization should be moved into a separate variable, to make it easier to read.

The let skip variable should still be declared as late as possible, so just before the Config constructor.

[Kobzol]

Could you please sort the fields in this constructor, and apply // tidy-alphabetical-start + // tidy-alphabetical-end to make sure that they stay sorted? Currently some relates values are far from one another.

[Kobzol]

This is brittle and not much better than the original situation where we incrementally modified Config. What if we created a local macro?

// Macro for creating a new download context. For every usage that needs it, we want to
        // create it anew, to ensure that it uses the latest values of the various variables used
        // in this function.
        macro_rules! make_dwn_ctx {
            () => {
                DownloadContext {
                    path_modification_cache: path_modification_cache.clone(),
                    src: &src,
                    rust_info: rust_info.clone(),
                    submodules: &submodules,
                    download_rustc_commit: download_rustc_commit.clone(),
                    host_target,
                    llvm_from_ci,
                    target_config: target_config.clone(),
                    out: out.clone(),
                    patch_binaries_for_nix,
                    exec_ctx: &exec_ctx,
                    stage0_metadata: &stage0_metadata,
                    llvm_assertions,
                    bootstrap_cache_path: &bootstrap_cache_path,
                    is_running_on_ci,
                }
            };
        }

Then you can just put make_dwn_ctx!() everywhere we need a download context, and it will be created fresh with the latest values of all required local variables, without requiring us to ensure that the download context is kept updated.