bootstrap seems to default to downloading the LLVM submodule

[RalfJung]

Summary

Run the following (in a GH CI job):

git clone https://github.com/rust-lang/rust
cd rust
./x check miri

Expected behaviour

The bootstrap.example.toml indicates that a prebuilt LLVM is downloaded by default. That's also what happens when I run these commands locally.

Actual behaviour

In CI, I see:

Updating submodule src/llvm-project
Submodule 'src/llvm-project' (https://github.com/rust-lang/llvm-project.git) registered for path 'src/llvm-project'
Cloning into '/home/runner/work/miri/rust/src/llvm-project'...
remote: Enumerating objects: 3, done.        
remote: Counting objects:  33% (1/3)        
remote: Counting objects:  66% (2/3)        
remote: Counting objects: 100% (3/3)        
remote: Counting objects: 100% (3/3), done.        
remote: Compressing objects:  50% (1/2)        
remote: Compressing objects: 100% (2/2)        
remote: Compressing objects: 100% (2/2), done.        
remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 (from 0)        
Receiving objects:  33% (1/3)
Receiving objects:  66% (2/3)
Receiving objects: 100% (3/3)
Receiving objects: 100% (3/3), done.
From https://github.com/rust-lang/llvm-project
 * branch                333793696b0a08002acac6af983adc7a5233fc56 -> FETCH_HEAD
Submodule path 'src/llvm-project': checked out '333793696b0a08002acac6af983adc7a5233fc56'

This takes close to 2 minutes and wastes a bunch of bandwidth.

Bootstrap configuration (bootstrap.toml)

(none)

Operating system

ubuntu-latest

HEAD

2f4dfc7

[Kobzol]

The default behavior on CI is "if-unchanged", locally it's true. I guess that we could change that, because on our CI we override the default value explicitly anyway.

[RalfJung]

But why does if-unchanged download the submodule? If I didn't check out the submodule, clearly I didn't change anything in it.

The build later does download the pre-built LLVM as expected, FWIW. It just first wastes some time and bandwidth also downloading the LLVM git repo.

[Kobzol]

Bootstrap just checks it out proactively and then checks if it has any changes, for simplicity of implementation, I suppose. I'm not sure if it's possible to change the commit to which the submodule points to without actually having it checked out though? So maybe we wouldn't have to check it out.

[RalfJung]

Even if you change the commit it points to, that won't actually have any effect until you check it out.

[RalfJung]

Hm, I just still got an unexpected LLVM submodule download on my laptop (i.e., not on CI) with these settings

change-id = 125535

[build]
# Reduce RAM consumption
jobs = 12

[rust]
# The safe & fast option.
download-rustc = "if-unchanged"
# Get `debug!`/`trace!` output and more sanity checks.
# FIXME: should be 'true' but then it doesn't download rustc...
debug-assertions = false
# Get actually-useful information from backtraces, profiling, etc. with minimal added bytes
debuginfo-level = "line-tables-only"
# Faster builds after small changes.
incremental = true
# Make the compiler and standard library faster to build, at the expense of a ~20% runtime slowdown.
lto = "off"

# Don't fail the build due to warnings.
deny-warnings = false
# Prints each test name as it is executed.
#verbose-tests = true

[llvm]
# The safe & fast option.
download-ci-llvm = "if-unchanged"

This is in a feature branch based on 15283f6.