Tell cargo to track logging environment variables

[jyn514]

Previously, changing the value of RUSTC_LOG would not rebuild, and,
because cargo caches output, show exactly the same logging as before.
This was confusing. Emit RUSTC_LOG with other depinfo dependencies
so cargo knows to rebuild when it's changed.

Note that this only adds RUSTC_LOG to depinfo. Unfortunately, adding
RUSTDOC_LOG only when running as rustdoc doesn't help because rustdoc
doesn't emit depinfo at all. Unconditionally emitting RUSTDOC_LOG in the
depinfo doesn't help because cargo ignores it for everything except
rustc itself. I am not yet sure how to fix this; see
rust-lang/cargo#8374 for more info.

[rust-highfive]

r? @petrochenkov

(rust-highfive has picked a reviewer for you, use r? to override)

[cuviper]

Alternatively, could you just add a build script emitting rerun-if-env-changed?

Wait, no, you don't want to rebuild the compiler, just rerun it for any given project. Nevermind me.

[jyn514]

Wait, no, you don't want to rebuild the compiler, just rerun it for any given project. Nevermind me.

Right - I want this to work for arbitrary projects, not just projects I work on a lot.

[Aaron1011]

I don't think this is a good idea. RUSTC_LOG is most useful when debugging the compiler, where it's important to have tight control over when the compiler is invoked.

For example, when debugging an ICE in a project with a large number of dependencies, it can be very useful to repeatedly rebuild the root crate while varying RUSTC_LOG. With this change, the entire dependency graph will be rebuilt, greatly slowing down debugging. If the crash is due to a bug in incremental compilation, then enabling logging will cause the incr cache to be rebuilt, hiding the bug.

[jyn514]

With this change, the entire dependency graph will be rebuilt

Oh good catch, I hadn't thought of that. Is there a way to tell cargo to rebuild only the outermost crate (or maybe just path dependencies)? If not I guess cargo would have to special case logging.

[cuviper]

cargo rustc might be what you want. There's a cargo rustdoc too.

[jyn514]

cargo rustc might be what you want. There's a cargo rustdoc too.

Right, I know about both, but that doesn't help with being surprised that RUSTC_LOG=info has no output, or has cached and incorrect output. See https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/RUSTC_LOG.20output.20get.20somehow.20.22recorded.22 - this is especially bad when going through bootstrap, because there's no clear way to invalidate the cache.

[petrochenkov]

because there's no clear way to invalidate the cache

touch compiler/rustc_crate_of_interest/src/lib.rs

?

[petrochenkov]

Regarding putting env vars to depinfo in general, rustc uses many env vars, I guess for some of them you want to rerun the compiler on change, but for some you do not.
I certainly wouldn't want variables like SOMETHING_COLOR cause rebuilds.

I'm not sure who should make this choice, but I suspect it would be better to leave this job to a higher level tool (like cargo).
rustc could simply write all such variables to depinfo, using a separate format, e.g. # rustc-env-dep instead of # env-dep, and leave it to others to decide what to do with them.

We already have functions for tracked access to env variables in proc macros (tracked_env::var) and rustc build scripts (tracked_env_var_os), rustc itself could get such function as well.
We could even catch uses of untracked env::var(_os)s with an internal lint.

[Aaron1011]

I was debugging some incremental compilation issues the other day, and this change would have made doing do unecessarily difficult. Once I got to an incremental-related ICE, I was able to re-run the compiler with different RUSTC_LOG values, which would repeatedly trigger the ICE with different log output. With this change, I would need to set up the corrupted state each time I changed RUSTC_LOG