Flags for retpoline mitigation

[Darksonn]

Proposal

Add two new flags to the compiler called -Zretpoline and -Zretpoline-external-thunk to configure the compiler to generate return trampolines. The retpoline mitigation is used to mitigate a sidechannel vulnerability known as "Spectre".

The flags will be implemented by enabling the following LLVM target features:

• -Zretpoline-external-thunk enables +retpoline-external-thunk, +retpoline-indirect-branches, +retpoline-indirect-calls.
• -Zretpoline enables +retpoline-indirect-branches, +retpoline-indirect-calls.

The naming of these flags is taken from clang, where they are called -mretpoline and -mretpoline-external-thunk respectively. For uncommon flags such as these, I believe matching the clang names is the best approach. Note that on clang, the latter flag implies the former.

I suggest that the flags should utilize the target modifier infrastructure to prevent mixing compilation units with and without the flags because such misuse breaks the mitigation. However, the flag to opt-out from this check does not necessarily need the word "unsafe" because it's not actually part of the ABI

These flags are added with the intent of later stabilizing them, hence this MCP.

The Rust issue for this feature is rust-lang/rust#116852.

Comparison to GCC:

• The clang flag -mretpoline is equivalent to -mindirect-branch=thunk-inline -mindirect-branch-register on gcc.
• The clang flag -mretpoline-external-thunk is equivalent to -mindirect-branch=thunk-extern -mindirect-branch-register on gcc.

Process

The main points of the Major Change Process are as follows:

• File an issue describing the proposal.
• A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
  • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
  • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
• Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

[rustbot]

Important

This issue is not meant to be used for technical discussion. There is a Zulip stream for that.
Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

Concerns or objections can formally be registered here by adding a comment.

@rfcbot concern reason-for-concern
<description of the concern>

Concerns can be lifted with:

@rfcbot resolve reason-for-concern

See documentation at https://forge.rust-lang.org

cc @rust-lang/compiler