add `extern "custom"` functions #140770

folkertdev · 2025-05-07T20:26:31Z

tracking issue: #140829
previous discussion: #140566

In short, an extern "custom" function is a function with a custom ABI, that rust does not know about. Therefore, such functions can only be defined with #[unsafe(naked)] and naked_asm!, or via an extern "C" { /* ... */ } block. These functions cannot be called using normal rust syntax: calling them can only be done from inline assembly.

The motivation is low-level scenarios where a custom calling convention is used. Currently, we often pick extern "C", but that is a lie because the function does not actually respect the C calling convention.

At the moment "custom" seems to be the name with the most support. That name is not final, but we need to pick something to actually implement this.

r? @traviscross
cc @tgross35

rustbot · 2025-05-07T20:26:39Z

These commits modify compiler targets.
(See the Target Tier Policy.)

rustbot · 2025-05-07T20:53:08Z

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

compiler/rustc_smir/src/stable_mir/abi.rs

compiler/rustc_passes/src/custom_abi.rs

compiler/rustc_passes/messages.ftl

compiler/rustc_smir/src/stable_mir/abi.rs

traviscross · 2025-05-08T02:49:24Z

tests/ui/abi/custom.rs

+#[unsafe(naked)]
+unsafe extern "custom" fn double(a: u64) -> u64 {
+    naked_asm!("add rax, rax", "ret");
+}


For now, I think I'd suggest just not allowing parameters and a return type -- or putting this under a separate feature gate.

We could always allow this later. It seems like one of the few question marks here, so it'd be better to do the minimal thing to start.

tests/ui/abi/bad-custom.rs

bjorn3 · 2025-05-08T18:39:10Z

Emit some extern "custom" errors in ast_validation:wq

Did you forget to press escape before quitting vim?

folkertdev · 2025-05-08T18:45:19Z

ah, yeah. Well it'll probably get squashed before this is ready

folkertdev · 2025-05-08T20:59:28Z

I've made a proper tracking issue at #140829

bors · 2025-05-17T07:02:22Z

☔ The latest upstream changes (presumably #141002) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2025-06-04T03:24:22Z

☔ The latest upstream changes (presumably #141984) made this pull request unmergeable. Please resolve the merge conflicts.

folkertdev · 2025-06-05T21:36:54Z

I've rebased and changed the implementation to match #141774, which is both shorter and apparently faster too.

bors · 2025-06-06T16:05:55Z

☔ The latest upstream changes (presumably #142099) made this pull request unmergeable. Please resolve the merge conflicts.

folkertdev · 2025-06-06T16:56:51Z

@traviscross are we waiting for T-lang for something here? Otherwise we could get a T-compiler review to move this forward right?

bors · 2025-06-06T22:07:31Z

☔ The latest upstream changes (presumably #141774) made this pull request unmergeable. Please resolve the merge conflicts.

workingjubilee · 2025-06-07T03:27:01Z

I believe T-lang has already basically okayed this experiment per TC's comment here and I think any concerns that TC winds up having, in addition to the review TC already gave, can be addressed in followup. It's not like lang's gonna FCP this tomorrow, is it? Rolling a compiler review.

r? compiler

workingjubilee

I'm sorry but I have a huge and tedious nit that should be a tidy rule but isn't because Reasons.

compiler/rustc_abi/src/canon_abi.rs

compiler/rustc_ast_passes/src/errors.rs

compiler/rustc_codegen_cranelift/src/abi/mod.rs

compiler/rustc_codegen_gcc/src/abi.rs

compiler/rustc_codegen_llvm/src/abi.rs

compiler/rustc_target/src/spec/abi_map.rs

traviscross · 2025-06-07T05:15:47Z

Yes, this is OK as a lang matter. I'd probably suggest...

r? tgross35

tgross35 · 2025-06-08T02:40:54Z

Happy to review on Monday, but @workingjubilee if you've already taken a look at the rest then feel free to pick it up as well

bors · 2025-06-08T02:49:25Z

☔ The latest upstream changes (presumably #142181) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2025-06-09T09:11:48Z

☔ The latest upstream changes (presumably #141435) made this pull request unmergeable. Please resolve the merge conflicts.

rustbot assigned traviscross May 7, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 7, 2025

This comment has been minimized.

Sign in to view

folkertdev force-pushed the custom-abi branch from f61cada to 847b357 Compare May 7, 2025 20:53

bjorn3 reviewed May 7, 2025

View reviewed changes

compiler/rustc_smir/src/stable_mir/abi.rs Show resolved Hide resolved

This comment has been minimized.

Sign in to view

folkertdev force-pushed the custom-abi branch from 847b357 to 93971b9 Compare May 7, 2025 22:17

tgross35 reviewed May 7, 2025

View reviewed changes

compiler/rustc_passes/src/custom_abi.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

mejrs reviewed May 7, 2025

View reviewed changes

compiler/rustc_passes/src/custom_abi.rs Outdated Show resolved Hide resolved

compiler/rustc_passes/messages.ftl Outdated Show resolved Hide resolved

compiler/rustc_smir/src/stable_mir/abi.rs Show resolved Hide resolved

traviscross reviewed May 8, 2025

View reviewed changes

tgross35 reviewed May 8, 2025

View reviewed changes

tests/ui/abi/bad-custom.rs Outdated Show resolved Hide resolved

folkertdev force-pushed the custom-abi branch from 93971b9 to a72606d Compare May 8, 2025 15:44

This comment has been minimized.

Sign in to view

folkertdev force-pushed the custom-abi branch from a72606d to 64868d3 Compare May 8, 2025 17:02

This comment has been minimized.

Sign in to view

folkertdev force-pushed the custom-abi branch from 64868d3 to 7807720 Compare May 8, 2025 17:59

This comment has been minimized.

Sign in to view

folkertdev force-pushed the custom-abi branch from 7807720 to 9d529f8 Compare May 8, 2025 18:37

folkertdev mentioned this pull request May 8, 2025

Tracking Issue for abi_custom #140829

Open

3 tasks

folkertdev force-pushed the custom-abi branch from 9d529f8 to 6ebe16c Compare May 8, 2025 20:58

folkertdev force-pushed the custom-abi branch from 6ebe16c to f9e1963 Compare May 17, 2025 19:22

folkertdev force-pushed the custom-abi branch from f9e1963 to 4c967f4 Compare June 5, 2025 20:19

rustbot added the A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. label Jun 5, 2025

folkertdev force-pushed the custom-abi branch from 4c967f4 to a705a09 Compare June 5, 2025 21:31

This comment has been minimized.

Sign in to view

folkertdev force-pushed the custom-abi branch from a705a09 to 5c5e04e Compare June 5, 2025 21:44

folkertdev force-pushed the custom-abi branch from 5c5e04e to 6073b4c Compare June 6, 2025 16:39

folkertdev force-pushed the custom-abi branch from 6073b4c to e433f07 Compare June 6, 2025 23:09

rustbot assigned fee1-dead and unassigned traviscross Jun 7, 2025

workingjubilee added A-naked Area: `#[naked]`, prologue and epilogue-free, functions, https://git.io/vAzzS A-ABI Area: Concerning the application binary interface (ABI) labels Jun 7, 2025

workingjubilee requested changes Jun 7, 2025

View reviewed changes

This comment was marked as resolved.

Sign in to view

rustbot assigned tgross35 and unassigned fee1-dead Jun 7, 2025

folkertdev force-pushed the custom-abi branch from e433f07 to 675d6e6 Compare June 7, 2025 10:40

folkertdev force-pushed the custom-abi branch from 675d6e6 to a3fe4a3 Compare June 8, 2025 09:26

add extern "custom" functions

823e466

folkertdev force-pushed the custom-abi branch from a3fe4a3 to 823e466 Compare June 9, 2025 10:23

add extern "custom" functions #140770

Are you sure you want to change the base?

add extern "custom" functions #140770

Conversation

folkertdev commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented May 7, 2025

Uh oh!

This comment has been minimized.

rustbot commented May 7, 2025

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

traviscross May 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

bjorn3 commented May 8, 2025

Uh oh!

folkertdev commented May 8, 2025

Uh oh!

folkertdev commented May 8, 2025

Uh oh!

bors commented May 17, 2025

Uh oh!

bors commented Jun 4, 2025

Uh oh!

folkertdev commented Jun 5, 2025

Uh oh!

This comment has been minimized.

bors commented Jun 6, 2025

Uh oh!

folkertdev commented Jun 6, 2025

Uh oh!

bors commented Jun 6, 2025

Uh oh!

workingjubilee commented Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

workingjubilee left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

traviscross commented Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

tgross35 commented Jun 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors commented Jun 8, 2025

Uh oh!

bors commented Jun 9, 2025

Uh oh!

Uh oh!

add `extern "custom"` functions #140770

add `extern "custom"` functions #140770

folkertdev commented May 7, 2025 •

edited

Loading

workingjubilee commented Jun 7, 2025 •

edited

Loading

traviscross commented Jun 7, 2025 •

edited

Loading

tgross35 commented Jun 8, 2025 •

edited

Loading