Precompiled shaders macro #8217

SupaMaggie70Incorporated · 2025-09-12T05:57:22Z

Connections
Works towards #3103

Description
This adds a new proc macro crate and corresponding proc macro to compile shaders with naga at compile time.

The crate is within the workspace root and not the wgpu crate directory because eventually it will probably be used by wgpu-core to precompile internal compute shaders, and because proc-macro crates must be separate crates.

Example ripped from test

ctx
    .device
    .create_shader_module_passthrough(wgpu::include_precompiled_wgsl!(
        "tests/wgpu-gpu/precompile/shader.wgsl",
        "vs_main",
        all
    ))

Limitations

The vertex/mesh outputs and fragment inputs in a pipeline must always match in a pipeline. DirectX and Metal backends currently strip unused vertex outputs, but this won't work when precompiling
Only one entry point may be written
Assumes all features/capabilities are available on the device, won't error if they aren't
Assumes the device is using minimum language version (e.g. spirv 1.0)
Assumes that stuff like workgroup memory zero initialization must be polyfilled
Cannot be configured at all. E.g. you cannot disable performance-harming things like safety features
Not easily compatible with wgpu-core, so things like timestamp normalization still require runtime shader compiling
Probably tons more that is broken

Testing
I have ensured that the hello_triangle example works using precompiled shaders, on the backends Metal, Vulkan, DX12 with HLSL or DXIL, and GLES. WGSL should in theory also be perfectly fine. Also added a test to ensure that precompiled shaders can be compiled and can be constructed into shader modules.

Squash or Rebase?

Squash

Checklist

Run cargo fmt.
Run taplo format.
Run cargo clippy --tests. If applicable, add:
- --target wasm32-unknown-unknown
Run cargo xtask test to run tests.
If this contains user-facing changes, add a CHANGELOG.md entry.

… (unsure why), testing if its only on macos/msl

…guage

SupaMaggie70Incorporated · 2025-09-15T22:56:43Z

@cwfitzgerald This doesn't require any immediate action from your part, but I think it probably should be brought up in the next maintainers meeting, in terms of where stuff should live (new crate) and how the dependency stuff should work.

cwfitzgerald · 2025-09-24T04:39:52Z

(I'm doing this to see how it does, I'll do a full human review tomorrow, no action is needed based on its comments).

Copilot

Pull Request Overview

This PR adds precompiled shader support to wgpu by introducing a new proc macro crate and corresponding macros to compile shaders with naga at compile time. The key changes enable developers to precompile WGSL, GLSL, and SPIR-V shaders into various target formats (MSL, HLSL, SPIRV, GLSL, DXIL, WGSL) at build time rather than runtime.

New wgpu-precompile-macro crate with proc macros for compile-time shader compilation
Updated ShaderModuleDescriptorPassthrough API to use structured descriptors instead of raw strings/bytes
Added feature-gated precompilation support with conditional compilation for different backends

Reviewed Changes

Copilot reviewed 24 out of 27 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
wgpu/src/macros.rs	Adds new precompilation macros for WGSL, GLSL, and SPIR-V
wgpu/src/lib.rs	Exports new passthrough descriptor types
wgpu-types/src/lib.rs	Defines new structured descriptor types for shader passthroughs
wgpu-precompile-macro/src/lib.rs	Core proc macro implementation for compile-time shader compilation
wgpu/Cargo.toml	Adds precompile feature and dependencies
tests/ and examples/	Updates to use new precompilation API and test the functionality

Comments suppressed due to low confidence (1)

wgpu-precompile-macro/src/lib.rs:1

There's a logic error here. args.shader_stage is an Option<naga::ShaderStage>, but you're calling unwrap() directly in an if condition, which will panic if None. This should be args.shader_stage.is_some() or pattern matching.

use hashbrown::HashSet;

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

wgpu/src/macros.rs

wgpu-precompile-macro/src/lib.rs

Co-authored-by: Copilot <[email protected]>

SupaMaggie70Incorporated · 2025-09-24T04:49:42Z

Comments suppressed due to low confidence (1)

wgpu-precompile-macro/src/lib.rs:1
* There's a logic error here. `args.shader_stage` is an `Option<naga::ShaderStage>`, but you're calling `unwrap()` directly in an if condition, which will panic if `None`. This should be `args.shader_stage.is_some()` or pattern matching.
use hashbrown::HashSet;
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Lmao, hashbrown is to blame!

SupaMaggie70Incorporated · 2025-09-24T17:18:30Z

I'd just like to say about this PR: I'm very satisfied with how it works. I haven't really done much macro programming before but writing it was surprisingly simple and pleasant, I'm shocked it works as well as it does already! Even though its surely a little rough around the edges, not easily configurable, etc.

Also, I'd really really like if we could get this in before the release next week, since it'd probably be useful to several wgpu consumers, so if this could be fast tracked I'd appreciate it.

cwfitzgerald

This is really cool stuff! I have some high level concerns about this and some initial nits when I'm looking it over. Very happy to see progress here though! Doesn't seem to be as awful a transform as I feared.

I do have some conserns about how resource mappings are handled, as the naga backend does take information about where to put various resources (flattening bindings on metal/dx12, dealing with bindless samplers on dx12, etc)

I think we should split this into two separate bits of code. One which is a normal function which does the shader compilation and stuff, then a macro which is a frontend to that function. This will help separation of concerns and will also support doing precompilation in a build script or an external tool, which I would expect to be a more common desire (and I personally would want, as opposed to a proc macro). It should also vastly simplify the cfg situation.

Happy to discuss more about this plan. We can have two crates (wgpu-precompile and wgpu-precompile-macro).

examples/features/src/hello_triangle/mod.rs

tests/tests/wgpu-dependency/main.rs

wgpu-hal/src/auxil/dxgi/result.rs

cwfitzgerald · 2025-09-24T22:41:31Z

wgpu-precompile-macro/src/lib.rs

+    std::fs::write(&input_file, args.hlsl_code.as_bytes())
+        .expect("Failed to write to HLSL input file");
+    let temporary_file_location = temporary_folder_location.join("file.dxil");
+    let output = std::process::Command::new("dxc")


Definitely need to make sure the errors when external tools aren't found are very good.

I've updated the error messages slightly. You probably have a way you'd prefer to do it though

cwfitzgerald · 2025-09-24T22:42:57Z

wgpu-precompile-macro/src/lib.rs

+            // Related features: dx12
+            // Note: this differs slightly from platform-deps which enables dx12 even on linux/android, but just doesn't
+            // let you use it


This feels weird? what's up with this?

From the windows-linux-android platform deps crate:

[features] gles = ["wgpu-hal/gles"] vulkan = ["wgpu-hal/vulkan"] dx12 = ["wgpu-hal/dx12"] renderdoc = ["wgpu-hal/renderdoc"] # Depend on wgpu-hal conditionally, so that the above features only apply to wgpu-hal on this set of platforms. [target.'cfg(any(windows, target_os = "linux", target_os = "android", target_os = "freebsd"))'.dependencies] wgpu-hal.workspace = true

so the wgpu-hal feature for dx12 is enabled even on linux. So I'm saying here that in the precompile macro, we diverge from that to avoid shelling into DXC on other platforms. Otherwise I tried to copy pretty closely the platform-deps crate

cwfitzgerald · 2025-09-24T22:44:55Z

wgpu-precompile-macro/src/lib.rs

+        CompileTarget::Msl => {
+            // Related features: metal
+            #[cfg(feature = "metal")]
+            quote! {
+                target_vendor = "apple"
+            }
+            // Always false
+            #[cfg(not(feature = "metal"))]
+            always_false
+        }


I have concerns with how far from the syntax of the normal cfgs in wgpu-core etc are. Would there be some way of duplicating the cfg (in the build script) so that this code used the same cfg as wgpu-core does? Keeping this updated is going to be difficult. Additionally this is running at build time, so we can use runtime cfg constructs rather than compile time, which are easier to read.

The macro crate isn't necessarily compiled specifically for a target, and there's not a way currently to know what target you're compiling for. So all of the platform specific cfgs have to be in the generated code, and the features have to be in the non-generated code. So I'm not sure there's a clean way to match wgpu-core unless I'm misunderstanding

I do generally agree that what we have right now isn't ideal and wouldn't be very maintainable.

wgpu-precompile-macro/src/lib.rs

SupaMaggie70Incorporated · 2025-09-25T03:05:02Z

This is really cool stuff! I have some high level concerns about this and some initial nits when I'm looking it over. Very happy to see progress here though! Doesn't seem to be as awful a transform as I feared.

I do have some conserns about how resource mappings are handled, as the naga backend does take information about where to put various resources (flattening bindings on metal/dx12, dealing with bindless samplers on dx12, etc)

I think we should split this into two separate bits of code. One which is a normal function which does the shader compilation and stuff, then a macro which is a frontend to that function. This will help separation of concerns and will also support doing precompilation in a build script or an external tool, which I would expect to be a more common desire (and I personally would want, as opposed to a proc macro). It should also vastly simplify the cfg situation.

Happy to discuss more about this plan. We can have two crates (wgpu-precompile and wgpu-precompile-macro).

@cwfitzgerald Yeah I kind of just sent it with this PR lol. Do you want to discuss this on a call at some point? Also, do you think that the eventual solution will actually involve a macro, even if it isn't the most popular option?

SupaMaggie70Incorporated · 2025-09-25T03:10:30Z

Also feel free to bring this to matrix

cwfitzgerald · 2025-09-25T03:14:30Z

Do you want to discuss this on a call at some point?

Sure if that would make it easier to sync up, that's fine! I'll message you on matrix to set this up.

Also, do you think that the eventual solution will actually involve a macro, even if it isn't the most popular option?

I think you could implement the macro in terms of the regular function, so there's no code duplication.

wgpu-precompile-macro/src/lib.rs

Co-authored-by: Connor Fitzgerald <[email protected]>

…g check against non-uploaded image)

…macro crate, and the whole gang

SupaMaggie70Incorporated and others added 18 commits September 11, 2025 23:04

Initial work for macro

ce14ac3

More work for macro that wasn't commited?

d974548

Worked on macro a lot

c1f9549

Removed shader stage parameter from the thing

95491ce

Updated macro slightly

9d52aca

Updated some stuff

80509c6

Made more stuff build properly

59db4c7

Refactored slightly

1dfc2b2

Tried to fix a compile fail

f7ecc4c

Merge branch 'trunk' into precompiled-shaders-macro

930b35b

Added precompile test

309919a

Added more related macros

bac4a48

Worked more on the testing and such. Currently glsl include is broken…

12d5e26

… (unsure why), testing if its only on macos/msl

Added comment explaining why compiled spv is in tree

a28790d

Made entry point specific to each backend in passthrough

1a1d563

Enhanced macro slightly

2f0293d

Worked a little more

2229f05

Fixed vulkan backend y flip

11c9bca

SupaMaggie70Incorporated mentioned this pull request Sep 12, 2025

Precompiled Shaders #3103

Open

SupaMaggie70Incorporated added 6 commits September 12, 2025 13:38

Allowed precompiling dxil (will break everything again)

662c76c

Added a comment explaining why entry point name differs by shader lan…

9d45fd9

…guage

Tried to fix various compile errors/typos

78250d3

Added changelog entry

9c8988e

Updated changelog entry again for the spirv passthrough change

f12f60f

Refactored to allow compilation guards

d1f1675

SupaMaggie70Incorporated marked this pull request as ready for review September 15, 2025 18:37

SupaMaggie70Incorporated and others added 4 commits September 15, 2025 13:37

Reformatted cargo.toml

ed9934b

Tried updating syn and quote to see if it fixes anything

be1504d

Merge branch 'trunk' into precompiled-shaders-macro

038fb11

Attempted to fix windows compilation

b66e991

SupaMaggie70Incorporated mentioned this pull request Sep 15, 2025

Wgpu-dependency tests fail on dev dependencies #8231

Closed

Fixed dependency tests

9281184

SupaMaggie70Incorporated and others added 2 commits September 20, 2025 22:38

Updated deps to reflect gfx-rs#8246

9e41b07

Merge branch 'trunk' into precompiled-shaders-macro

f2a3620

cwfitzgerald requested a review from Copilot September 24, 2025 04:39

Copilot AI reviewed Sep 24, 2025

View reviewed changes

wgpu/src/macros.rs Show resolved Hide resolved

wgpu-precompile-macro/src/lib.rs Outdated Show resolved Hide resolved

wgpu-precompile-macro/src/lib.rs Outdated Show resolved Hide resolved

SupaMaggie70Incorporated and others added 2 commits September 23, 2025 23:44

Update wgpu-precompile-macro/src/lib.rs

113a75e

Co-authored-by: Copilot <[email protected]>

Merge branch 'trunk' into precompiled-shaders-macro

c860d50

cwfitzgerald self-assigned this Sep 24, 2025

Merge branch 'trunk' into precompiled-shaders-macro

5b9616e

cwfitzgerald requested changes Sep 25, 2025

View reviewed changes

I don't even understand what happened here

6ea7ea7

Now removes temporary folder regardless of crashing/etc

a0f613b

cwfitzgerald reviewed Sep 25, 2025

View reviewed changes

wgpu-precompile-macro/src/lib.rs Outdated Show resolved Hide resolved

SupaMaggie70Incorporated and others added 7 commits September 24, 2025 23:01

Update wgpu-precompile-macro/src/lib.rs

c01e505

Co-authored-by: Connor Fitzgerald <[email protected]>

Merge branch 'trunk' into precompiled-shaders-macro

fc4e1c7

Updated to latest dx12 mesh shader changes

bedc327

Tried to improve error messages for dxc precompiling

dff0697

Made dependency tests also check build deps

8344a81

Fixed tests sorta (broke mesh shader test most likely due to requirin…

537b32e

…g check against non-uploaded image)

Other part to last commit

b06ce5e

SupaMaggie70Incorporated marked this pull request as draft September 25, 2025 15:29

Initial work on new shaders crate that will be used by wgpu-hal, the …

1675170

…macro crate, and the whole gang

Precompiled shaders macro #8217

Are you sure you want to change the base?

Precompiled shaders macro #8217

Uh oh!

Conversation

SupaMaggie70Incorporated commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SupaMaggie70Incorporated commented Sep 15, 2025

Uh oh!

cwfitzgerald commented Sep 24, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SupaMaggie70Incorporated commented Sep 24, 2025

Uh oh!

SupaMaggie70Incorporated commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cwfitzgerald left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cwfitzgerald Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

SupaMaggie70Incorporated Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

cwfitzgerald Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

SupaMaggie70Incorporated Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cwfitzgerald Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

SupaMaggie70Incorporated Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

SupaMaggie70Incorporated Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SupaMaggie70Incorporated commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SupaMaggie70Incorporated commented Sep 25, 2025

Uh oh!

cwfitzgerald commented Sep 25, 2025

Uh oh!

Uh oh!

Uh oh!

SupaMaggie70Incorporated commented Sep 12, 2025 •

edited

Loading

SupaMaggie70Incorporated commented Sep 24, 2025 •

edited

Loading

SupaMaggie70Incorporated Sep 25, 2025 •

edited

Loading

SupaMaggie70Incorporated commented Sep 25, 2025 •

edited

Loading