Skip to content

cmake: clean up external project logic for vulkan-shaders-gen #14179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 16, 2025

Conversation

bandoti
Copy link
Collaborator

@bandoti bandoti commented Jun 13, 2025

This PR cleans up the the ExternalProject_Add logic for vulkan-shaders-gen when multi-configuration generators are used.

These changes expand on #14047, which is to fix an issue causing the llama-cpp-rs rust bindings to no longer compile for vulkan, see: utilityai/llama-cpp-rs#747

@github-actions github-actions bot added Vulkan Issues specific to the Vulkan backend devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning labels Jun 13, 2025
@bandoti bandoti requested a review from slaren June 13, 2025 23:08
@0cc4m
Copy link
Collaborator

0cc4m commented Jun 15, 2025

This is unrelated to this PR, but since we went with the external project logic, cmake doesn't recognize file changes in shaders properly anymore. Sometimes shaders aren't rebuilt, which can be annoying to figure out. You can resolve using "--clean-first", but previously it was working fine. Is this something we can fix?

@bandoti
Copy link
Collaborator Author

bandoti commented Jun 15, 2025

Oh no! That's not good.

At first glance I notice that _ggml_vk_shader_deps globs the shader files and then the set command is used afterward, appending the vulkan-shaders-gen targets.

My first guess is to separate those dependencies because it might be messing with the list.

Another option is that we could regenerate the list of globbed files at build time instead of generate-time using the glob flag:

Added in version 3.12: If the CONFIGURE_DEPENDS flag is specified, CMake will add logic to the main build system check target to rerun the flagged GLOB commands at build time. If any of the outputs change, CMake will regenerate the build system.

@bandoti
Copy link
Collaborator Author

bandoti commented Jun 15, 2025

I am happy to put the fix in with this one as it is related—and small enough. Thanks for letting me know. 😊

@bandoti bandoti requested a review from 0cc4m June 15, 2025 15:51
@bandoti
Copy link
Collaborator Author

bandoti commented Jun 15, 2025

@0cc4m I added you as a reviewer primarily to check that the shader re-compile works. I spot tested on my system and it seems to rebuild properly when the shaders get modified. Thanks!

@0cc4m
Copy link
Collaborator

0cc4m commented Jun 16, 2025

Thank you, I couldn't reproduce the issue currently so I can't say for certain if this fixes it, but I'll watch it.

@AsbjornOlling
Copy link
Contributor

I tested this by running nix build 'github:bandoti/llama.cpp/cmake-vulkan-multi-config#vulkan'

This compiles without error, unlike the current master branch.

For comparison, running nix build 'github:ggml-org/llama.cpp/b5679#vulkan' fails.

@bandoti bandoti merged commit 0dbcabd into ggml-org:master Jun 16, 2025
47 checks passed
@bandoti
Copy link
Collaborator Author

bandoti commented Jun 16, 2025

@AsbjornOlling Thanks for the contribution/help fixing the issue. 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning Vulkan Issues specific to the Vulkan backend
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants