Add CUDA forward compatibility hook #948
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change adds an nvidia-cdi-hook enable-cuda-compat hook that checks the container for cuda compat libs and updates /etc/ld.so.conf.d to include their parent folder if their driver major version is sufficient. This allows CUDA Forward Compatibility to be used when this is not available through the libnvidia-container. Signed-off-by: Evan Lezar <[email protected]>
This change adds the enable-cuda-compat hook to the incomming OCI runtime spec if the allow-cuda-compat-libs-from-container feature flag is not enabled. An update-ldcache hook is also injected to ensure that the required folders are processed. Signed-off-by: Evan Lezar <[email protected]>
Signed-off-by: Evan Lezar <[email protected]>
elezar
requested review from
cdesiniotis,
tariq1890,
klueska and
ArangoGutierrez
Signed-off-by: Evan Lezar <[email protected]>
Signed-off-by: Evan Lezar <[email protected]>
Signed-off-by: Evan Lezar <[email protected]>
elezar
force-pushed
the
add-compat-lib-hook
branch
from
3307cb1 to
c1bac28
Compare
ArangoGutierrez
approved these changes
Feb 28, 2025
luodw
reviewed
May 24, 2025
| default: | ||
| return []string{"mode", "graphics", "feature-gated"} | ||
| return []string{"feature-gated", "graphics", "mode"} |
There was a problem hiding this comment.
@elezar Hi, I have a question here. In this modifier order, it will create CreateContainer Hook like this ["enable-cuda-compat", "update-ldcache", "create-symlinks"]. As "update-ldcache" runs before "create-symlinks", so hook "create-symlinks" do some bind mount so(dynamic link library) in container will not add into ldcache?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With #877 the default behaviour of the NVIDIA Container Runtime / NVIDIA Container Runtime Hook was changed to not mount compat libraries from the container into the container. This removed "automatic" support for CUDA Forward compatibility.
This change attempts to address this by adding a
createContainerHookthat will create a file in/etc/ld.so.conf.d/in the container to ensure that the/usr/local/cuda/compatlibraries are added to the ldcache over the libraries mounted from the host. The provided host diver version is compared to the version of the compat libraries in the container and the config update is only performed if the compat libraries are newer than the host drivers.Note that the hook only creates a file in the container's file system and does not perform any mount operations. This means that this mechanism is not present the same vulnerabilities causing CVE-2024-0132 and CVE-2025-23359.
In the case of the legacy runtime, this behaviour is only triggered if the
allow-cuda-compat-libs-from-containerfeature flag is not enabled. The CDI spec generation has also been extended to include this hook.This backports #906