Skip to content

Conversation

copybara-service[bot]
Copy link

@copybara-service copybara-service bot commented Jun 23, 2025

Add Hermetic C++ Toolchains for Linux x86_64 builds.

Hermetic toolchains provide builds that are isolated from the host system, cutting down on unexpected dependencies and side effects.

By default, JAX now builds for Linux x86_64 architectures (both CPU and CUDA-enabled GPU) using hermetic C++ toolchains. For non-hermetic builds, add the flag --config=clang_local. For remote builds with a non-hermetic toolchain, simply append _clang_local to the existing RBE flag. For example, if hermetic RBE build runs with --config=rbe_linux_cpu, the non-hermetic version would be --config=rbe_linux_cpu_clang_local.

Example 1: Run CPU tests for Linux x86_64

For hermetic tests run the command for Linux x86_64 build without env variables CC, CXX, etc.:

bazel test //tests:cpu_tests \
	--config=avx_posix \
	--config=clang \
	--linkopt=-lrt \
	--host_linkopt=-lrt \
	--repo_env=HERMETIC_PYTHON_VERSION="3.13" \
	--test_tag_filters=-multiaccelerator

For non-hermetic tests use commands with the flag "--config=clang_local" and env variables CC, CXX, etc.:

bazel test //tests:cpu_tests \
	--config=clang_local \
	--config=avx_posix \
	--config=clang \
	--linkopt=-lrt \
	--host_linkopt=-lrt \
	--repo_env=HERMETIC_PYTHON_VERSION="3.13" \
	--test_tag_filters=-multiaccelerator \
	--action_env=CLANG_COMPILER_PATH=/usr/lib/llvm-18/bin/clang \
	--repo_env=CC=/usr/lib/llvm-18/bin/clang \
	--repo_env=CXX=/usr/lib/llvm-18/bin/clang++ \
	--repo_env=BAZEL_COMPILER=/usr/lib/llvm-18/bin/clang \
	--action_env=CCC_OVERRIDE_OPTIONS="^--gcc-install-dir=/usr/lib/gcc/x86_64-linux-gnu/13"

Example 2: Run CUDA GPU tests for Linux x86_64

For hermetic tests, run the command without env variables CC, CXX, etc.:
bazel test //tests:gpu_tests \
	--config=cuda \
	--config=clang \
	--config=build_cuda_with_nvcc \
	--config=avx_posix \
	--linkopt=-lrt \
	--host_linkopt=-lrt \
	--test_tag_filters=-multiaccelerator \
	--repo_env=HERMETIC_PYTHON_VERSION=3.13

For non-hermetic tests use commands with the flag "--config=clang_local" and env variables CC, CXX, etc.:

bazel test //tests:gpu_tests \
	--config=clang_local \
	--config=avx_posix \
	--config=build_cuda_with_nvcc \
	--config=clang \
	--config=cuda \
	--linkopt=-lrt \
	--host_linkopt=-lrt \
	--action_env=CCC_OVERRIDE_OPTIONS="^--gcc-install-dir=/usr/lib/gcc/x86_64-linux-gnu/9" \
	--test_tag_filters=-multiaccelerator \
	--action_env=CLANG_COMPILER_PATH=/usr/lib/llvm-18/bin/clang \
	--repo_env=CC=/usr/lib/llvm-18/bin/clang \
	--repo_env=CXX=/usr/lib/llvm-18/bin/clang++ \
	--repo_env=BAZEL_COMPILER=/usr/lib/llvm-18/bin/clang \
	--repo_env=HERMETIC_PYTHON_VERSION=3.13

Example 3: Run RBE build for Linux x86_64

For hermetic build, run the same command as before modification:
bazel build \
    --config=ci_linux_x86_64 \
    --repo_env=HERMETIC_PYTHON_VERSION=3.13 \
    //jaxlib/tools:jaxlib_wheel

For non-hermetic builds, add "_clang_local" suffix to RBE configuration:
bazel build \
    --config=ci_linux_x86_64_clang_local \
    --repo_env=HERMETIC_PYTHON_VERSION=3.13 \
    //jaxlib/tools:jaxlib_wheel

We don't support other "rbe_*_clang_local" configuration flags, and you could construct your own combinations.

@copybara-service copybara-service bot force-pushed the test_774963837 branch 9 times, most recently from b95e587 to a401a8a Compare July 16, 2025 21:24
@copybara-service copybara-service bot changed the title Add Hermetic C++ Toolchains for JAX project. Add Hermetic C++ Toolchains for Linux x86_64 builds. Jul 16, 2025
@copybara-service copybara-service bot force-pushed the test_774963837 branch 3 times, most recently from e1f12fe to f60b0e1 Compare July 23, 2025 17:18
Hermetic toolchains provide builds that are isolated from the host system, cutting down on unexpected dependencies and side effects.

By default, JAX now builds for Linux x86_64 architectures (both CPU and CUDA-enabled GPU) using hermetic C++ toolchains. For non-hermetic builds, add the flag --config=clang_local. For remote builds with a non-hermetic toolchain, simply append _clang_local to the existing RBE flag. For example, if hermetic RBE build runs with --config=rbe_linux_cpu, the non-hermetic version would be --config=rbe_linux_cpu_clang_local.

    Example 1: Run CPU tests for Linux x86_64

    For hermetic tests run the command for Linux x86_64 build without env variables CC, CXX, etc.:

	bazel test //tests:cpu_tests \
		--config=avx_posix \
		--config=clang \
		--linkopt=-lrt \
		--host_linkopt=-lrt \
		--repo_env=HERMETIC_PYTHON_VERSION="3.13" \
		--test_tag_filters=-multiaccelerator

    For non-hermetic tests use commands with the flag "--config=clang_local" and env variables CC, CXX, etc.:

	bazel test //tests:cpu_tests \
		--config=clang_local \
		--config=avx_posix \
		--config=clang \
		--linkopt=-lrt \
		--host_linkopt=-lrt \
		--repo_env=HERMETIC_PYTHON_VERSION="3.13" \
		--test_tag_filters=-multiaccelerator \
		--action_env=CLANG_COMPILER_PATH=/usr/lib/llvm-18/bin/clang \
		--repo_env=CC=/usr/lib/llvm-18/bin/clang \
		--repo_env=CXX=/usr/lib/llvm-18/bin/clang++ \
		--repo_env=BAZEL_COMPILER=/usr/lib/llvm-18/bin/clang \
		--action_env=CCC_OVERRIDE_OPTIONS="^--gcc-install-dir=/usr/lib/gcc/x86_64-linux-gnu/13"

    Example 2: Run CUDA GPU tests for Linux x86_64

    For hermetic tests, run the command without env variables CC, CXX, etc.:
	bazel test //tests:gpu_tests \
		--config=cuda \
		--config=clang \
		--config=build_cuda_with_nvcc \
		--config=avx_posix \
		--linkopt=-lrt \
		--host_linkopt=-lrt \
		--test_tag_filters=-multiaccelerator \
		--repo_env=HERMETIC_PYTHON_VERSION=3.13

    For non-hermetic tests use commands with the flag "--config=clang_local" and env variables CC, CXX, etc.:

	bazel test //tests:gpu_tests \
		--config=clang_local \
		--config=avx_posix \
		--config=build_cuda_with_nvcc \
		--config=clang \
		--config=cuda \
		--linkopt=-lrt \
		--host_linkopt=-lrt \
		--action_env=CCC_OVERRIDE_OPTIONS="^--gcc-install-dir=/usr/lib/gcc/x86_64-linux-gnu/9" \
		--test_tag_filters=-multiaccelerator \
		--action_env=CLANG_COMPILER_PATH=/usr/lib/llvm-18/bin/clang \
		--repo_env=CC=/usr/lib/llvm-18/bin/clang \
		--repo_env=CXX=/usr/lib/llvm-18/bin/clang++ \
		--repo_env=BAZEL_COMPILER=/usr/lib/llvm-18/bin/clang \
		--repo_env=HERMETIC_PYTHON_VERSION=3.13

    Example 3: Run RBE build for Linux x86_64

    For hermetic build, run the same command as before modification:
    bazel build \
        --config=ci_linux_x86_64 \
        --repo_env=HERMETIC_PYTHON_VERSION=3.13 \
        //jaxlib/tools:jaxlib_wheel

    For non-hermetic builds, add "_clang_local" suffix to RBE configuration:
    bazel build \
        --config=ci_linux_x86_64_clang_local \
        --repo_env=HERMETIC_PYTHON_VERSION=3.13 \
        //jaxlib/tools:jaxlib_wheel

    We don't support other "rbe_*_clang_local" configuration flags, and you could construct your own combinations.

PiperOrigin-RevId: 786346447
@copybara-service copybara-service bot merged commit bf25960 into main Jul 23, 2025
@copybara-service copybara-service bot deleted the test_774963837 branch July 23, 2025 18:00
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Jul 23, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 783942761
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Jul 23, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 783942761
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Jul 23, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 783942761
copybara-service bot pushed a commit to tensorflow/tensorflow that referenced this pull request Jul 23, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 783942761
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Jul 24, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 783942761
copybara-service bot pushed a commit to tensorflow/tensorflow that referenced this pull request Jul 24, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 783942761
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Jul 24, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 786533277
copybara-service bot pushed a commit to tensorflow/tensorflow that referenced this pull request Jul 24, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 786533277
CodeHotel pushed a commit to CodeHotel/tensorflow that referenced this pull request Jul 28, 2025
…SPACE files.

Upgrading manylinux compliancy tag in [JAX PR](jax-ml/jax#29672) enabled building targets with linked `nvshmem` libraries.

PiperOrigin-RevId: 786533277
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant