[libc++][WIP] Move to Github hosted builders

[ldionne]

Libc++ experiences periodic outages of our Github actions builders, which are quite disruptive. Unfortunately, the Github self-hosted runner setup is very opaque. It works extremely well when it works, but most contributors are unable to troubleshoot issues related to the setup when they happen.

This patch attempts to move the libc++ builders to normal Github hosted runners. This is primarily to evaluate the feasibility of such a change, which would simplify our CI setup but may not scale sufficiently for our needs.

[llvmbot]

@llvm/pr-subscribers-libcxx

@llvm/pr-subscribers-github-workflow

Author: Louis Dionne (ldionne)

Changes

Libc++ experiences periodic outages of our Github actions builders, which are quite disruptive. Unfortunately, the Github self-hosted runner setup is very opaque. It works extremely well when it works, but most contributors are unable to troubleshoot issues related to the setup when they happen.

This patch attempts to move the libc++ builders to normal Github hosted runners. This is primarily to evaluate the feasibility of such a change, which would simplify our CI setup but may not scale sufficiently for our needs.

---

Full diff: https://github.com/llvm/llvm-project/pull/109720.diff

1 Files Affected:

• (modified) .github/workflows/libcxx-build-and-test.yaml (+16-16)

diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index b5e60781e00064..bdf4c1b74fdbaa 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -49,7 +49,13 @@ env:
 jobs:
   stage1:
     if: github.repository_owner == 'llvm'
-    runs-on: libcxx-runners-8-set
+    runs: &docker-image
+      using: 'docker'
+      image: 'libcxx/utils/ci/Dockerfile'
+      args:
+        - BASE_IMAGE: ghcr.io/actions/actions-runner:2.317.0
+        - GCC_LATEST_VERSION: 14
+        - LLVM_HEAD_VERSION: 20
     continue-on-error: false
     strategy:
       fail-fast: false
@@ -84,7 +90,8 @@ jobs:
             **/crash_diagnostics/*
   stage2:
     if: github.repository_owner == 'llvm'
-    runs-on: libcxx-runners-8-set
+    runs:
+      <<: *docker-image
     needs: [ stage1 ]
     continue-on-error: false
     strategy:
@@ -136,12 +143,15 @@ jobs:
       matrix:
         config: [
           'generic-abi-unstable',
+          'generic-asan',
+          'generic-cxx26',
           'generic-hardening-mode-debug',
           'generic-hardening-mode-extensive',
           'generic-hardening-mode-fast',
           'generic-hardening-mode-fast-with-abi-breaks',
           'generic-merged',
           'generic-modules-lsv',
+          'generic-msan',
           'generic-no-exceptions',
           'generic-no-experimental',
           'generic-no-filesystem',
@@ -155,25 +165,15 @@ jobs:
           'generic-no-rtti',
           'generic-optimized-speed',
           'generic-static',
+          'generic-tsan',
+          'generic-ubsan',
           # TODO Find a better place for the benchmark and bootstrapping builds to live. They're either very expensive
           # or don't provide much value since the benchmark run results are too noise on the bots.
           'benchmarks',
           'bootstrapping-build'
         ]
-        machine: [ 'libcxx-runners-8-set' ]
-        include:
-        - config: 'generic-cxx26'
-          machine: libcxx-runners-8-set
-        - config: 'generic-asan'
-          machine: libcxx-runners-8-set
-        - config: 'generic-tsan'
-          machine: libcxx-runners-8-set
-        - config: 'generic-ubsan'
-          machine: libcxx-runners-8-set
-        # Use a larger machine for MSAN to avoid timeout and memory allocation issues.
-        - config: 'generic-msan'
-          machine: libcxx-runners-8-set
-    runs-on: ${{ matrix.machine }}
+    runs:
+      <<: *docker-image
     steps:
       - uses: actions/checkout@v4
       - name: ${{ matrix.config }}

[tstellar]

Would you be able to use ccache or sccache for this job? It might be necessary to keep the usage down to manageable levels. See https://github.com/hendrikmuhs/ccache-action.

[ldionne]

Would you be able to use ccache or sccache for this job? It might be necessary to keep the usage down to manageable levels. See https://github.com/hendrikmuhs/ccache-action.

We could try that. I am a bit worried about running the test suite under ccache, though. IMO another alternative that could bring test times down is to run more configurations under Clang Modules, which become a lot faster once we land #107638.

We could also build the dylib (which takes ~1 minute) in a single job and then reuse it from various downstream jobs. I can look into that once we get the basic thing working -- at the moment our CI has been down for about a week and everything is stalled.

[ldionne]

Untagging to prevent spam.

[EricWF]

This won't work because we can't run containers on images within the container image that the bot runs in.

[boomanaiden154]

This won't work because we can't run containers on images within the container image that the bot runs in.

This is switching over to the Github Actions runners which are hosted in VMs, so can run containers just fine.

[ldionne]

Closing in favor of #110303.