Skip to content

Precommit release #973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 68 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
3093657
[workflows] Port buildkite Windows config to GitHub actions
tstellar Feb 16, 2024
673c078
Workflow improvements:
tstellar Feb 17, 2024
9e9aee1
Automatically continue the CI job when it times out
tstellar Apr 29, 2024
a7ff883
Re-write continue-timeout-job to be more generic
tstellar May 3, 2024
3ff27a4
UPdate continue time-out-job
tstellar May 7, 2024
0406329
Make ci work on all platforms
tstellar May 7, 2024
97bde01
DEBUG Add a file so that the CI triggers
tstellar Apr 27, 2024
dd4e153
XXX: Add cI-tsts
tstellar May 7, 2024
a99fae8
XXX: disable windows tests
tstellar May 7, 2024
d515083
Fixes
tstellar May 7, 2024
857b97c
FXX
tstellar May 7, 2024
bad285c
Fix
tstellar May 7, 2024
486a771
Fix
tstellar May 7, 2024
f0ecc4b
Fix
tstellar May 7, 2024
ba5c9c3
XXX
tstellar May 7, 2024
7752d58
XXX: fix
tstellar May 7, 2024
a913c91
XXX: fix
tstellar May 7, 2024
98971b9
XXX: fix
tstellar May 7, 2024
27c3bfc
XXX:
tstellar May 7, 2024
a2c09a8
XXX:
tstellar May 7, 2024
ac5203d
Install ninja
tstellar May 7, 2024
2a5542f
XXX fix
tstellar May 7, 2024
6356bcb
XXX:
tstellar May 7, 2024
3551d83
XXX: fix timeout
tstellar May 7, 2024
6cdb82b
XXX
tstellar May 7, 2024
2cb707c
Fix continue timeout
tstellar May 7, 2024
0aa9e5c
Fix job name for timeouts
tstellar May 7, 2024
6771a8e
Fix timeout job
tstellar May 7, 2024
fb45efe
XXX: Fix
tstellar May 7, 2024
1eef1c7
Fix continue job
tstellar May 7, 2024
2e19de8
Fix timeout
tstellar May 7, 2024
5e5cea3
Fix restart
tstellar May 8, 2024
e1595ef
Update matrix
tstellar May 8, 2024
c50c8f9
Update matrix
tstellar May 8, 2024
dfb9e3b
Update matrix
tstellar May 8, 2024
448bfe7
Fix matrix
tstellar May 8, 2024
8a17f3a
Fixed
tstellar May 8, 2024
8ef7b6e
Fix job_id
tstellar May 8, 2024
36b082a
Fix sccache upload
tstellar May 8, 2024
c1e2cf6
Simplify-timeout
tstellar May 8, 2024
c39e786
Update continue-timeout job
tstellar May 8, 2024
e4310b5
Updates
tstellar May 8, 2024
17d2f41
Updates
tstellar May 8, 2024
5eb256d
Add missing files
tstellar May 8, 2024
372d11f
XX: Fix
tstellar May 8, 2024
074961a
Fix timeout id
tstellar May 8, 2024
c982084
Fixes/debug
tstellar May 8, 2024
b95e966
Fix date compare
tstellar May 8, 2024
d879fb4
Fix timeout detection
tstellar May 8, 2024
11885d2
Fix artifact name
tstellar May 8, 2024
600bf47
Fix typo
tstellar May 8, 2024
7052a03
Fix continue
tstellar May 8, 2024
9e349ca
XXX: test fail
tstellar May 8, 2024
398beb8
continue tmeout debug
tstellar May 8, 2024
7a24e50
Revert "XXX: test fail"
tstellar May 8, 2024
3934d79
timeout minutes
tstellar May 8, 2024
ba9641f
Fix pr-sccache for mac
tstellar May 8, 2024
784fc3f
timeout 1
tstellar May 8, 2024
024db66
XXX: fix
tstellar May 8, 2024
84d7617
Fix pr-sccache-save
tstellar May 8, 2024
f3a4414
Fix pr-sccache-save
tstellar May 8, 2024
3779e37
Fix sccache download
tstellar May 9, 2024
f67668a
Fix typo
tstellar May 9, 2024
79bc911
Fix typo
tstellar May 9, 2024
a3bbacc
Fix typo
tstellar May 9, 2024
c1bb3e2
Fix typo
tstellar May 9, 2024
b452883
Fix typo
tstellar May 9, 2024
b0883ec
Fix typo
tstellar May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 157 additions & 0 deletions .github/workflows/ci-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
name: "CI Tests"

permissions:
contents: read
actions: write

on:
pull_request:
types:
- opened
- synchronize
- reopened
# When a PR is closed, we still start this workflow, but then skip
# all the jobs, which makes it effectively a no-op. The reason to
# do this is that it allows us to take advantage of concurrency groups
# to cancel in progress CI jobs whenever the PR is closed.
- closed
branches:
- main

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: True

jobs:
compute-test-configs:
name: "Compute Configurations to Test"
if: github.event.action != 'closed'
runs-on: ubuntu-22.04
outputs:
projects: ${{ steps.vars.outputs.projects }}
check-targets: ${{ steps.vars.outputs.check-targets }}
test-build: ${{ steps.vars.outputs.check-targets != '' }}
test-platforms: ${{ steps.platforms.outputs.result }}
steps:
- name: Fetch LLVM sources
uses: actions/checkout@v4
with:
fetch-depth: 2

- name: Compute projects to test
id: vars
uses: ./.github/workflows/compute-projects-to-test

- name: Compute platforms to test
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #v7.0.1
id: platforms
with:
script: |
linuxConfig = {
name: "linux-x86_64",
runs_on: "ubuntu-22.04"
}
windowsConfig = {
name: "windows-x86_64",
runs_on: "windows-2022"
}
macConfig = {
name: "macos-x86_64",
runs_on: "macos-13"
}
macArmConfig = {
name: "macos-aarch64",
runs_on: "macos-14"
}
configs = [
linuxConfig,
windowsConfig,
macConfig,
macArmConfig
]
const base_ref = process.env.GITHUB_BASE_REF;
if (base_ref.startsWith('release/')) {
// This is a pull request against a release branch.
}
return configs;
ci-build-test:
# If this job name is changed, then we need to update the job-name
# paramater for the timeout-save step below.
name: "Build"
needs:
- compute-test-configs
permissions:
actions: write #pr-sccache-save may delete artifacts.
runs-on: ${{ matrix.runs_on }}
strategy:
fail-fast: false
matrix:
include: ${{ fromJson(needs.compute-test-configs.outputs.test-platforms) }}
if: needs.compute-test-configs.outputs.test-build == 'true'
steps:
- name: Fetch LLVM sources
uses: actions/checkout@v4

- name: Timeout Restore
id: timeout
uses: ./.github/workflows/timeout-restore
with:
artifact-name-suffix: ${{ matrix.name }}

- name: Setup Windows
uses: llvm/actions/setup-windows@main
if: ${{ runner.os == 'Windows' }}
with:
arch: amd64

- name: Install Ninja
uses: llvm/actions/install-ninja@main

- name: Setup sccache
uses: hendrikmuhs/ccache-action@v1
with:
max-size: 2G
variant: sccache
key: ci-${{ matrix.name }}

- name: Restore sccache from previous PR run
uses: ./.github/workflows/pr-sccache-restore
with:
artifact-name-suffix: ${{ matrix.name }}

- name: Configure
if: ${{ steps.timeout.outputs.exists != 'true' }}
shell: bash
run: |
cmake -B build -GNinja \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="${{ needs.compute-test-configs.outputs.projects }}" \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_LIT_ARGS="-v --no-progress-bar" \
-DCMAKE_C_COMPILER_LAUNCHER=sccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache \
-S llvm
- name: Build
shell: bash
timeout-minutes: 1
run: |
ninja -C build -k 0 ${{ needs.compute-test-configs.outputs.check-targets }}
- name: Timeout Save
if: always()
uses: ./.github/workflows/timeout-save
with:
job-name: "Build (${{ matrix.name }}, ${{ matrix.runs_on }})"
artifact-name-suffix: ${{ matrix.name }}
timeout-step: "Build"
timeout-minutes: 1

- name: Save sccache for next PR run
if: always()
uses: ./.github/workflows/pr-sccache-save
with:
artifact-name-suffix: ${{ matrix.name }}
21 changes: 21 additions & 0 deletions .github/workflows/compute-projects-to-test/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: 'Compute Projects To Test'
inputs:
projects:
required: false
type: 'string'

outputs:
check-targets:
description: "A space delimited list of check-targets to pass to ninja."
value: ${{ steps.compute-projects.outputs.check-targets }}

projects:
description: "A semi-colon delimited list of projects to pass to -DLLVM_ENABLE_PROJECTS."
value: ${{ steps.compute-projects.outputs.projects }}

runs:
using: "composite"
steps:
- id: compute-projects
run: .github/workflows/compute-projects-to-test/compute-projects-to-test.sh ${{ inputs.projects }}
shell: bash
221 changes: 221 additions & 0 deletions .github/workflows/compute-projects-to-test/compute-projects-to-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
#!/usr/bin/env bash
#===----------------------------------------------------------------------===##
#
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
#
#===----------------------------------------------------------------------===##

#
# This file generates a Buildkite pipeline that triggers the various CI jobs for
# the LLVM project during pre-commit CI.
#
# See https://buildkite.com/docs/agent/v3/cli-pipeline#pipeline-format.
#
# As this outputs a yaml file, it's possible to log messages to stderr or
# prefix with "#".


set -eu
set -o pipefail

# Environment variables script works with:

# Set by GitHub
: ${GITHUB_OUTPUT:=}
: ${RUNNER_OS:=}

# Allow users to specify which projects to build.
all_projects="bolt clang clang-tools-extra compiler-rt cross-project-tests flang libc libclc lld lldb llvm mlir openmp polly pstl"
if [ "$#" -ne 0 ]; then
wanted_projects="${@}"
else
wanted_projects="${all_projects}"
fi

# List of files affected by this commit
: ${MODIFIED_FILES:=$(git diff --name-only HEAD~1...HEAD)}

echo "Files modified:" >&2
echo "$MODIFIED_FILES" >&2
modified_dirs=$(echo "$MODIFIED_FILES" | cut -d'/' -f1 | sort -u)
echo "Directories modified:" >&2
echo "$modified_dirs" >&2
echo "wanted_projects: $wanted_projects"

function remove-unwanted-projects() {
projects=${@}
for project in ${projects}; do
if echo "$wanted_projects" | tr ' ' '\n' | grep -q -E "^${project}$"; then
echo "${project}"
fi
done
}

function compute-projects-to-test() {
projects=${@}
for project in ${projects}; do
echo "${project}"
case ${project} in
lld)
for p in bolt cross-project-tests; do
echo $p
done
;;
llvm)
for p in bolt clang clang-tools-extra flang lld lldb mlir polly; do
echo $p
done
;;
clang)
for p in clang-tools-extra compiler-rt flang libc lldb openmp cross-project-tests; do
echo $p
done
;;
clang-tools-extra)
echo libc
;;
mlir)
echo flang
;;
*)
# Nothing to do
;;
esac
done
}

function add-dependencies() {
projects=${@}
for project in ${projects}; do
echo "${project}"
case ${project} in
bolt)
for p in lld llvm; do
echo $p
done
;;
cross-project-tests)
for p in lld clang; do
echo $p
done
;;
clang-tools-extra)
for p in llvm clang; do
echo $p
done
;;
compiler-rt|libc|openmp)
echo clang lld
;;
flang|lldb)
for p in llvm clang; do
echo $p
done
;;
lld|mlir|polly)
echo llvm
;;
*)
# Nothing to do
;;
esac
done
}

function exclude-linux() {
projects=${@}
for project in ${projects}; do
case ${project} in
cross-project-tests) ;; # tests failing
lldb) ;; # tests failing
openmp) ;; # https://github.com/google/llvm-premerge-checks/issues/410
*)
echo "${project}"
;;
esac
done
}

function exclude-windows() {
projects=${@}
for project in ${projects}; do
case ${project} in
cross-project-tests) ;; # tests failing
compiler-rt) ;; # tests taking too long
openmp) ;; # TODO: having trouble with the Perl installation
libc) ;; # no Windows support
lldb) ;; # tests failing
bolt) ;; # tests are not supported yet
*)
echo "${project}"
;;
esac
done
}

# Prints only projects that are both present in $modified_dirs and the passed
# list.
function keep-modified-projects() {
projects=${@}
for project in ${projects}; do
if echo "$modified_dirs" | grep -q -E "^${project}$"; then
echo "${project}"
fi
done
}

function check-targets() {
projects=${@}
for project in ${projects}; do
case ${project} in
clang-tools-extra)
echo "check-clang-tools"
;;
compiler-rt)
echo "check-all"
;;
cross-project-tests)
echo "check-cross-project"
;;
lldb)
echo "check-all" # TODO: check-lldb may not include all the LLDB tests?
;;
pstl)
echo "check-all"
;;
libclc)
echo "check-all"
;;
*)
echo "check-${project}"
;;
esac
done
}

# Generic pipeline for projects that have not defined custom steps.
#
# Individual projects should instead define the pre-commit CI tests that suits their
# needs while letting them run on the infrastructure provided by LLVM.

# Figure out which projects need to be built on each platform
modified_projects="$(keep-modified-projects ${all_projects})"
echo "modified_projects: $modified_projects"

if [ "${RUNNER_OS}" = "Linux" ]; then
projects_to_test=$(exclude-linux $(compute-projects-to-test ${modified_projects}))
elif [ "${RUNNER_OS}" = "Windows" ]; then
projects_to_test=$(exclude-windows $(compute-projects-to-test ${modified_projects}))
else
echo "Unknown runner OS: $RUNNER_OS"
exit 1
fi
check_targets=$(check-targets $(remove-unwanted-projects ${projects_to_test}) | sort | uniq)
projects=$(remove-unwanted-projects $(add-dependencies ${projects_to_test}) | sort | uniq)

echo "check-targets=$(echo ${check_targets} | tr ' ' ' ')" >> $GITHUB_OUTPUT
echo "projects=$(echo ${projects} | tr ' ' ';')" >> $GITHUB_OUTPUT

cat $GITHUB_OUTPUT
75 changes: 75 additions & 0 deletions .github/workflows/continue-timeout-job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Continue Timeout Job

on:
workflow_run:
workflows:
- "Windows Precommit Tests"
- "CI Tests"
types:
- completed

permissions:
contents: read

jobs:
restart:
name: "Restart Job"
permissions:
actions: write
runs-on: ubuntu-22.04
if: github.event.workflow_run.conclusion == 'failure'
steps:
- name: "Restart Job"
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #v7.0.1
with:
script: |
const response = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.payload.workflow_run.id
})
job_ids = [];
for (artifact of response.data.artifacts) {
console.log(artifact);
const match = artifact.name.match(/timeout-([0-9]+)/);
console.log(match);
if (!match) {
continue;
}
job_ids.push(match[1]);
// Delete the timeout artifact to prepare for the next run
await github.rest.actions.deleteArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: artifact.id
});
}
if (job_ids.length == 0) {
return;
}
if (job_ids.length > 1) {
// We aren't able to re-start multiple jobs individually, so our
// only option is to restart all failed jobs.
await github.rest.actions.reRunWorkflowFailedJobs({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.payload.workflow_run.id
})
console.log("Restarted workflow: " + context.payload.workflow_run.id);
return;
}
job_id = job_ids[0];
// This function does not exist even though it is in the document
//github.rest.actions.reRunJobForWorkflow({
await github.request('POST /repos/{owner}/{repo}/actions/jobs/{job_id}/rerun', {
owner: context.repo.owner,
repo: context.repo.repo,
job_id: job_id
})
console.log("Restarted job: " + job_id);
30 changes: 30 additions & 0 deletions .github/workflows/get-job-id/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Get Job ID
inputs:
job-name:
required: false
type: 'string'

outputs:
job-id:
description: "A space delimited list of check-targets to pass to ninja."
value: ${{ steps.job-id.outputs.result }}

runs:
using: "composite"
steps:
- uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #v7.0.1
id: job-id
with:
script: |
const job_data = await github.rest.actions.listJobsForWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
});
for (job of job_data.data.jobs) {
console.log(job)
if (job.name == "${{ inputs.job-name }}") {
return job.id
}
}
26 changes: 26 additions & 0 deletions .github/workflows/pr-sccache-restore/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: PR sccache restore

inputs:
artifact-name-suffix:
desciption: The suffix to append to the artifict name (sccache-pr#)
required: true

runs:
using: "composite"
steps:
- uses: ./.github/workflows/unprivileged-download-artifact
id: download-artifact
with:
artifact-name: sccache-pr${{ github.event.pull_request.number }}-${{ inputs.artifact-name-suffix }}

- shell: bash
if: steps.download-artifact.outputs.filename != ''
run: |
# Is this the best way to clear the cache?
rm -Rf .sccache/
unzip ${{ steps.download-artifact.outputs.filename }}
rm ${{ steps.download-artifact.outputs.filename }}
tar --zstd -xf sccache.tar.zst
rm sccache.tar.zst
ls -ltr
50 changes: 50 additions & 0 deletions .github/workflows/pr-sccache-save/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: PR sccache save
inputs:
artifact-name-suffix:
desciption: The suffix to append to the artifict name (sccache-pr#)
required: true

runs:
using: "composite"
steps:
- uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #v7.0.1
with:
script: |
const data = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
name: 'sccache-pr' + context.issue.number + "-${{ inputs.artifact-name-suffix }}"
})
console.log(data.data.artifacts)
if (data.data.artifacts.length == 0) {
return '';
}
console.log(data.data.artifacts[0])
const artifact_id = data.data.artifacts[0].id
// Delete the exisiting artifact so we can upload a new one with the same name.
github.rest.actions.deleteArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: artifact_id
})
- name: Package sccache Directory
shell: bash
run: |
# Dereference symlinks so that this works on Windows.
tar -h -c .sccache | zstd -T0 -c > sccache.tar.zst
- uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
with:
name: 'sccache-pr${{ github.event.number }}-${{ inputs.artifact-name-suffix }}'
path: sccache.tar.zst
retention-days: 7

- shell: bash
run: |
rm sccache.tar.zst
sccache --show-stats
153 changes: 153 additions & 0 deletions .github/workflows/precommit-windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
name: "Windows Precommit Tests"

permissions:
contents: read
actions: write

on:
pull_request:
types:
- opened
- synchronize
- reopened
# When a PR is closed, we still start this workflow, but then skip
# all the jobs, which makes it effectively a no-op. The reason to
# do this is that it allows us to take advantage of concurrency groups
# to cancel in progress CI jobs whenever the PR is closed.
- closed
branches:
- main

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: True

jobs:
compute-projects:
name: "Compute Projects to Test"
if: github.event.action != 'closed' && false
runs-on: ubuntu-22.04
outputs:
projects: ${{ steps.vars.outputs.projects }}
check-targets: ${{ steps.vars.outputs.check-targets }}
test-build: ${{ steps.vars.outputs.check-targets != '' }}
steps:
- name: Fetch LLVM sources
uses: actions/checkout@v4
with:
fetch-depth: 2

- name: Compute projects to test
id: vars
uses: ./.github/workflows/compute-projects-to-test

build-windows:
# If this job name is chagned, then we need to update the job-name
# paramater for the write-timeout-file step below.
name: "Build"
runs-on: windows-2022
permissions:
actions: write #pr-sccache-save may delete artifacts.
outputs:
build-timeout: ${{ steps.build.outputs.timeout }}
build-failed: ${{ steps.build.outputs.failed }}
needs:
- compute-projects
if: ${{ needs.compute-projects.outputs.test-build == 'true' }}
steps:
- name: Download Artifact
uses: actions/download-artifact@6b208ae046db98c579e8a3aa621ab581ff575935 # v4.1.1
with:
pattern: timeout-build
merge-multiple: true

- name: Unpack Artifact
id: timeout-artifact
shell: bash
run: |
if [ -e llvm-project.tar.zst ]; then
tar --zstd -xf llvm-project.tar.zst
rm llvm-project.tar.zst
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "exists=false" >> $GITHUB_OUTPUT
fi
- name: Setup Windows
uses: llvm/actions/setup-windows@main
with:
arch: amd64

- name: Fetch LLVM sources
if: ${{ steps.timeout-artifact.outputs.exists != 'true' }}
uses: actions/checkout@v4

- name: Setup sccache
uses: hendrikmuhs/ccache-action@v1
with:
max-size: 2G
variant: sccache
key: precommit-windows

- name: Restore sccache from previous PR run
uses: ./.github/workflows/pr-sccache-restore

- name: Configure
if: ${{ steps.timeout-artifact.outputs.exists != 'true' }}
shell: bash
run: |
cmake -B build -GNinja \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="${{ needs.compute-projects.outputs.projects }}" \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_LIT_ARGS="-v --no-progress-bar" \
-DCMAKE_C_COMPILER_LAUNCHER=sccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache \
-S llvm
- name: Write Timeout File
uses: ./.github/workflows/write-timeout-file
with:
job-name: "Build"

- name: Build
shell: bash
id: build
timeout-minutes: 330
run: |
touch timeout
ninja -C build -k 0 ${{ needs.compute-projects.outputs.check-targets }} && pass=1
rm timeout
[ $pass ] || false
- name: Upload Timeout Message
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
id: timeout
if: always()
with:
name: timeout
path: timeout
retention-days: 2

- name: Save sccache for next PR run
if: always()
uses: ./.github/workflows/pr-sccache-save

- name: Package Build Directory
shell: bash
if: always() && steps.timeout.outputs.artifact-id != ''
run: |
# Remove the timeout file, so the next build isn't categorized
# as a timeout.
rm -f timeout
# Dereference symlinks so that this works on Windows.
tar -h -c . | zstd -T0 -c > ../llvm-project.tar.zst
mv ../llvm-project.tar.zst .
- name: Upload Build
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
if: always() && steps.timeout.outputs.artifact-id != ''
with:
name: timeout-build
path: llvm-project.tar.zst
retention-days: 2
33 changes: 33 additions & 0 deletions .github/workflows/timeout-restore/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Timeout Restore
description: Save build state from a timed out job.
inputs:
artifact-name-suffix:
desciption: Suffix to add to the name of the artifact containing the build state.
required: true

outputs:
exists:
description: "This is true if a previous timeout build was restored, false otherwise."
value: ${{ steps.timeout-artifact.exists }}

runs:
using: "composite"
steps:
- name: Download Artifact
uses: actions/download-artifact@6b208ae046db98c579e8a3aa621ab581ff575935 # v4.1.1
with:
pattern: timeout-build-${{ inputs.artifact-name-suffix }}
merge-multiple: true

- name: Unpack Artifact
id: timeout-artifact
shell: bash
run: |
if [ -e llvm-project.tar.zst ]; then
tar --zstd -xf llvm-project.tar.zst
rm llvm-project.tar.zst
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "exists=false" >> $GITHUB_OUTPUT
fi
94 changes: 94 additions & 0 deletions .github/workflows/timeout-save/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: Timeout Save
description: Save build state when a timeout occurs so that it can be reused when the job is restarted.
inputs:
job-name:
description: The name of the job. This is used to look up the job id.
required: true
artifact-name-suffix:
desciption: Suffix to add to the name of the artifact containing the build state.
required: true
timeout-step:
description: The step that we want to restart if there is a timeout.
required: true
timeout-minutes:
description: The value of timeout-minutes for this step.
required: true

runs:
using: "composite"
steps:
- name: Check for timeout
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #v7.0.1
id: timeout
with:
script: |
const job_data = await github.rest.actions.listJobsForWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
});
for (job of job_data.data.jobs) {
console.log(job)
if (job.name != "${{ inputs.job-name }}") {
continue;
}
for (step of job.steps) {
console.log(step);
if (step.name != "${{ inputs.timeout-step }}") {
continue;
}
if (step.conclusion != "failure") {
return false;
}
// This run failed, check for timeout.
console.log(step.completed_at)
console.log(step.started_at)
completed_at = Date.parse(step.completed_at);
started_at = Date.parse(step.started_at);
console.log(completed_at)
console.log(started_at)
const diff = (completed_at - started_at) / 1000;
console.log(diff);
if (diff > 60 * ${{ inputs.timeout-minutes }})
return true;
}
}
return false;

- name: Save Working Directory
shell: bash
if: steps.timeout.outputs.result == 'true'
run: |
# Dereference symlinks so that this works on Windows.
tar -h -c . | zstd -T0 -c > ../llvm-project.tar.zst
mv ../llvm-project.tar.zst .
- name: Upload Working Directory
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
if: steps.timeout.outputs.result == 'true'
with:
name: timeout-build-${{ inputs.artifact-name-suffix }}
path: llvm-project.tar.zst
retention-days: 2

- name: Get Job ID
id: job-id
if: steps.timeout.outputs.result == 'true'
uses: ./.github/workflows/get-job-id
with:
job-name: ${{ inputs.job-name }}

- name: Create Timeout File
shell: bash
run: touch timeout-${{ steps.job-id.outputs.job-id }}

- name: Upload Timeout File
if: steps.timeout.outputs.result == 'true'
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
with:
name: timeout-${{ steps.job-id.outputs.job-id }}
path: timeout-${{ steps.job-id.outputs.job-id }}
77 changes: 77 additions & 0 deletions .github/workflows/unprivileged-download-artifact/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: Unprivileged Download Artifact
description: Download artifacts from another workflow run without using an access token.
inputs:
run-id:
description: The run-id for the workflow run that you want to download the artifact from. If ommited it will download the most recently created artifact from the repo with the artifact-name.
required: false
artifact-name:
desciption: The name of the artifact to download.
required: true


outputs:
filename:
description: "The filename of the downloaded artifact or the empty string if the artifact was not found."
value: ${{ steps.download-artifact.outputs.filename }}
artifact-id:
description: "The id of the artifact being downloaded."
value: ${{ steps.artifact-url.outputs.id }}


runs:
using: "composite"
steps:
- uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #v7.0.1
id: artifact-url
with:
script: |
var response;
if (!"${{ inputs.run-id }}") {
response = await github.rest.actions.listArtifactsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
name: "${{ inputs.artifact-name }}"
})
} else {
const response = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: "${{ inputs.run-id }}",
name: "${{ inputs.artifact-name }}"
})
}
console.log(response)
for (artifact of response.data.artifacts) {
console.log(artifact);
}
if (response.data.artifacts.length == 0) {
console.log("Could not find artifact ${{ inputs.artifact-name }} for workflow run ${{ inputs.run-id }}")
return;
}
const url_response = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: response.data.artifacts[0].id,
archive_format: "zip"
})
core.setOutput("url", url_response.url);
core.setOutput("id", response.data.artifacts[0].id);
- shell: bash
run: |
echo "${{ steps.artifact-url.outputs.url }}"
echo "${{ steps.artifact-url.outputs.result }}"
echo "${{ steps.artifact-url.outputs.result.url }}"
echo "${{ steps.artifact-url.outputs.result.id }}"
- shell: bash
if: steps.artifact-url.outputs.url != ''
id: download-artifact
run: |
curl -L -o ${{ inputs.artifact-name }}.zip "${{ steps.artifact-url.outputs.url }}"
echo "filename=${{ inputs.artifact-name }}.zip" >> $GITHUB_OUTPUT
20 changes: 20 additions & 0 deletions .github/workflows/write-timeout-file/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Write Timeout File
inputs:
job-name:
required: true
type: 'string'
filename:
required: false
default: timeout

runs:
using: "composite"
steps:
- uses: ./.github/workflows/get-job-id
id: job-id
with:
job-name: ${{ inputs.job-name }}

- shell: bash
run: |
echo '{"job_id": "${{ steps.job-id.outputs.job-id }}"}' > timeout-${{ steps.job-id
Empty file added llvm/a
Empty file.