-
Notifications
You must be signed in to change notification settings - Fork 277
[CI][Refactor] Merge test CI workflow files into one #973
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 72 commits
Commits
Show all changes
74 commits
Select commit
Hold shift + click to select a range
25ff076
refactor: merge test CI workflow files into one
XuehaiPan 08251db
chore: set `UV_INDEX_STRATEGY=unsafe-best-match`
XuehaiPan b063691
feat: add AST test with Python 3.8
XuehaiPan ee48ce5
feat: implement manual caching mechanism for self-hosted runners
XuehaiPan c95691f
refactor: simplify cache logic for self-hosted runners
XuehaiPan aa678e6
chore: clear uv cache on failure
XuehaiPan b5596a1
chore: print format.sh output to logs
XuehaiPan 1ec3d45
chore: improve uv caching
XuehaiPan d3692be
chore: disable parallel test
XuehaiPan f7934a3
chore: use `PYTHONDEVMODE=1` in CI
XuehaiPan c52da65
feat: enable coredump generation
XuehaiPan 43df320
fix: fix perfbench condition
XuehaiPan 3cbf54a
Revert "feat: enable coredump generation"
XuehaiPan e2291b1
Merge branch 'main' into merge-ci-files
XuehaiPan 9d8e650
chore: move example CI down
XuehaiPan da19576
Revert "chore: move example CI down"
XuehaiPan 67aa4c2
chore: skip example `test_example_mha_sink_bwd_bhsd`
XuehaiPan a5f4f82
chore: skip example `test_example_gqa_sink_bwd_bhsd`
XuehaiPan 7553943
fix: fix example argument passing
XuehaiPan 3dcb3a3
fix: loosen test criteria
XuehaiPan c347bcd
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan 55324c5
chore: rename `CMAKE_CONFIGURE_OPTIONS` -> `CLANG_TIDY_CMAKE_OPTIONS`…
XuehaiPan 3e67c0f
feat: enable parallel testings
XuehaiPan f3cc9a8
chore: update pytest options
XuehaiPan 7985250
Merge branch 'main' of https://github.com/tile-ai/tilelang into merge…
LeiWang1999 dd68ff1
remove skipped test as now been resolved
LeiWang1999 cffc832
chore: empty commit to re-trigger ci
XuehaiPan c163e4e
Merge branch 'main' of https://github.com/tile-ai/tilelang into merge…
LeiWang1999 3ccdd15
Merge branch 'merge-ci-files' of https://github.com/XuehaiPan/tilelan…
LeiWang1999 6fc3855
test for n 1
LeiWang1999 372da61
chore: remove ` --numprocesses=1` option in example
XuehaiPan 911487e
chore: disable failfast
XuehaiPan 19bab29
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan 958c835
chore: update cibw selection
XuehaiPan 3ca2f9e
fix: fix git submodule clone
XuehaiPan 0627c36
chore: update cibw commands
XuehaiPan 83bf0e9
fix: fix yapf multiprocessing
XuehaiPan 3459b72
chore: setup ccache for CIBW on macOS only
XuehaiPan f68f649
chore: update comments
XuehaiPan f5e5a46
chore: update artifact listing
XuehaiPan 1b627f9
fix: do not fail if not found nvcc in PATH
XuehaiPan 0873045
fix: fix flash-attn installation
XuehaiPan 7c18c41
chore: update dist workflow trigger
XuehaiPan ea4c2e1
chore: remove outdated comments
XuehaiPan 6d339eb
chore(workflows/dist): simplify build matrix strategy
XuehaiPan f00b6f1
fix: fix CUDA path finding
XuehaiPan 965868a
fix: fix CUDA path finding
XuehaiPan 2599db9
chore: imcrease CI timeout
XuehaiPan e215d35
ci: disable failfast
XuehaiPan d930f69
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan 36d8872
fix: hide path prefix
XuehaiPan d790458
chore: more verbose
XuehaiPan 5e4a56f
chore: disable PR trigger for dist workflow
XuehaiPan 5d703d8
fix: seed for tests
XuehaiPan 908ceb6
fix: use nightly torch for ROCm tests
XuehaiPan 944af90
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan 5523011
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan f815d9a
chore: enable PR trigger for dist workflow
XuehaiPan 33e1718
chore: stop uploading debug wheels as artifacts in PR
XuehaiPan e1949b5
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan 3f250f2
chore: do not run workflows in forks
XuehaiPan a38baf9
chore: housekeep requirements
XuehaiPan 2dfe6c8
chore: use Nightly-ROCm-6.3 for CI
XuehaiPan 3bef042
chore: use Nightly-ROCm-6.4 for CI
XuehaiPan 6caf72b
Update ROCm toolkit version to 7.0
Alex4210987 82d255c
chore: restore previous rocm-ci.yml for test
XuehaiPan ab98648
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan 7e90df6
fix: cleanup PYTHONPATH
XuehaiPan 5f6bd12
chore: remove previous rocm-ci.yml
XuehaiPan 4f3fc20
ci fix
LeiWang1999 f43a075
Merge branch 'merge-ci-files' of https://github.com/XuehaiPan/tilelan…
LeiWang1999 01b7551
chore: remove previous rocm-ci.yml
XuehaiPan e08ff2d
chore: enable parallel example run
XuehaiPan 01eaba2
Merge remote-tracking branch 'upstream/main' into merge-ci-files
XuehaiPan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,60 +2,125 @@ name: Dist | |
on: | ||
schedule: | ||
# gemini said this is 6:00 china time | ||
- cron: '0 22 * * *' | ||
- cron: "0 22 * * *" | ||
pull_request: | ||
types: | ||
- opened | ||
- synchronize | ||
- reopened | ||
- ready_for_review | ||
paths: | ||
- setup.py | ||
- setup.cfg | ||
- pyproject.toml | ||
- MANIFEST.in | ||
- CMakeLists.txt | ||
- version_provider.py | ||
- .github/workflows/dist.yml | ||
release: | ||
types: [ published ] | ||
types: | ||
- published | ||
|
||
env: | ||
PYTHON_VERSION: '3.12' | ||
permissions: | ||
contents: read | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.ref }} | ||
group: "${{ github.workflow }}-${{ github.ref }}" | ||
cancel-in-progress: true | ||
|
||
jobs: | ||
build-wheels: | ||
name: Build wheels for Python ${{ matrix.python-version }} on ${{ matrix.target.runner }} with ${{ matrix.target.toolkit }} | ||
if: | | ||
github.repository_owner == 'tile-ai' && | ||
(github.event_name != 'pull_request' || !github.event.pull_request.draft) | ||
strategy: | ||
matrix: | ||
os: [ubuntu-22.04, ubuntu-22.04-arm, macos-16] | ||
include: | ||
- os: ubuntu-22.04 | ||
cuda_version: "12.1" | ||
- os: ubuntu-22.04-arm | ||
cuda_version: "12.8" | ||
fail-fast: true | ||
runs-on: ${{ matrix.os }} | ||
target: | ||
- { runner: ubuntu-latest, toolkit: "CUDA-12.1" } | ||
- { runner: ubuntu-24.04-arm, toolkit: "CUDA-12.8" } | ||
- { runner: macos-latest, toolkit: "Metal" } | ||
python-version: | ||
- "3.8" | ||
# TVM is built with Python 3.8 Limited API, it should work with all Python >= 3.8. | ||
# - "3.9" | ||
# - "3.10" | ||
# - "3.11" | ||
# - "3.12" | ||
# - "3.13" | ||
# - "3.14" | ||
fail-fast: false | ||
timeout-minutes: 120 | ||
runs-on: ${{ matrix.target.runner }} | ||
env: | ||
CUDA_VERSION: ${{ matrix.cuda_version }} | ||
NO_VERSION_LABEL: ${{ github.event_name != 'release' }} | ||
NO_VERSION_LABEL: ${{ github.event_name == 'release' && 'OFF' || 'ON' }} | ||
XuehaiPan marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 1 | ||
submodules: recursive | ||
|
||
- name: ccache | ||
uses: hendrikmuhs/[email protected] | ||
if: startsWith(matrix.os, 'macos') | ||
with: | ||
create-symlink: true | ||
key: ${{ github.job }}-${{ matrix.os }} | ||
|
||
- name: Build wheels | ||
uses: pypa/[email protected] | ||
with: | ||
output-dir: wheelhouse | ||
config-file: "{package}/pyproject.toml" | ||
|
||
# just for now to list all files | ||
- name: List wheels | ||
id: ls-whl | ||
run: echo "whl_name=$(ls wheelhouse | head -n1)" >> $GITHUB_OUTPUT | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: ${{ steps.ls-whl.outputs.whl_name }}.zip | ||
path: wheelhouse/${{ steps.ls-whl.outputs.whl_name }} | ||
compression-level: 0 | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 1 | ||
submodules: recursive | ||
|
||
# NB: CIBW builds wheels in containers on Linux | ||
- name: Setup ccache (macOS only) | ||
if: runner.os == 'macOS' | ||
uses: hendrikmuhs/ccache-action@v1 | ||
with: | ||
XuehaiPan marked this conversation as resolved.
Show resolved
Hide resolved
|
||
create-symlink: true | ||
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.python-version }}-${{ matrix.target.toolkit }} | ||
evict-old-files: "7d" | ||
|
||
XuehaiPan marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- name: Set CIBW_BUILD | ||
run: | | ||
PYTHON_VERSION="${{ matrix.python-version }}" | ||
PYTHON_VERSION_MAJMIN="$(echo "${PYTHON_VERSION}" | cut -d '.' -f-2)" | ||
PYTHON_VERSION_MAJMIN_NODOT="${PYTHON_VERSION_MAJMIN//./}" | ||
echo "CIBW_BUILD=cp${PYTHON_VERSION_MAJMIN_NODOT}-*" | tee -a "${GITHUB_ENV}" | ||
|
||
if [[ "${{ matrix.target.toolkit }}" == *"CUDA"* ]]; then | ||
CUDA_VERSION="${{ matrix.target.toolkit }}" | ||
CUDA_VERSION="${CUDA_VERSION#CUDA-}" | ||
echo "CUDA_VERSION=${CUDA_VERSION}" | tee -a "${GITHUB_ENV}" | ||
fi | ||
|
||
- name: Build wheels | ||
uses: pypa/[email protected] | ||
with: | ||
package-dir: . | ||
output-dir: wheelhouse | ||
config-file: "{package}/pyproject.toml" | ||
|
||
- name: Upload wheels | ||
# Not PR to save artifact storage, as wheels are only needed for releases. | ||
if: github.event_name != 'pull_request' | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: wheels-${{ matrix.python-version }}-${{ runner.os }}-${{ runner.arch }}-${{ matrix.target.toolkit }} | ||
path: wheelhouse/*.whl | ||
if-no-files-found: error | ||
|
||
list-artifacts: | ||
name: List artifacts | ||
# Not PR to save artifact storage, as wheels are only needed for releases. | ||
if: github.event_name != 'pull_request' | ||
runs-on: ubuntu-latest | ||
needs: [build-wheels] | ||
timeout-minutes: 15 | ||
steps: | ||
- name: Download built wheels | ||
uses: actions/download-artifact@v5 | ||
with: | ||
pattern: wheels-* | ||
path: dist | ||
merge-multiple: true | ||
|
||
- name: List distributions | ||
run: ls -lh dist/* | ||
|
||
- name: Upload artifacts | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: artifacts | ||
path: dist/* | ||
if-no-files-found: error |
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GA have a 2000 minutes limit per month. I think building release on each PR (even just for changes on specific files) would be too often.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GitHub only charges for private repositories. See: https://docs.github.com/en/billing/concepts/product-billing/github-actions
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's free but not unlimited: https://docs.github.com/en/actions/reference/limits#storage-limits-for-all-github-hosted-runners
See: #799 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This means the runtime hard disk in the runner, such as temporary built objects. It is a per-run limit, but not a limit for how many runs we can have.
Additionally, the generated artifacts and logs will be rotated if the total storage reaches or exceeds the 90-day limit. See also: https://docs.github.com/en/organizations/managing-organization-settings/configuring-the-retention-period-for-github-actions-artifacts-and-logs-in-your-organization
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I disabled wheel artifact upload in PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. I remember hitting some runner limit before but I didn't find other docs about it. This LGTM.