Skip to content

Commit d241aa4

Browse files
andersy005andersy005
authored
Migrate CI from azure pipelines to GitHub Actions (pydata#4730)
* Run isort, black, mypy, flake8 checks via pre-commit * Fix line ending * Fix end of line * Fix mypy pre-commit hook. Thanks @keewis * Add main CI * Add additional CI * Fetch all history for all branches and tags * Add windows environment * Import xarray * Add doctests workflow * Add minimum version policy workflow * Simplify if logic * Add flaky and backend-api-v2 settings * Fix if elif else statements * Fix typo * Remove azure pipelines configurations * Fix environment file name * Upload code coverage for additional CI * Cache conda pkgs_dir * Fix cache key * Remove unnecessary cache number variable * Use `runner.os` instead of `matrix.os` * Use RUNNER_OS env variable * Disable name for the time being * Another attempt at setting name * `runner.os` doesn't work. Use `matrix.os` instead * Update env creation guidelines * Add `pre-commit run --all-files` check * Update blackdoc version * Add new pre-commit hooks * Add some of the out-of-the box hooks * Formatting only * Remove bad change * Remove isort and add pre-commit * Fix bad merge * Enable `cfgrib` on windows for the time being * Disable cfgrib on windows * Remove coveralls * Formatting only * Remove remaining reference to azure pipelines * Remove py 3.6 from CI matrix * Use py37 * Remove all references to py36 env file * Add check for skip ci * rename job to `detect ci trigger` * [skip ci] Empty commit * [skip-ci] Test skip CI trigger * Update PR template * Fix typ * GH markdown doesn't like lists in <sub></sub> * Remove the `-OO` flag for consistency
1 parent db6f4be commit d241aa4

36 files changed

+369
-382
lines changed

‎.github/ISSUE_TEMPLATE/config.yml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,5 @@ contact_links:
44
url: https://github.com/pydata/xarray/discussions
55
about: |
66
Ask questions and discuss with other community members here.
7-
If you have a question like "How do I concatenate a list of datasets?" then
7+
If you have a question like "How do I concatenate a list of datasets?" then
88
please include a self-contained reproducible example if possible.
9-
10-

‎.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,14 @@
22

33
- [ ] Closes #xxxx
44
- [ ] Tests added
5-
- [ ] Passes `isort . && black . && mypy . && flake8`
5+
- [ ] Passes `pre-commit run --all-files`
66
- [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst`
77
- [ ] New functions/methods are listed in `api.rst`
88

99

10-
<sub>By default, the upstream dev CI is disabled on pull request and push events. You can override this behavior per commit by adding a `[test-upstream]` tag to the first line of the commit message.</sub>
10+
<sub>
11+
<h3>
12+
Overriding CI behaviors
13+
</h3>
14+
By default, the upstream dev CI is disabled on pull request and push events. You can override this behavior per commit by adding a `[test-upstream]` tag to the first line of the commit message. For documentation-only commits, you can skip the CI per commit by adding a `[skip-ci]` tag to the first line of the commit message
15+
</sub>

‎.github/stale.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,4 +56,4 @@ limitPerRun: 1 # start with a small number
5656

5757
# issues:
5858
# exemptLabels:
59-
# - confirmed
59+
# - confirmed

‎.github/workflows/ci-additional.yaml

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
name: CI Additional
2+
on:
3+
push:
4+
branches:
5+
- "*"
6+
pull_request:
7+
branches:
8+
- "*"
9+
workflow_dispatch: # allows you to trigger manually
10+
11+
jobs:
12+
detect-ci-trigger:
13+
name: detect ci trigger
14+
runs-on: ubuntu-latest
15+
if: github.event_name == 'push' || github.event_name == 'pull_request'
16+
outputs:
17+
triggered: ${{ steps.detect-trigger.outputs.trigger-found }}
18+
steps:
19+
- uses: actions/checkout@v2
20+
- uses: ./.github/actions/detect-ci-trigger
21+
id: detect-trigger
22+
with:
23+
keyword: "[skip-ci]"
24+
25+
test:
26+
name: ${{ matrix.os }} ${{ matrix.env }}
27+
runs-on: ${{ matrix.os }}
28+
needs: detect-ci-trigger
29+
if: needs.detect-ci-trigger.outputs.triggered == 'false'
30+
defaults:
31+
run:
32+
shell: bash -l {0}
33+
strategy:
34+
fail-fast: false
35+
matrix:
36+
os: ["ubuntu-latest"]
37+
env:
38+
[
39+
"py37-bare-minimum",
40+
"py37-min-all-deps",
41+
"py37-min-nep18",
42+
"py38-all-but-dask",
43+
"py38-backend-api-v2",
44+
"py38-flaky",
45+
]
46+
steps:
47+
- name: Cancel previous runs
48+
uses: styfle/[email protected]
49+
with:
50+
access_token: ${{ github.token }}
51+
- uses: actions/checkout@v2
52+
with:
53+
fetch-depth: 0 # Fetch all history for all branches and tags.
54+
55+
- name: Set environment variables
56+
run: |
57+
if [[ ${{ matrix.env }} == "py38-backend-api-v2" ]] ;
58+
then
59+
echo "CONDA_ENV_FILE=ci/requirements/environment.yml" >> $GITHUB_ENV
60+
echo "XARRAY_BACKEND_API=v2" >> $GITHUB_ENV
61+
62+
elif [[ ${{ matrix.env }} == "py38-flaky" ]] ;
63+
then
64+
echo "CONDA_ENV_FILE=ci/requirements/environment.yml" >> $GITHUB_ENV
65+
echo "PYTEST_EXTRA_FLAGS=--run-flaky --run-network-tests" >> $GITHUB_ENV
66+
67+
else
68+
echo "CONDA_ENV_FILE=ci/requirements/${{ matrix.env }}.yml" >> $GITHUB_ENV
69+
fi
70+
- name: Cache conda
71+
uses: actions/cache@v2
72+
with:
73+
path: ~/conda_pkgs_dir
74+
key:
75+
${{ runner.os }}-conda-${{ matrix.env }}-${{
76+
hashFiles('ci/requirements/**.yml') }}
77+
78+
- uses: conda-incubator/setup-miniconda@v2
79+
with:
80+
channels: conda-forge
81+
channel-priority: strict
82+
mamba-version: "*"
83+
activate-environment: xarray-tests
84+
auto-update-conda: false
85+
python-version: 3.8
86+
use-only-tar-bz2: true
87+
88+
- name: Install conda dependencies
89+
run: |
90+
mamba env update -f $CONDA_ENV_FILE
91+
92+
- name: Install xarray
93+
run: |
94+
python -m pip install --no-deps -e .
95+
96+
- name: Version info
97+
run: |
98+
conda info -a
99+
conda list
100+
python xarray/util/print_versions.py
101+
- name: Import xarray
102+
run: |
103+
python -c "import xarray"
104+
- name: Run tests
105+
run: |
106+
python -m pytest -n 4 \
107+
--cov=xarray \
108+
--cov-report=xml \
109+
$PYTEST_EXTRA_FLAGS
110+
111+
- name: Upload code coverage to Codecov
112+
uses: codecov/codecov-action@v1
113+
with:
114+
file: ./coverage.xml
115+
flags: unittests,${{ matrix.env }}
116+
env_vars: RUNNER_OS
117+
name: codecov-umbrella
118+
fail_ci_if_error: false
119+
doctest:
120+
name: Doctests
121+
runs-on: "ubuntu-latest"
122+
needs: detect-ci-trigger
123+
if: needs.detect-ci-trigger.outputs.triggered == 'false'
124+
defaults:
125+
run:
126+
shell: bash -l {0}
127+
128+
steps:
129+
- name: Cancel previous runs
130+
uses: styfle/[email protected]
131+
with:
132+
access_token: ${{ github.token }}
133+
- uses: actions/checkout@v2
134+
with:
135+
fetch-depth: 0 # Fetch all history for all branches and tags.
136+
- uses: conda-incubator/setup-miniconda@v2
137+
with:
138+
channels: conda-forge
139+
channel-priority: strict
140+
mamba-version: "*"
141+
activate-environment: xarray-tests
142+
auto-update-conda: false
143+
python-version: "3.8"
144+
145+
- name: Install conda dependencies
146+
run: |
147+
mamba env update -f ci/requirements/environment.yml
148+
- name: Install xarray
149+
run: |
150+
python -m pip install --no-deps -e .
151+
- name: Version info
152+
run: |
153+
conda info -a
154+
conda list
155+
python xarray/util/print_versions.py
156+
- name: Run doctests
157+
run: |
158+
python -m pytest --doctest-modules xarray --ignore xarray/tests
159+
160+
min-version-policy:
161+
name: Minimum Version Policy
162+
runs-on: "ubuntu-latest"
163+
needs: detect-ci-trigger
164+
if: needs.detect-ci-trigger.outputs.triggered == 'false'
165+
defaults:
166+
run:
167+
shell: bash -l {0}
168+
169+
steps:
170+
- name: Cancel previous runs
171+
uses: styfle/[email protected]
172+
with:
173+
access_token: ${{ github.token }}
174+
- uses: actions/checkout@v2
175+
with:
176+
fetch-depth: 0 # Fetch all history for all branches and tags.
177+
- uses: conda-incubator/setup-miniconda@v2
178+
with:
179+
channels: conda-forge
180+
channel-priority: strict
181+
mamba-version: "*"
182+
auto-update-conda: false
183+
184+
- name: minimum versions policy
185+
run: |
186+
mamba install -y pyyaml
187+
python ci/min_deps_check.py ci/requirements/py37-bare-minimum.yml
188+
python ci/min_deps_check.py ci/requirements/py37-min-all-deps.yml

‎.github/workflows/ci-pre-commit.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
name: linting
2+
3+
on:
4+
push:
5+
branches: "*"
6+
pull_request:
7+
branches: "*"
8+
9+
jobs:
10+
linting:
11+
name: "pre-commit hooks"
12+
runs-on: ubuntu-latest
13+
steps:
14+
- uses: actions/checkout@v2
15+
- uses: actions/setup-python@v2
16+
- uses: pre-commit/[email protected]

‎.github/workflows/ci.yaml

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
name: CI
2+
on:
3+
push:
4+
branches:
5+
- "*"
6+
pull_request:
7+
branches:
8+
- "*"
9+
workflow_dispatch: # allows you to trigger manually
10+
11+
jobs:
12+
detect-ci-trigger:
13+
name: detect ci trigger
14+
runs-on: ubuntu-latest
15+
if: github.event_name == 'push' || github.event_name == 'pull_request'
16+
outputs:
17+
triggered: ${{ steps.detect-trigger.outputs.trigger-found }}
18+
steps:
19+
- uses: actions/checkout@v2
20+
- uses: ./.github/actions/detect-ci-trigger
21+
id: detect-trigger
22+
with:
23+
keyword: "[skip-ci]"
24+
test:
25+
name: ${{ matrix.os }} py${{ matrix.python-version }}
26+
runs-on: ${{ matrix.os }}
27+
needs: detect-ci-trigger
28+
if: needs.detect-ci-trigger.outputs.triggered == 'false'
29+
defaults:
30+
run:
31+
shell: bash -l {0}
32+
strategy:
33+
fail-fast: false
34+
matrix:
35+
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
36+
python-version: ["3.7", "3.8"]
37+
steps:
38+
- name: Cancel previous runs
39+
uses: styfle/[email protected]
40+
with:
41+
access_token: ${{ github.token }}
42+
- uses: actions/checkout@v2
43+
with:
44+
fetch-depth: 0 # Fetch all history for all branches and tags.
45+
- name: Set environment variables
46+
run: |
47+
if [[ ${{ matrix.os }} == windows* ]] ;
48+
then
49+
echo "CONDA_ENV_FILE=ci/requirements/environment-windows.yml" >> $GITHUB_ENV
50+
else
51+
echo "CONDA_ENV_FILE=ci/requirements/environment.yml" >> $GITHUB_ENV
52+
53+
fi
54+
echo "PYTHON_VERSION=${{ matrix.python-version }}" >> $GITHUB_ENV
55+
56+
- name: Cache conda
57+
uses: actions/cache@v2
58+
with:
59+
path: ~/conda_pkgs_dir
60+
key:
61+
${{ runner.os }}-conda-py${{ matrix.python-version }}-${{
62+
hashFiles('ci/requirements/**.yml') }}
63+
- uses: conda-incubator/setup-miniconda@v2
64+
with:
65+
channels: conda-forge
66+
channel-priority: strict
67+
mamba-version: "*"
68+
activate-environment: xarray-tests
69+
auto-update-conda: false
70+
python-version: ${{ matrix.python-version }}
71+
use-only-tar-bz2: true
72+
73+
- name: Install conda dependencies
74+
run: |
75+
mamba env update -f $CONDA_ENV_FILE
76+
77+
- name: Install xarray
78+
run: |
79+
python -m pip install --no-deps -e .
80+
81+
- name: Version info
82+
run: |
83+
conda info -a
84+
conda list
85+
python xarray/util/print_versions.py
86+
- name: Import xarray
87+
run: |
88+
python -c "import xarray"
89+
- name: Run tests
90+
run: |
91+
python -m pytest -n 4 \
92+
--cov=xarray \
93+
--cov-report=xml
94+
95+
- name: Upload code coverage to Codecov
96+
uses: codecov/codecov-action@v1
97+
with:
98+
file: ./coverage.xml
99+
flags: unittests
100+
env_vars: RUNNER_OS,PYTHON_VERSION
101+
name: codecov-umbrella
102+
fail_ci_if_error: false

‎.github/workflows/upstream-dev-ci.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: CI
1+
name: CI Upstream
22
on:
33
push:
44
branches:
@@ -23,7 +23,7 @@ jobs:
2323
id: detect-trigger
2424
with:
2525
keyword: "[test-upstream]"
26-
26+
2727
upstream-dev:
2828
name: upstream-dev
2929
runs-on: ubuntu-latest
@@ -60,7 +60,7 @@ jobs:
6060
python-version: ${{ matrix.python-version }}
6161
- name: Set up conda environment
6262
run: |
63-
mamba env update -f ci/requirements/py38.yml
63+
mamba env update -f ci/requirements/environment.yml
6464
bash ci/install-upstream-wheels.sh
6565
conda list
6666
- name: import xarray
@@ -148,8 +148,8 @@ jobs:
148148
}
149149
const result = await github.graphql(query, variables)
150150
151-
// If no issue is open, create a new issue,
152-
// else update the body of the existing issue.
151+
// If no issue is open, create a new issue,
152+
// else update the body of the existing issue.
153153
if (result.repository.issues.edges.length === 0) {
154154
github.issues.create({
155155
owner: variables.owner,

‎.pre-commit-config.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# https://pre-commit.com/
22
repos:
3+
- repo: https://github.com/pre-commit/pre-commit-hooks
4+
rev: v3.4.0
5+
hooks:
6+
- id: trailing-whitespace
7+
- id: end-of-file-fixer
8+
- id: check-yaml
39
# isort should run before black as black sometimes tweaks the isort output
410
- repo: https://github.com/PyCQA/isort
511
rev: 5.7.0
@@ -22,6 +28,7 @@ repos:
2228
rev: v0.790 # Must match ci/requirements/*.yml
2329
hooks:
2430
- id: mypy
31+
exclude: "properties|asv_bench"
2532
# run this occasionally, ref discussion https://github.com/pydata/xarray/pull/3194
2633
# - repo: https://github.com/asottile/pyupgrade
2734
# rev: v1.22.1

0 commit comments

Comments
 (0)