Skip to content

Build image in github actions #6026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/build-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Build Image

on:
push:
branches: [ master ]
paths:
- 'build-image/**'
- '.github/workflows/build-image.yml'
pull_request:
branches: [ master ]
paths:
- 'build-image/**'
- '.github/workflows/build-image.yml'

jobs:
build:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
name: Checkout
with:
fetch-depth: 0

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Save image
run: make save-multiarch-build-image

- name: Upload Docker Images Artifacts
uses: actions/upload-artifact@v4
with:
name: build-image
path: |
./build-image-amd64.tar
./build-image-arm64.tar
if-no-files-found: error

push:
needs: build
if: (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')) && github.repository == 'cortexproject/cortex'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
name: Checkout
with:
fetch-depth: 0

- name: Download Docker Images Artifacts
uses: actions/download-artifact@v4
with:
name: build-image

- name: Load image
run: make load-multiarch-build-image

- name: Login to Quay.io
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{secrets.QUAY_REGISTRY_USER}}
password: ${{secrets.QUAY_REGISTRY_PASSWORD}}

- name: Push image
run: make push-multiarch-build-image
6 changes: 6 additions & 0 deletions .github/workflows/test-build-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,13 @@ on:
branches: [master]
tags:
- v[0-9]+.[0-9]+.[0-9]+** # Tag filters not as strict due to different regex system on Github Actions
paths-ignore:
- 'build-image/**'
- '.github/workflows/build-image.yml'
pull_request:
paths-ignore:
- 'build-image/**'
- '.github/workflows/build-image.yml'

jobs:
lint:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,6 @@ Makefile.local
.vscode
compose
compose-simple

/build-image-arm64.tar
/build-image-amd64.tar
9 changes: 1 addition & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,7 @@ fetch-build-image:
docker tag $(BUILD_IMAGE):$(LATEST_BUILD_IMAGE_TAG) $(BUILD_IMAGE):latest
touch build-image/.uptodate

push-multiarch-build-image:
@echo
# Build image for each platform separately... it tends to generate fewer errors.
$(SUDO) docker buildx build --platform linux/amd64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) build-image/
$(SUDO) docker buildx build --platform linux/arm64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) build-image/
# This command will run the same build as above, but it will reuse existing platform-specific images,
# put them together and push to registry.
$(SUDO) docker buildx build -o type=registry --platform linux/amd64,linux/arm64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) -t $(IMAGE_PREFIX)build-image:$(IMAGE_TAG) build-image/
-include build-image/Makefile

# We don't want find to scan inside a bunch of directories, to accelerate the
# 'make: Entering directory '/go/src/github.com/cortexproject/cortex' phase.
Expand Down
17 changes: 17 additions & 0 deletions build-image/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
save-multiarch-build-image:
@echo
# Build image for each platform separately... it tends to generate fewer errors.
$(SUDO) docker buildx build --platform linux/amd64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) -t $(IMAGE_PREFIX)build-image:$(IMAGE_TAG)-amd64 --output type=docker,dest=./build-image-amd64.tar build-image/
$(SUDO) docker buildx build --platform linux/arm64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) -t $(IMAGE_PREFIX)build-image:$(IMAGE_TAG)-arm64 --output type=docker,dest=./build-image-arm64.tar build-image/

load-multiarch-build-image:
$(SUDO) docker load -i build-image-amd64.tar
$(SUDO) docker load -i build-image-arm64.tar

push-multiarch-build-image:
# This command will run the same build as multiarch-build-image, but it will reuse existing platform-specific images,
# put them together and push to registry.
$(SUDO) docker push $(IMAGE_PREFIX)build-image:${IMAGE_TAG}-amd64
$(SUDO) docker push $(IMAGE_PREFIX)build-image:${IMAGE_TAG}-arm64
$(SUDO) docker manifest create $(IMAGE_PREFIX)build-image:$(IMAGE_TAG) --amend $(IMAGE_PREFIX)build-image:${IMAGE_TAG}-amd64 --amend $(IMAGE_PREFIX)build-image:${IMAGE_TAG}-arm64
$(SUDO) docker manifest push $(IMAGE_PREFIX)build-image:$(IMAGE_TAG)
8 changes: 3 additions & 5 deletions docs/contributing/how-to-update-the-build-image.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,10 @@ weight: 5
slug: how-to-update-the-build-image
---

The build image currently can only be updated by a Cortex maintainer. If you're not a maintainer you can still open a PR with the changes, asking a maintainer to assist you publishing the updated image. The procedure is:
The procedure is:

1. Update `build-image/Dockerfile`
1. Run `go env` and make sure `GOPROXY=https://proxy.golang.org,direct` (Go's default). Some environment may required `GOPROXY=direct`, and if you push a build image with this, build workflow on GitHub will take a lot longer to download modules.
1. `docker login quay.io`. Note that pushing to `quay.io/cortexproject/build-image` repository can only be done by a maintainer.
1. Build the and publish the image by using `make push-multiarch-build-image`. This will build and push multi-platform docker image (for linux/amd64 and linux/arm64). Running this step successfully requires [Docker Buildx](https://docs.docker.com/buildx/working-with-buildx/), but does not require a specific platform.
1. Replace the image tag in `.github/workflows/*` (_there may be multiple references_) and Makefile (variable `LATEST_BUILD_IMAGE_TAG`).
1. Create a PR to master with that changed, after the PR is merged to master, the new build image is available in the quay.io repository. Check github action logs [here](https://github.com/cortexproject/cortex/actions/workflows/build-image.yml) for to find the image tag.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extraneous "for" in second sentence.

1. Create another PR to replace the image tag in `.github/workflows/*` (_there may be multiple references_) and Makefile (variable `LATEST_BUILD_IMAGE_TAG`).
1. If you are updating Go's runtime version be sure to change `actions/setup-go`'s `go-version` in ``.github/workflows/*`.
1. Open a PR and make sure the CI with new build-image passes
Loading