-
-
Notifications
You must be signed in to change notification settings - Fork 4
TEST: Feature/multi arch building pipeline #556
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
d550c99
MA Operator pipeline
Maleware 4419655
Remove harbor for testing
Maleware 962ee49
Manuell trigger for testing
Maleware 5ad946b
Add reminder for after testing
Maleware c9c80e6
fix cosign image string
Maleware f004bff
Fix cosign string for another part
Maleware 0e657ca
Adding version to docker output parsing
Maleware 2d9f6e8
Fixing another path for syft
Maleware 6f5079d
Workaround for buildjet error
Maleware 6039775
Remove cargo clean as we have to rework this action
Maleware 4f10fe5
Using a different action to set up cargo install cargo-edit
Maleware a9375a5
Install helm for arm64 manually
Maleware 7589e47
Rework arch selection mechanism
Maleware 8144cd3
Rework arch selection mechanism 2
Maleware c0d7ced
Another try for arch selection
Maleware 01d60b1
Still not working...
Maleware 9ae2d83
Fixing makefile for manifest list build (maybe)
Maleware 9868377
Fixing makefile for manifest list build (maybe 2)
Maleware 6f7e7bc
Fix name in makefile
Maleware 28fbb48
Not using vars rather directly use env
Maleware 8e3281b
Cating makefile during action
Maleware 99049f1
Fixing manifestpublish
Maleware 878798c
Adding checkout
Maleware 31046d7
Reverting test changes
Maleware 870d6ff
Better vars
Maleware eddb14e
Edit vars
Maleware 52cbc24
Trigger pipeline
Maleware ffade8b
Using 205 to see vars
Maleware 35529c7
Different variables again..
Maleware a32a93c
Binding maybe
Maleware fcb1dbe
Remove ;
Maleware 235a748
Removing vars..
Maleware b645465
Remove man list for harbor for testing
Maleware cdb1981
Remove vars everywhere
Maleware f9cd0cf
Typo
Maleware 7c6b18b
Adding login to nexus
Maleware 2d18610
Using correct makefile syntax
Maleware da12d0f
Escaping single line more often
Maleware b9e882f
Adding Harbor to the pipeline
Maleware ba1bf4a
Updating version if PR
Maleware 62509b7
Updating arch tag for signing
Maleware 33f4cf5
Wrong placement of {ARCH}
Maleware 2720c99
Adding cargo-edit since needed
Maleware 04e668a
typo
Maleware d4c78cb
Using stackable action for cargo-set-version again
Maleware 07bf962
Merge branch 'main' into feature/multi-arch-building-pipeline
Maleware 19bed55
Using stackable action to install cargo bins
Maleware cd85e00
Merge branch 'feature/multi-arch-building-pipeline' of https://github…
Maleware a828b56
Updating dependencies
Maleware e0016fb
Merge branch 'main' into feature/multi-arch-building-pipeline
Maleware bd9eda0
Reworking preflight tests
Maleware b650588
Merge branch 'feature/multi-arch-building-pipeline' of https://github…
Maleware File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,6 +16,8 @@ on: | |
- '[0-9][0-9].[0-9]+.[0-9]+' | ||
pull_request: | ||
merge_group: | ||
# TODO: Remove after testing | ||
workflow_dispatch: | ||
|
||
env: | ||
CARGO_TERM_COLOR: always | ||
|
@@ -305,7 +307,10 @@ jobs: | |
needs: | ||
- tests_passed | ||
- select_helm_repo | ||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: | ||
runner: ["ubuntu-latest", "buildjet-2vcpu-ubuntu-2204-arm"] | ||
runs-on: ${{ matrix.runner }} | ||
permissions: | ||
id-token: write | ||
env: | ||
|
@@ -352,7 +357,16 @@ jobs: | |
- name: Install syft | ||
uses: anchore/sbom-action/download-syft@7ccf588e3cf3cc2611714c2eeae48550fbc17552 # v0.15.11 | ||
- name: Build Docker image and Helm chart | ||
run: make -e build | ||
run: | | ||
# Installing helm on BuildJet only | ||
if [ "$(arch)" = "aarch64" ]; then | ||
curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null | ||
sudo apt-get -y install apt-transport-https --yes | ||
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list | ||
sudo apt-get -y update | ||
sudo apt-get -y install helm | ||
fi | ||
make -e build | ||
- name: Publish Docker image and Helm chart | ||
if: ${{ !github.event.pull_request.head.repo.fork }} | ||
run: make -e publish | ||
|
@@ -362,6 +376,47 @@ jobs: | |
if: ${{ !github.event.pull_request.head.repo.fork }} | ||
run: echo "IMAGE_TAG=$(make -e print-docker-tag)" >> $GITHUB_OUTPUT | ||
|
||
create_manifest_list: | ||
name: Build and publish manifest list | ||
needs: | ||
- package_and_publish | ||
runs-on: ubuntu-latest | ||
permissions: | ||
id-token: write | ||
env: | ||
NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }} | ||
OCI_REGISTRY_SDP_PASSWORD: ${{ secrets.HARBOR_ROBOT_SDP_GITHUB_ACTION_BUILD_SECRET }} | ||
OCI_REGISTRY_SDP_USERNAME: "robot$sdp+github-action-build" | ||
OCI_REGISTRY_SDP_CHARTS_PASSWORD: ${{ secrets.HARBOR_ROBOT_SDP_CHARTS_GITHUB_ACTION_BUILD_SECRET }} | ||
OCI_REGISTRY_SDP_CHARTS_USERNAME: "robot$sdp-charts+github-action-build" | ||
steps: | ||
- name: Install cosign | ||
uses: sigstore/cosign-installer@59acb6260d9c0ba8f4a2f9d9b48431a222b68e20 # tag=v3.5.0 | ||
- name: Checkout | ||
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 | ||
with: | ||
submodules: recursive | ||
# - uses: actions-rs/[email protected] | ||
# with: | ||
# crate: cargo-edit | ||
# version: latest | ||
# This step checks if the current run was triggered by a push to a pr (or a pr being created). | ||
# If this is the case it changes the version of this project in all Cargo.toml files to include the suffix | ||
# "-pr<prnumber>" so that the published artifacts can be linked to this PR. | ||
- uses: stackabletech/cargo-install-action@main | ||
with: | ||
crate: cargo-edit | ||
bin: cargo-set-version | ||
- name: Update version if PR | ||
if: ${{ github.event_name == 'pull_request' }} | ||
run: cargo set-version --offline --workspace 0.0.0-pr${{ github.event.pull_request.number }} | ||
- name: Build manifest list | ||
run: | | ||
# Creating manifest list | ||
make -e docker-manifest-list-build | ||
# Pushing and signing manifest list | ||
make -e docker-manifest-list-publish | ||
|
||
openshift_preflight: | ||
name: Run the OpenShift Preflight check on the published images | ||
if: ${{ !github.event.pull_request.head.repo.fork }} | ||
|
@@ -380,4 +435,4 @@ jobs: | |
ARCH_FOR_PREFLIGHT="$(arch | sed -e 's#x86_64#amd64#' | sed -e 's#aarch64#arm64#')" | ||
./preflight-linux-amd64 check container "$IMAGE_TAG" --platform "${ARCH_FOR_PREFLIGHT}" > preflight.out | ||
- name: "Passed?" | ||
run: '[ "$(./preflight-linux-amd64 check container "$IMAGE_TAG" --platform "${ARCH_FOR_PREFLIGHT}" | jq -r .passed)" == true ]' | ||
run: '[ "$(cat preflight.out | jq -r .passed)" == true ]' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -12,8 +12,10 @@ | |
TAG := $(shell git rev-parse --short HEAD) | ||
OPERATOR_NAME := opa-operator | ||
VERSION := $(shell cargo metadata --format-version 1 | jq -r '.packages[] | select(.name=="stackable-${OPERATOR_NAME}") | .version') | ||
ARCH := $(shell arch | sed -e 's#x86_64#amd64#' | sed -e 's#aarch64#arm64#') | ||
|
||
DOCKER_REPO := docker.stackable.tech | ||
# TODO: Change to stackable after testing | ||
ORGANIZATION := stackable | ||
OCI_REGISTRY_HOSTNAME := oci.stackable.tech | ||
OCI_REGISTRY_PROJECT_IMAGES := sdp | ||
|
@@ -22,25 +24,24 @@ OCI_REGISTRY_PROJECT_CHARTS := sdp-charts | |
HELM_REPO := https://repo.stackable.tech/repository/helm-dev | ||
HELM_CHART_NAME := ${OPERATOR_NAME} | ||
HELM_CHART_ARTIFACT := target/helm/${OPERATOR_NAME}-${VERSION}.tgz | ||
|
||
SHELL=/usr/bin/env bash -euo pipefail | ||
|
||
render-readme: | ||
scripts/render_readme.sh | ||
|
||
## Docker related targets | ||
docker-build: | ||
docker build --force-rm --build-arg VERSION=${VERSION} -t "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}" -f docker/Dockerfile . | ||
docker tag "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}" "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}" | ||
docker build --force-rm --build-arg VERSION=${VERSION} -t "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-${ARCH}" -f docker/Dockerfile . | ||
docker tag "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-${ARCH}" "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}-${ARCH}" | ||
|
||
docker-publish: | ||
# Push to Nexus | ||
echo "${NEXUS_PASSWORD}" | docker login --username github --password-stdin "${DOCKER_REPO}" | ||
DOCKER_OUTPUT=$$(docker push --all-tags "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}");\ | ||
# Obtain the digest of the pushed image from the output of `docker push`, because signing by tag is deprecated and will be removed from cosign in the future\ | ||
REPO_DIGEST_OF_IMAGE=$$(echo "$$DOCKER_OUTPUT" | awk '/^${VERSION}: digest: sha256:[0-9a-f]{64} size: [0-9]+$$/ { print $$3 }');\ | ||
REPO_DIGEST_OF_IMAGE=$$(echo "$$DOCKER_OUTPUT" | awk '/^${VERSION}-${ARCH}: digest: sha256:[0-9a-f]{64} size: [0-9]+$$/ { print $$3 }');\ | ||
if [ -z "$$REPO_DIGEST_OF_IMAGE" ]; then\ | ||
echo 'Could not find repo digest for container image: ${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}';\ | ||
echo 'Could not find repo digest for container image: ${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-${ARCH}';\ | ||
exit 1;\ | ||
fi;\ | ||
# This generates a signature and publishes it to the registry, next to the image\ | ||
|
@@ -51,8 +52,8 @@ docker-publish: | |
# Determine the PURL for the container image\ | ||
PURL="pkg:docker/${ORGANIZATION}/${OPERATOR_NAME}@$$REPO_DIGEST_OF_IMAGE?repository_url=${DOCKER_REPO}";\ | ||
# Get metadata from the image\ | ||
IMAGE_DESCRIPTION=$$(docker inspect --format='{{.Config.Labels.description}}' "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}");\ | ||
IMAGE_NAME=$$(docker inspect --format='{{.Config.Labels.name}}' "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}");\ | ||
IMAGE_DESCRIPTION=$$(docker inspect --format='{{.Config.Labels.description}}' "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-${ARCH}");\ | ||
IMAGE_NAME=$$(docker inspect --format='{{.Config.Labels.name}}' "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-${ARCH}");\ | ||
# Merge the SBOM with the metadata for the operator\ | ||
jq -s '{"metadata":{"component":{"description":"'"$$IMAGE_NAME. $$IMAGE_DESCRIPTION"'","supplier":{"name":"Stackable GmbH","url":["https://stackable.tech/"]},"author":"Stackable GmbH","purl":"'"$$PURL"'","publisher":"Stackable GmbH"}}} * .[0]' sbom.json > sbom.merged.json;\ | ||
# Attest the SBOM to the image\ | ||
|
@@ -63,9 +64,9 @@ docker-publish: | |
docker login --username '${value OCI_REGISTRY_SDP_USERNAME}' --password '${OCI_REGISTRY_SDP_PASSWORD}' '${OCI_REGISTRY_HOSTNAME}' | ||
DOCKER_OUTPUT=$$(docker push --all-tags '${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}');\ | ||
# Obtain the digest of the pushed image from the output of `docker push`, because signing by tag is deprecated and will be removed from cosign in the future\ | ||
REPO_DIGEST_OF_IMAGE=$$(echo "$$DOCKER_OUTPUT" | awk '/^${VERSION}: digest: sha256:[0-9a-f]{64} size: [0-9]+$$/ { print $$3 }');\ | ||
REPO_DIGEST_OF_IMAGE=$$(echo "$$DOCKER_OUTPUT" | awk '/^${VERSION}-${ARCH}: digest: sha256:[0-9a-f]{64} size: [0-9]+$$/ { print $$3 }');\ | ||
if [ -z "$$REPO_DIGEST_OF_IMAGE" ]; then\ | ||
echo 'Could not find repo digest for container image: ${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}';\ | ||
echo 'Could not find repo digest for container image: ${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}-${ARCH}';\ | ||
exit 1;\ | ||
fi;\ | ||
# This generates a signature and publishes it to the registry, next to the image\ | ||
|
@@ -76,13 +77,39 @@ docker-publish: | |
# Determine the PURL for the container image\ | ||
PURL="pkg:docker/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}@$$REPO_DIGEST_OF_IMAGE?repository_url=${OCI_REGISTRY_HOSTNAME}";\ | ||
# Get metadata from the image\ | ||
IMAGE_DESCRIPTION=$$(docker inspect --format='{{.Config.Labels.description}}' "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}");\ | ||
IMAGE_NAME=$$(docker inspect --format='{{.Config.Labels.name}}' "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}");\ | ||
IMAGE_DESCRIPTION=$$(docker inspect --format='{{.Config.Labels.description}}' "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}-${ARCH}");\ | ||
IMAGE_NAME=$$(docker inspect --format='{{.Config.Labels.name}}' "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}-${ARCH}");\ | ||
# Merge the SBOM with the metadata for the operator\ | ||
jq -s '{"metadata":{"component":{"description":"'"$$IMAGE_NAME. $$IMAGE_DESCRIPTION"'","supplier":{"name":"Stackable GmbH","url":["https://stackable.tech/"]},"author":"Stackable GmbH","purl":"'"$$PURL"'","publisher":"Stackable GmbH"}}} * .[0]' sbom.json > sbom.merged.json;\ | ||
# Attest the SBOM to the image\ | ||
cosign attest -y --predicate sbom.merged.json --type cyclonedx "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}@$$REPO_DIGEST_OF_IMAGE" | ||
|
||
# This assumes "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-amd64 and "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-arm64 being build and pushed | ||
docker-manifest-list-build: | ||
docker manifest create "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}" --amend "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-amd64" --amend "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}-arm64" | ||
docker manifest create "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}" --amend "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}-amd64" --amend "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}-arm64" | ||
|
||
docker-manifest-list-publish: | ||
# Push to Nexus | ||
echo "${NEXUS_PASSWORD}" | docker login --username github --password-stdin "${DOCKER_REPO}" | ||
# `docker manifest push` directly returns the digest of the manifest list | ||
# As it is an experimental feature, this might change in the future | ||
# Further reading: https://docs.docker.com/reference/cli/docker/manifest/push/ | ||
DIGEST_NEXUS=$$(docker manifest push "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}");\ | ||
# Refer to image via its digest (oci.stackable.tech/sdp/airflow@sha256:0a1b2c...)\ | ||
# This generates a signature and publishes it to the registry, next to the image\ | ||
# Uses the keyless signing flow with Github Actions as identity provider\ | ||
cosign sign -y "${DOCKER_REPO}/${ORGANIZATION}/${OPERATOR_NAME}:${VERSION}@$$DIGEST_NEXUS" | ||
|
||
# Push to Harbor | ||
# We need to use "value" here to prevent the variable from being recursively expanded by make (username contains a dollar sign, since it's a Harbor bot) | ||
docker login --username '${value OCI_REGISTRY_SDP_USERNAME}' --password '${OCI_REGISTRY_SDP_PASSWORD}' '${OCI_REGISTRY_HOSTNAME}' | ||
DIGEST_HARBOR=$$(docker manifest push "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}");\ | ||
# Refer to image via its digest (oci.stackable.tech/sdp/airflow@sha256:0a1b2c...);\ | ||
# This generates a signature and publishes it to the registry, next to the image\ | ||
# Uses the keyless signing flow with Github Actions as identity provider\ | ||
cosign sign -y "${OCI_REGISTRY_HOSTNAME}/${OCI_REGISTRY_PROJECT_IMAGES}/${OPERATOR_NAME}:${VERSION}@$$DIGEST_HARBOR" | ||
|
||
# TODO remove if not used/needed | ||
docker: docker-build docker-publish | ||
|
||
|
@@ -144,7 +171,7 @@ clean: chart-clean | |
regenerate-charts: chart-clean compile-chart | ||
|
||
regenerate-nix: | ||
nix run -f . regenerateNixLockfiles | ||
nix run --extra-experimental-features nix-command --extra-experimental-features flakes -f . regenerateNixLockfiles | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FYI you can set it for your machine with
|
||
|
||
build: regenerate-charts regenerate-nix helm-package docker-build | ||
|
||
|
@@ -158,7 +185,7 @@ check-kubernetes: | |
|
||
run-dev: check-nix check-kubernetes | ||
kubectl apply -f deploy/stackable-operators-ns.yaml | ||
nix run -f. tilt -- up --port 5430 --namespace stackable-operators | ||
nix run --extra-experimental-features nix-command --extra-experimental-features flakes -f. tilt -- up --port 5430 --namespace stackable-operators | ||
|
||
stop-dev: check-nix check-kubernetes | ||
nix run -f. tilt -- down |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pls also add https://github.com/stackabletech/stackable-cockpit/blob/main/.github/actionlint.yaml