From c64d811d884a13e41543f66f922ca3c9532dba78 Mon Sep 17 00:00:00 2001 From: jaellio Date: Tue, 14 Feb 2023 15:17:36 -0800 Subject: [PATCH 1/9] Update Istio/SPIRE integration demo to use SPIRE Controller Manager instead of k8s workload registration. Signed-off-by: jaellio --- .spelling | 1 + .../en/docs/ops/integrations/spire/index.md | 162 +++++++++++++----- 2 files changed, 119 insertions(+), 44 deletions(-) diff --git a/.spelling b/.spelling index 0881aee04b3bf..50b2bed302ec7 100644 --- a/.spelling +++ b/.spelling @@ -209,6 +209,7 @@ CloudNativeCon CloudWatch cluster1 cluster2 +ClusterSPIFFEID CNCF-hosted CNI cnn.com diff --git a/content/en/docs/ops/integrations/spire/index.md b/content/en/docs/ops/integrations/spire/index.md index c8cd265e38a16..7f702d8bbfeff 100644 --- a/content/en/docs/ops/integrations/spire/index.md +++ b/content/en/docs/ops/integrations/spire/index.md @@ -36,7 +36,7 @@ $ kubectl apply -f @samples/security/spire/spire-quickstart.yaml@ {{< /text >}} This will deploy SPIRE into your cluster, along with two additional components: the [SPIFFE CSI Driver](https://github.com/spiffe/spiffe-csi) — used to share the SPIRE Agent's UNIX Domain Socket with the other -pods throughout the node — and the [SPIRE Kubernetes Workload Registrar](https://github.com/spiffe/spire/tree/main/support/k8s/k8s-workload-registrar), a facilitator that performs automatic workload registration +pods throughout the node — and the [SPIRE Controller Manager](https://github.com/spiffe/spire-controller-manager), a facilitator that performs workload registration and establishes federation relationships within Kubernetes. See [Install Istio](#install-istio) to configure Istio and integrate with the SPIFFE CSI Driver. ### Option 2: Configure a custom SPIRE installation @@ -100,6 +100,7 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b - name: workload-socket csi: driver: "csi.spiffe.io" + readOnly: true components: ingressGateways: - name: istio-ingressgateway @@ -117,6 +118,7 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b name: workload-socket csi: driver: "csi.spiffe.io" + readOnly: true - path: spec.template.spec.containers.[name:istio-proxy].volumeMounts.[name:workload-socket] value: name: workload-socket @@ -151,20 +153,16 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b This will also add an initContainer to the gateway that will wait for SPIRE to create the UNIX Domain Socket before starting the istio-proxy. If the SPIRE agent is not ready or has not been properly configured with the same socket path, the Ingress Gateway initContainer will wait forever. -1. Use [sidecar injection](/docs/setup/additional-setup/sidecar-injection) to inject the `istio-proxy` - container into the pods within your mesh. See [custom templates](/docs/setup/additional-setup/sidecar-injection/#custom-templates-experimental) - for information on how to apply the custom defined `spire` template into `istio-proxy`. This allows for the CSI driver to mount the UDS on the sidecars. +Check Ingress-gateway pod state: - Check Ingress-gateway pod state: - - {{< text bash >}} - $ kubectl get pods -n istio-system - NAME READY STATUS RESTARTS AGE - istio-ingressgateway-5b45864fd4-lgrxs 0/1 Running 0 17s - istiod-989f54d9c-sg7sn 1/1 Running 0 23s - {{< /text >}} +{{< text bash >}} +$ kubectl get pods -n istio-system +NAME READY STATUS RESTARTS AGE +istio-ingressgateway-5b45864fd4-lgrxs 0/1 Running 0 17s +istiod-989f54d9c-sg7sn 1/1 Running 0 23s +{{< /text >}} -Data plane containers will only reach `Ready` if a corresponding registration entry is created for them on the SPIRE Server. Then, +The Ingress-gateway pod and data plane containers will only reach `Ready` if a corresponding registration entry is created for them on the SPIRE Server. Then, Envoy will be able to fetch cryptographic identities from SPIRE. See [Register workloads](#register-workloads) to register entries for services in your mesh. @@ -172,15 +170,90 @@ See [Register workloads](#register-workloads) to register entries for services i This section describes the options available for registering workloads in a SPIRE Server. -### Option 1: Automatic registration using the SPIRE workload registrar +### Option 1: Registration using the SPIRE Controller Manager + +By deploying [SPIRE Controller Manager](https://github.com/spiffe/spire-controller-manager) +along with a SPIRE Server, new entries can be automatically registered for each new pod that matches the selector defined in a [ClusterSPIFFEID](https://github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md) custom resource. -By deploying [SPIRE Kubernetes Workload Registrar](https://github.com/spiffe/spire/tree/main/support/k8s/k8s-workload-registrar) -along with a SPIRE Server, new entries are automatically registered for each new pod that is created. +1. Create an example ClusterSPIFFEID: + + {{< text bash >}} + $ kubectl apply -f - <}} + + The example ClusterSPIFFEID enables automatic workload registration for all workloads with the `spiffe.io/spiffe-id: "true"` label. For pods with this label, the values specified in the `spiffeIDTemplate` will be extracted to form the SPIFFE ID. + +1. Add the `spiffe.io/spiffe-id` label to the Ingress-gateway deployment to register the workload: + + {{< text bash >}} + $ kubectl patch deployment istio-ingressgateway -n istio-system -p '{"spec":{"template":{"metadata":{"labels":{"spiffe.io/spiffe-id": "true"}}}}}' + {{< /text >}} + +1. Deploy an example workload: + + {{< text bash >}} + $ istioctl kube-inject --filename @samples/security/spire/sleep-spire.yaml@ | kubectl apply -f - + {{< /text >}} + + In addition to needing `spiffe.io/spiffe-id` label, the workload will need the SPIFFE CSI Driver volume to access the SPIRE Agent socket. To accomplish this, + you can leverage the `spire` pod annotation template from the [Install Istio](#install-istio) section or add the CSI volume to + the deployment spec of your workload. Both of these alternatives are highlighted on the example snippet below: + + {{< text yaml >}} + apiVersion: apps/v1 + kind: Deployment + metadata: + name: sleep + spec: + replicas: 1 + selector: + matchLabels: + app: sleep + template: + metadata: + labels: + app: sleep + spiffe.io/spiffe-id: true + # Injects custom sidecar template + annotations: + inject.istio.io/templates: "sidecar,spire" + spec: + terminationGracePeriodSeconds: 0 + serviceAccountName: sleep + containers: + - name: sleep + image: curlimages/curl + command: ["/bin/sleep", "3650d"] + imagePullPolicy: IfNotPresent + volumeMounts: + - name: tmp + mountPath: /tmp + securityContext: + runAsUser: 1000 + volumes: + - name: tmp + emptyDir: {} + # CSI volume + - name: workload-socket + csi: + driver: "csi.spiffe.io" + readOnly: true + {{< /text >}} See [Verifying that identities were created for workloads](#verifying-that-identities-were-created-for-workloads) to check issued identities. -Note that `SPIRE workload registrar` is used in the [quick start](#option-1:-quick-start) section. +Note that `SPIRE Controller Manager` is used in the [quick start](#option-1:-quick-start) section. ### Option 2: Manual Registration @@ -328,35 +401,21 @@ See the [SPIRE help on Registering workloads](https://spiffe.io/docs/latest/depl Use the following command to confirm that identities were created for the workloads: {{< text bash >}} -$ kubectl exec -i -t $SPIRE_SERVER_POD -n spire -c spire-server -- /bin/sh -c "bin/spire-server entry show -socketPath /run/spire/sockets/server.sock" -Found 3 entries +$ kubectl exec -t $SPIRE_SERVER_POD -n spire -c spire-server -- ./bin/spire-server entry show +Found 2 entries Entry ID : c8dfccdc-9762-4762-80d3-5434e5388ae7 SPIFFE ID : spiffe://example.org/ns/istio-system/sa/istio-ingressgateway-service-account -Parent ID : spiffe://example.org/ns/spire/sa/spire-agent +Parent ID : spiffe://example.org/spire/agent/k8s_psat/demo-cluster/bea19580-ae04-4679-a22e-472e18ca4687 Revision : 0 TTL : default -Selector : k8s:ns:istio-system Selector : k8s:pod-uid:88b71387-4641-4d9c-9a89-989c88f7509d -Selector : k8s:sa:istio-ingressgateway-service-account -DNS name : istio-ingressgateway-5b45864fd4-lgrxs Entry ID : af7b53dc-4cc9-40d3-aaeb-08abbddd8e54 SPIFFE ID : spiffe://example.org/ns/default/sa/sleep -Parent ID : spiffe://example.org/ns/spire/sa/spire-agent +Parent ID : spiffe://example.org/spire/agent/k8s_psat/demo-cluster/bea19580-ae04-4679-a22e-472e18ca4687 Revision : 0 TTL : default -Selector : k8s:ns:default Selector : k8s:pod-uid:ee490447-e502-46bd-8532-5a746b0871d6 -DNS name : sleep-5f4d47c948-njvpk - -Entry ID : f0544fd7-1945-4bd1-88dc-0a5513fdae1c -SPIFFE ID : spiffe://example.org/ns/spire/sa/spire-agent -Parent ID : spiffe://example.org/spire/server -Revision : 0 -TTL : default -Selector : k8s_psat:agent_ns:spire -Selector : k8s_psat:agent_sa:spire-agent -Selector : k8s_psat:cluster:demo-cluster {{< /text >}} Check the Ingress-gateway pod state: @@ -405,12 +464,19 @@ This will allow Envoy to get federated bundles directly from SPIRE. ### Create federated registration entries -* If using the SPIRE Kubernetes Workload Registrar, create federated entries for workloads by adding the pod annotation `spiffe.io/federatesWith` to the service deployment spec, - specifying the trust domain you want the pod to federate with: +* If using the SPIRE Controller Manager, create federated entries for workloads by setting the `federatesWith` field of the [ClusterSPIFFEID CR](https://github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md) to the trust domains you want the pod to federate with: {{< text yaml >}} - podAnnotations: - spiffe.io/federatesWith: "" + apiVersion: spire.spiffe.io/v1alpha1 + kind: ClusterSPIFFEID + metadata: + name: federation + spec: + spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}" + podSelector: + matchLabels: + spiffe.io/spiffe-id: "true" + federatesWith: ["example.io", "example.ai"] {{< /text >}} * For manual registration see [Create Registration Entries for Federation](https://spiffe.io/docs/latest/architecture/federation/readme/#create-registration-entries-for-federation). @@ -421,16 +487,24 @@ If you installed SPIRE using the quick start SPIRE deployment provided by Istio, use the following commands to remove those Kubernetes resources: {{< text bash >}} -$ kubectl delete CustomResourceDefinition spiffeids.spiffeid.spiffe.io +$ kubectl delete CustomResourceDefinition clusterspiffeids.spiffeid.spiffe.io +$ kubectl delete CustomResourceDefinition clusterfederatedtrustdomains.spire.spiffe.io +$ kubectl delete -n spire configmap spire-bundle $ kubectl delete -n spire serviceaccount spire-agent $ kubectl delete -n spire configmap spire-agent -$ kubectl delete -n spire deployment spire-agent +$ kubectl delete -n spire daemonset spire-agent $ kubectl delete csidriver csi.spiffe.io +$ kubectl delete ValidatingWebhookConfiguration spire-controller-manager-webhook +$ kubectl delete -n spire configmap spire-controller-manager-config $ kubectl delete -n spire configmap spire-server +$ kubectl delete -n spire service spire-controller-manager-webhook-service +$ kubectl delete -n spire service spire-server-bundle-endpoint $ kubectl delete -n spire service spire-server $ kubectl delete -n spire serviceaccount spire-server -$ kubectl delete -n spire statefulset spire-server -$ kubectl delete clusterrole spire-server-trust-role spire-agent-cluster-role -$ kubectl delete clusterrolebinding spire-server-trust-role-binding spire-agent-cluster-role-binding +$ kubectl delete -n spire deployment spire-server +$ kubectl delete clusterrole spire-server-cluster-role spire-agent-cluster-role manager-role +$ kubectl delete clusterrolebinding spire-server-cluster-role-binding spire-agent-cluster-role-binding manager-role-binding +$ kubectl delete role spire-server-role leader-election-role +$ kubectl delete rolebinding spire-server-role-binding leader-election-role-binding $ kubectl delete namespace spire {{< /text >}} From d01a807f6fc4d8966baebd42481ed904fa1ba09b Mon Sep 17 00:00:00 2001 From: jaellio Date: Fri, 24 Feb 2023 10:49:26 -0800 Subject: [PATCH 2/9] Adds test for automatic workload registration via the SPIRE controller manager. During cleanup, removes generated istio.yaml and chaim.pem files. Updates label to spiffe.io/spire-managed-identity. Signed-off-by: jaellio --- .../spire/automatic_registration_test.sh | 69 +++++ .../en/docs/ops/integrations/spire/index.md | 89 +++--- .../en/docs/ops/integrations/spire/snips.sh | 268 ++++++++++++++++++ 3 files changed, 387 insertions(+), 39 deletions(-) create mode 100644 content/en/docs/ops/integrations/spire/automatic_registration_test.sh create mode 100644 content/en/docs/ops/integrations/spire/snips.sh diff --git a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh new file mode 100644 index 0000000000000..ccb0f539e21f1 --- /dev/null +++ b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh @@ -0,0 +1,69 @@ +#!/bin/bash +# shellcheck disable=SC2034,SC2153,SC2154,SC2155,SC2164 + +# Copyright Istio Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -e +set -u +set -o pipefail + +# Install SPIRE configured with k8s Controller Manager +snip_install_spire_with_controller_manager +_wait_for_daemonset spire spire-agent +_wait_for_deployment spire spire-server + +# Istall Istio +# @setup profile=none +set +u # Do not exit when value is unset. CHECK_FILE in the IstioOperator might be unset +snip_define_istio_operator +if ! istioctl install --set tag="$TAG" --set hub="$HUB" --skip-confirmation -f ./istio.yaml +then + echo "Deployment istio-ingressgateway is not ready as expected" +else + echo "Istio install succeeded, expected istio-ingressgateway to not be ready" + return 1 +fi +set -u # Exit on unset value +_wait_for_deployment istio-system istiod + +# Create ClusterSPIFFEID +snip_create_clusterspiffeid + +# Add registration label to ingress-gateway +snip_label_ingressgateway +_wait_for_deployment istio-system istio-ingressgateway + +# Deploy sleep application with registration label +snip_apply_sleep +_wait_for_deployment default sleep + +# Set spire-server pod variable +snip_set_spire_server_pod_name_var + +# Verify registration identities were created for sleep and ingress gateway +_verify_contains snip_verifying_that_identities_were_created_for_workloads_1 "spiffe://example.org/ns/default/sa/sleep" +_verify_contains snip_verifying_that_identities_were_created_for_workloads_1 "spiffe://example.org/ns/istio-system/sa/istio-ingressgateway-service-account" + +# Set sleep pod and pod uid variables +snip_set_sleep_pod_vars + +# Verify sleep workload identity was issued by SPIRE +snip_get_sleep_svid +_verify_contains snip_get_svid_subject "O = SPIRE" + +# @cleanup +snip_cleanup_spire_1 +istioctl uninstall --purge --skip-confirmation +kubectl delete ns istio-system \ No newline at end of file diff --git a/content/en/docs/ops/integrations/spire/index.md b/content/en/docs/ops/integrations/spire/index.md index 7f702d8bbfeff..61cb32bbc2591 100644 --- a/content/en/docs/ops/integrations/spire/index.md +++ b/content/en/docs/ops/integrations/spire/index.md @@ -5,7 +5,7 @@ weight: 31 keywords: [kubernetes,spiffe,spire] aliases: owner: istio/wg-networking-maintainers -test: no +test: yes --- [SPIRE](https://spiffe.io/docs/latest/spire-about/spire-concepts/) is a production-ready implementation of the SPIFFE specification that performs node and workload attestation in order to securely @@ -31,7 +31,7 @@ The integration is compatible with Istio upgrades. Istio provides a basic sample installation to quickly get SPIRE up and running: -{{< text bash >}} +{{< text syntax=bash snip_id=install_spire_with_controller_manager >}} $ kubectl apply -f @samples/security/spire/spire-quickstart.yaml@ {{< /text >}} @@ -73,8 +73,10 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b 1. After [deploying SPIRE](#install-spire) into your environment, and verifying that all deployments are in `Ready` state, install Istio with custom patches for the Ingress-gateway as well as for istio-proxy. - {{< text bash >}} - $ istioctl install --skip-confirmation -f - <}} + $ cat < ./istio.yaml apiVersion: install.istio.io/v1alpha1 kind: IstioOperator metadata: @@ -139,15 +141,21 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b - sh - "-c" - |- - echo `date -Iseconds` Waiting for: ${CHECK_FILE} + echo "$(date -Iseconds)" Waiting for: ${CHECK_FILE} while [[ ! -e ${CHECK_FILE} ]] ; do - echo `date -Iseconds` File does not exist: ${CHECK_FILE} + echo "$(date -Iseconds)" File does not exist: ${CHECK_FILE} sleep 15 done ls -l ${CHECK_FILE} EOF {{< /text >}} + Apply the configuration: + + {{< text bash >}} + $ istioctl install --skip-confirmation -f ./istio.yaml + {{< /text >}} + This will share the `spiffe-csi-driver` with the Ingress Gateway and the sidecars that are going to be injected on workload pods, granting them access to the SPIRE Agent's UNIX Domain Socket. @@ -155,7 +163,7 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b Check Ingress-gateway pod state: -{{< text bash >}} +{{< text syntax=bash snip_id=none >}} $ kubectl get pods -n istio-system NAME READY STATUS RESTARTS AGE istio-ingressgateway-5b45864fd4-lgrxs 0/1 Running 0 17s @@ -177,7 +185,7 @@ along with a SPIRE Server, new entries can be automatically registered for each 1. Create an example ClusterSPIFFEID: - {{< text bash >}} + {{< text syntax=bash snip_id=create_clusterspiffeid >}} $ kubectl apply -f - <}} - The example ClusterSPIFFEID enables automatic workload registration for all workloads with the `spiffe.io/spiffe-id: "true"` label. For pods with this label, the values specified in the `spiffeIDTemplate` will be extracted to form the SPIFFE ID. + The example ClusterSPIFFEID enables automatic workload registration for all workloads with the `spiffe.io/spire-managed-identity: "true"` label. For pods with this label, the values specified in the `spiffeIDTemplate` will be extracted to form the SPIFFE ID. -1. Add the `spiffe.io/spiffe-id` label to the Ingress-gateway deployment to register the workload: +1. Add the `spiffe.io/spire-managed-identity` label to the Ingress-gateway deployment to register the workload: - {{< text bash >}} - $ kubectl patch deployment istio-ingressgateway -n istio-system -p '{"spec":{"template":{"metadata":{"labels":{"spiffe.io/spiffe-id": "true"}}}}}' + {{< text syntax=bash snip_id=label_ingressgateway >}} + $ kubectl patch deployment istio-ingressgateway -n istio-system -p '{"spec":{"template":{"metadata":{"labels":{"spiffe.io/spire-managed-identity": "true"}}}}}' {{< /text >}} 1. Deploy an example workload: - {{< text bash >}} + {{< text syntax=bash snip_id=apply_sleep >}} $ istioctl kube-inject --filename @samples/security/spire/sleep-spire.yaml@ | kubectl apply -f - {{< /text >}} - In addition to needing `spiffe.io/spiffe-id` label, the workload will need the SPIFFE CSI Driver volume to access the SPIRE Agent socket. To accomplish this, + In addition to needing `spiffe.io/spire-managed-identity` label, the workload will need the SPIFFE CSI Driver volume to access the SPIRE Agent socket. To accomplish this, you can leverage the `spire` pod annotation template from the [Install Istio](#install-istio) section or add the CSI volume to the deployment spec of your workload. Both of these alternatives are highlighted on the example snippet below: - {{< text yaml >}} + {{< text syntax=yaml snip_id=none >}} apiVersion: apps/v1 kind: Deployment metadata: @@ -223,7 +231,7 @@ along with a SPIRE Server, new entries can be automatically registered for each metadata: labels: app: sleep - spiffe.io/spiffe-id: true + spiffe.io/spire-managed-identity: "true" # Injects custom sidecar template annotations: inject.istio.io/templates: "sidecar,spire" @@ -264,19 +272,19 @@ To improve workload attestation security robustness, SPIRE is able to verify aga {{< text bash >}} $ INGRESS_POD=$(kubectl get pod -l istio=ingressgateway -n istio-system -o jsonpath="{.items[0].metadata.name}") - $ INGRESS_POD_UID=$(kubectl get pods -n istio-system $INGRESS_POD -o jsonpath='{.metadata.uid}') + $ INGRESS_POD_UID=$(kubectl get pods -n istio-system "$INGRESS_POD" -o jsonpath='{.metadata.uid}') {{< /text >}} 1. Get the spire-server pod: - {{< text bash >}} + {{< text syntax=bash snip_id=set_spire_server_pod_name_var >}} $ SPIRE_SERVER_POD=$(kubectl get pod -l app=spire-server -n spire -o jsonpath="{.items[0].metadata.name}") {{< /text >}} 1. Register an entry for the SPIRE Agent running on the node: {{< text bash >}} - $ kubectl exec -n spire $SPIRE_SERVER_POD -- \ + $ kubectl exec -n spire "$SPIRE_SERVER_POD" -- \ /opt/spire/bin/spire-server entry create \ -spiffeID spiffe://example.org/ns/spire/sa/spire-agent \ -selector k8s_psat:cluster:demo-cluster \ @@ -297,14 +305,14 @@ To improve workload attestation security robustness, SPIRE is able to verify aga 1. Register an entry for the Ingress-gateway pod: {{< text bash >}} - $ kubectl exec -n spire $SPIRE_SERVER_POD -- \ + $ kubectl exec -n spire "$SPIRE_SERVER_POD" -- \ /opt/spire/bin/spire-server entry create \ -spiffeID spiffe://example.org/ns/istio-system/sa/istio-ingressgateway-service-account \ -parentID spiffe://example.org/ns/spire/sa/spire-agent \ -selector k8s:sa:istio-ingressgateway-service-account \ -selector k8s:ns:istio-system \ - -selector k8s:pod-uid:$INGRESS_POD_UID \ - -dns $INGRESS_POD \ + -selector k8s:pod-uid:"$INGRESS_POD_UID" \ + -dns "$INGRESS_POD" \ -dns istio-ingressgateway.istio-system.svc \ -socketPath /run/spire/sockets/server.sock @@ -330,7 +338,7 @@ To improve workload attestation security robustness, SPIRE is able to verify aga you can leverage the `spire` pod annotation template from the [Install Istio](#install-istio) section or add the CSI volume to the deployment spec of your workload. Both of these alternatives are highlighted on the example snippet below: - {{< text yaml >}} + {{< text syntax=yaml snip_id=none >}} apiVersion: apps/v1 kind: Deployment metadata: @@ -372,21 +380,21 @@ To improve workload attestation security robustness, SPIRE is able to verify aga 1. Get pod information: - {{< text bash >}} + {{< text syntax=bash snip_id=set_sleep_pod_vars >}} $ SLEEP_POD=$(kubectl get pod -l app=sleep -o jsonpath="{.items[0].metadata.name}") - $ SLEEP_POD_UID=$(kubectl get pods $SLEEP_POD -o jsonpath='{.metadata.uid}') + $ SLEEP_POD_UID=$(kubectl get pods "$SLEEP_POD" -o jsonpath='{.metadata.uid}') {{< /text >}} 1. Register the workload: {{< text bash >}} - $ kubectl exec -n spire spire-server-0 -- \ + $ kubectl exec -n spire "$SPIRE_SERVER_POD" -- \ /opt/spire/bin/spire-server entry create \ -spiffeID spiffe://example.org/ns/default/sa/sleep \ -parentID spiffe://example.org/ns/spire/sa/spire-agent \ -selector k8s:ns:default \ - -selector k8s:pod-uid:$SLEEP_POD_UID \ - -dns $SLEEP_POD \ + -selector k8s:pod-uid:"$SLEEP_POD_UID" \ + -dns "$SLEEP_POD" \ -socketPath /run/spire/sockets/server.sock {{< /text >}} @@ -401,26 +409,28 @@ See the [SPIRE help on Registering workloads](https://spiffe.io/docs/latest/depl Use the following command to confirm that identities were created for the workloads: {{< text bash >}} -$ kubectl exec -t $SPIRE_SERVER_POD -n spire -c spire-server -- ./bin/spire-server entry show +$ kubectl exec -t "$SPIRE_SERVER_POD" -n spire -c spire-server -- ./bin/spire-server entry show Found 2 entries Entry ID : c8dfccdc-9762-4762-80d3-5434e5388ae7 SPIFFE ID : spiffe://example.org/ns/istio-system/sa/istio-ingressgateway-service-account Parent ID : spiffe://example.org/spire/agent/k8s_psat/demo-cluster/bea19580-ae04-4679-a22e-472e18ca4687 Revision : 0 -TTL : default +X509-SVID TTL : default +JWT-SVID TTL : default Selector : k8s:pod-uid:88b71387-4641-4d9c-9a89-989c88f7509d Entry ID : af7b53dc-4cc9-40d3-aaeb-08abbddd8e54 SPIFFE ID : spiffe://example.org/ns/default/sa/sleep Parent ID : spiffe://example.org/spire/agent/k8s_psat/demo-cluster/bea19580-ae04-4679-a22e-472e18ca4687 Revision : 0 -TTL : default +X509-SVID TTL : default +JWT-SVID TTL : default Selector : k8s:pod-uid:ee490447-e502-46bd-8532-5a746b0871d6 {{< /text >}} Check the Ingress-gateway pod state: -{{< text bash >}} +{{< text syntax=bash snip_id=none >}} $ kubectl get pods -n istio-system NAME READY STATUS RESTARTS AGE istio-ingressgateway-5b45864fd4-lgrxs 1/1 Running 0 60s @@ -433,14 +443,14 @@ After registering an entry for the Ingress-gateway pod, Envoy receives the ident 1. Retrieve sleep's SVID identity document using the istioctl proxy-config secret command: - {{< text bash >}} - $ istioctl proxy-config secret $SLEEP_POD -o json | jq -r \ + {{< text syntax=bash snip_id=get_sleep_svid >}} + $ istioctl proxy-config secret "$SLEEP_POD" -o json | jq -r \ '.dynamicActiveSecrets[0].secret.tlsCertificate.certificateChain.inlineBytes' | base64 --decode > chain.pem {{< /text >}} 1. Inspect the certificate and verify that SPIRE was the issuer: - {{< text bash >}} + {{< text syntax=bash snip_id=get_svid_subject >}} $ openssl x509 -in chain.pem -text | grep SPIRE Subject: C = US, O = SPIRE, CN = sleep-5f4d47c948-njvpk {{< /text >}} @@ -466,7 +476,7 @@ This will allow Envoy to get federated bundles directly from SPIRE. * If using the SPIRE Controller Manager, create federated entries for workloads by setting the `federatesWith` field of the [ClusterSPIFFEID CR](https://github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md) to the trust domains you want the pod to federate with: - {{< text yaml >}} + {{< text syntax=yaml snip_id=none >}} apiVersion: spire.spiffe.io/v1alpha1 kind: ClusterSPIFFEID metadata: @@ -475,7 +485,7 @@ This will allow Envoy to get federated bundles directly from SPIRE. spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}" podSelector: matchLabels: - spiffe.io/spiffe-id: "true" + spiffe.io/spire-managed-identity: "true" federatesWith: ["example.io", "example.ai"] {{< /text >}} @@ -487,7 +497,7 @@ If you installed SPIRE using the quick start SPIRE deployment provided by Istio, use the following commands to remove those Kubernetes resources: {{< text bash >}} -$ kubectl delete CustomResourceDefinition clusterspiffeids.spiffeid.spiffe.io +$ kubectl delete CustomResourceDefinition clusterspiffeids.spire.spiffe.io $ kubectl delete CustomResourceDefinition clusterfederatedtrustdomains.spire.spiffe.io $ kubectl delete -n spire configmap spire-bundle $ kubectl delete -n spire serviceaccount spire-agent @@ -507,4 +517,5 @@ $ kubectl delete clusterrolebinding spire-server-cluster-role-binding spire-agen $ kubectl delete role spire-server-role leader-election-role $ kubectl delete rolebinding spire-server-role-binding leader-election-role-binding $ kubectl delete namespace spire +$ rm istio.yaml chain.pem {{< /text >}} diff --git a/content/en/docs/ops/integrations/spire/snips.sh b/content/en/docs/ops/integrations/spire/snips.sh new file mode 100644 index 0000000000000..914a8d74422aa --- /dev/null +++ b/content/en/docs/ops/integrations/spire/snips.sh @@ -0,0 +1,268 @@ +#!/bin/bash +# shellcheck disable=SC2034,SC2153,SC2155,SC2164 + +# Copyright Istio Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#################################################################################################### +# WARNING: THIS IS AN AUTO-GENERATED FILE, DO NOT EDIT. PLEASE MODIFY THE ORIGINAL MARKDOWN FILE: +# docs/ops/integrations/spire/index.md +#################################################################################################### + +snip_install_spire_with_controller_manager() { +kubectl apply -f samples/security/spire/spire-quickstart.yaml +} + +! read -r -d '' snip_spire_ca_integration_prerequisites_1 <<\ENDSNIP +socket_path = "/run/secrets/workload-spiffe-uds/socket" +ENDSNIP + +snip_define_istio_operator() { +cat < ./istio.yaml +apiVersion: install.istio.io/v1alpha1 +kind: IstioOperator +metadata: + namespace: istio-system +spec: + profile: default + meshConfig: + trustDomain: example.org + values: + global: + # This is used to customize the sidecar template + sidecarInjectorWebhook: + templates: + spire: | + spec: + containers: + - name: istio-proxy + volumeMounts: + - name: workload-socket + mountPath: /run/secrets/workload-spiffe-uds + readOnly: true + volumes: + - name: workload-socket + csi: + driver: "csi.spiffe.io" + readOnly: true + components: + ingressGateways: + - name: istio-ingressgateway + enabled: true + label: + istio: ingressgateway + k8s: + overlays: + - apiVersion: apps/v1 + kind: Deployment + name: istio-ingressgateway + patches: + - path: spec.template.spec.volumes.[name:workload-socket] + value: + name: workload-socket + csi: + driver: "csi.spiffe.io" + readOnly: true + - path: spec.template.spec.containers.[name:istio-proxy].volumeMounts.[name:workload-socket] + value: + name: workload-socket + mountPath: "/run/secrets/workload-spiffe-uds" + readOnly: true + - path: spec.template.spec.initContainers + value: + - name: wait-for-spire-socket + image: busybox:1.28 + volumeMounts: + - name: workload-socket + mountPath: /run/secrets/workload-spiffe-uds + readOnly: true + env: + - name: CHECK_FILE + value: /run/secrets/workload-spiffe-uds/socket + command: + - sh + - "-c" + - |- + echo "$(date -Iseconds)" Waiting for: ${CHECK_FILE} + while [[ ! -e ${CHECK_FILE} ]] ; do + echo "$(date -Iseconds)" File does not exist: ${CHECK_FILE} + sleep 15 + done + ls -l ${CHECK_FILE} +EOF +} + +snip_install_istio_2() { +istioctl install --set values.pilot.env.PILOT_ENABLE_CONFIG_DISTRIBUTION_TRACKING=true --skip-confirmation -f ./istio.yaml +} + +snip_create_clusterspiffeid() { +kubectl apply -f - < chain.pem +} + +snip_get_svid_subject() { +openssl x509 -in chain.pem -text | grep SPIRE +} + +! read -r -d '' snip_get_svid_subject_out <<\ENDSNIP + Subject: C = US, O = SPIRE, CN = sleep-5f4d47c948-njvpk +ENDSNIP + +snip_cleanup_spire_1() { +kubectl delete CustomResourceDefinition clusterspiffeids.spire.spiffe.io +kubectl delete CustomResourceDefinition clusterfederatedtrustdomains.spire.spiffe.io +kubectl delete -n spire configmap spire-bundle +kubectl delete -n spire serviceaccount spire-agent +kubectl delete -n spire configmap spire-agent +kubectl delete -n spire daemonset spire-agent +kubectl delete csidriver csi.spiffe.io +kubectl delete ValidatingWebhookConfiguration spire-controller-manager-webhook +kubectl delete -n spire configmap spire-controller-manager-config +kubectl delete -n spire configmap spire-server +kubectl delete -n spire service spire-controller-manager-webhook-service +kubectl delete -n spire service spire-server-bundle-endpoint +kubectl delete -n spire service spire-server +kubectl delete -n spire serviceaccount spire-server +kubectl delete -n spire deployment spire-server +kubectl delete clusterrole spire-server-cluster-role spire-agent-cluster-role manager-role +kubectl delete clusterrolebinding spire-server-cluster-role-binding spire-agent-cluster-role-binding manager-role-binding +kubectl delete role spire-server-role leader-election-role +kubectl delete rolebinding spire-server-role-binding leader-election-role-binding +kubectl delete namespace spire +rm istio.yaml chain.pem +} From 45dafa184171e19ffaacbcadf130bcdd57feac94 Mon Sep 17 00:00:00 2001 From: jaellio Date: Fri, 24 Feb 2023 11:03:01 -0800 Subject: [PATCH 3/9] Adds missing newline Signed-off-by: jaellio --- .../docs/ops/integrations/spire/automatic_registration_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh index ccb0f539e21f1..c118eee982691 100644 --- a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh +++ b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh @@ -66,4 +66,4 @@ _verify_contains snip_get_svid_subject "O = SPIRE" # @cleanup snip_cleanup_spire_1 istioctl uninstall --purge --skip-confirmation -kubectl delete ns istio-system \ No newline at end of file +kubectl delete ns istio-system From 812f0b1d729ac298115e2103989a2062987526f0 Mon Sep 17 00:00:00 2001 From: jaellio Date: Fri, 24 Feb 2023 12:30:41 -0800 Subject: [PATCH 4/9] Fix spelling error Signed-off-by: jaellio --- .../docs/ops/integrations/spire/automatic_registration_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh index c118eee982691..c258979038d3c 100644 --- a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh +++ b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh @@ -24,7 +24,7 @@ snip_install_spire_with_controller_manager _wait_for_daemonset spire spire-agent _wait_for_deployment spire spire-server -# Istall Istio +# Install Istio # @setup profile=none set +u # Do not exit when value is unset. CHECK_FILE in the IstioOperator might be unset snip_define_istio_operator From dab4acb798e5f84c46a91ecb296b2cf7c9960b3d Mon Sep 17 00:00:00 2001 From: jaellio Date: Mon, 27 Feb 2023 17:40:16 -0800 Subject: [PATCH 5/9] Add missing ns flag on role and rolebinding resource commands Signed-off-by: jaellio --- content/en/docs/ops/integrations/spire/index.md | 4 ++-- content/en/docs/ops/integrations/spire/snips.sh | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/ops/integrations/spire/index.md b/content/en/docs/ops/integrations/spire/index.md index 61cb32bbc2591..365fbaf54db6f 100644 --- a/content/en/docs/ops/integrations/spire/index.md +++ b/content/en/docs/ops/integrations/spire/index.md @@ -514,8 +514,8 @@ $ kubectl delete -n spire serviceaccount spire-server $ kubectl delete -n spire deployment spire-server $ kubectl delete clusterrole spire-server-cluster-role spire-agent-cluster-role manager-role $ kubectl delete clusterrolebinding spire-server-cluster-role-binding spire-agent-cluster-role-binding manager-role-binding -$ kubectl delete role spire-server-role leader-election-role -$ kubectl delete rolebinding spire-server-role-binding leader-election-role-binding +$ kubectl delete -n spire role spire-server-role leader-election-role +$ kubectl delete -n spire rolebinding spire-server-role-binding leader-election-role-binding $ kubectl delete namespace spire $ rm istio.yaml chain.pem {{< /text >}} diff --git a/content/en/docs/ops/integrations/spire/snips.sh b/content/en/docs/ops/integrations/spire/snips.sh index 914a8d74422aa..b56750514192f 100644 --- a/content/en/docs/ops/integrations/spire/snips.sh +++ b/content/en/docs/ops/integrations/spire/snips.sh @@ -261,8 +261,8 @@ kubectl delete -n spire serviceaccount spire-server kubectl delete -n spire deployment spire-server kubectl delete clusterrole spire-server-cluster-role spire-agent-cluster-role manager-role kubectl delete clusterrolebinding spire-server-cluster-role-binding spire-agent-cluster-role-binding manager-role-binding -kubectl delete role spire-server-role leader-election-role -kubectl delete rolebinding spire-server-role-binding leader-election-role-binding +kubectl delete -n spire role spire-server-role leader-election-role +kubectl delete -n spire rolebinding spire-server-role-binding leader-election-role-binding kubectl delete namespace spire rm istio.yaml chain.pem } From 4da77162106e4ebd1e9e21ece25a7c24fb89ae69 Mon Sep 17 00:00:00 2001 From: jaellio Date: Tue, 28 Feb 2023 15:26:02 -0800 Subject: [PATCH 6/9] Delete sleep resources and uninstall before SPIRE Signed-off-by: jaellio --- .../docs/ops/integrations/spire/automatic_registration_test.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh index c258979038d3c..e7edb73acae53 100644 --- a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh +++ b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh @@ -64,6 +64,7 @@ snip_get_sleep_svid _verify_contains snip_get_svid_subject "O = SPIRE" # @cleanup -snip_cleanup_spire_1 +kubectl delete -f samples/security/spire/sleep-spire.yaml istioctl uninstall --purge --skip-confirmation kubectl delete ns istio-system +snip_cleanup_spire_1 From 10e334109c185e12cd66f11ef81bf3939e22809a Mon Sep 17 00:00:00 2001 From: jaellio Date: Wed, 1 Mar 2023 15:06:59 -0800 Subject: [PATCH 7/9] Reconfigures demo so istio install is not expected to fail. Created ClusterSPIFFEID before install istio. Previously install would fail because the ingress gateway wasn't registered/ Signed-off-by: jaellio --- .spelling | 1 + .../spire/automatic_registration_test.sh | 22 +-- .../en/docs/ops/integrations/spire/index.md | 182 ++++++++++++++---- .../en/docs/ops/integrations/spire/snips.sh | 102 ++++++++-- 4 files changed, 236 insertions(+), 71 deletions(-) diff --git a/.spelling b/.spelling index 50b2bed302ec7..54227e4488086 100644 --- a/.spelling +++ b/.spelling @@ -489,6 +489,7 @@ Idit ILBs incentivized Incrementality +initContainer initializer initializers injector diff --git a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh index e7edb73acae53..7e3f69d170c4e 100644 --- a/content/en/docs/ops/integrations/spire/automatic_registration_test.sh +++ b/content/en/docs/ops/integrations/spire/automatic_registration_test.sh @@ -19,30 +19,22 @@ set -e set -u set -o pipefail +# @setup profile=none + # Install SPIRE configured with k8s Controller Manager snip_install_spire_with_controller_manager _wait_for_daemonset spire spire-agent _wait_for_deployment spire spire-server +# Create ClusterSPIFFEID +snip_create_clusterspiffeid + # Install Istio -# @setup profile=none set +u # Do not exit when value is unset. CHECK_FILE in the IstioOperator might be unset -snip_define_istio_operator -if ! istioctl install --set tag="$TAG" --set hub="$HUB" --skip-confirmation -f ./istio.yaml -then - echo "Deployment istio-ingressgateway is not ready as expected" -else - echo "Istio install succeeded, expected istio-ingressgateway to not be ready" - return 1 -fi +snip_define_istio_operator_for_auto_registration +snip_apply_istio_operator_configuration set -u # Exit on unset value _wait_for_deployment istio-system istiod - -# Create ClusterSPIFFEID -snip_create_clusterspiffeid - -# Add registration label to ingress-gateway -snip_label_ingressgateway _wait_for_deployment istio-system istio-ingressgateway # Deploy sleep application with registration label diff --git a/content/en/docs/ops/integrations/spire/index.md b/content/en/docs/ops/integrations/spire/index.md index 365fbaf54db6f..dc146009c6f60 100644 --- a/content/en/docs/ops/integrations/spire/index.md +++ b/content/en/docs/ops/integrations/spire/index.md @@ -68,14 +68,36 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b ## Install Istio -1. [Download Istio release 1.14+](/docs/setup/getting-started/#download). +### Option 1: Configuration for Workload Registration with the SPIRE Controller Manager -1. After [deploying SPIRE](#install-spire) into your environment, and verifying that all deployments are in `Ready` state, - install Istio with custom patches for the Ingress-gateway as well as for istio-proxy. +By deploying [SPIRE Controller Manager](https://github.com/spiffe/spire-controller-manager) +along with a SPIRE Server, new entries can be automatically registered for each new pod that matches the selector defined in a [ClusterSPIFFEID](https://github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md) custom resource. - Create Istio configuration: +A ClusterSPIFFEID must be applied prior to installing Istio in order for the Ingress-gateway to obtain its certificates. Additionally, the Ingress-gateway pod must be configured to match the selector defined in the ClusterSPIFFEID. If a registration entry for the Ingress Gateway workload was not automatically created during install, the workload would not reach a `Ready` state and installation would fail. + +1. Create example ClusterSPIFFEID: + + {{< text syntax=bash snip_id=create_clusterspiffeid >}} + $ kubectl apply -f - <}} - {{< text syntax=bash snip_id=define_istio_operator >}} + The example ClusterSPIFFEID enables automatic workload registration for all workloads with the `spiffe.io/spire-managed-identity: "true"` label. For pods with this label, the values specified in the `spiffeIDTemplate` will be extracted to form the SPIFFE ID. + +1. [Download Istio release 1.14+](/docs/setup/getting-started/#download). + +1. Create the Istio configuration with custom patches for the Ingress-gateway and istio-proxy. The Ingress Gateway component includes the `spiffe.io/spire-managed-identity: "true"` label. + + {{< text syntax=bash snip_id=define_istio_operator_for_auto_registration >}} $ cat < ./istio.yaml apiVersion: install.istio.io/v1alpha1 kind: IstioOperator @@ -109,6 +131,7 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b enabled: true label: istio: ingressgateway + spiffe.io/spire-managed-identity: "true" k8s: overlays: - apiVersion: apps/v1 @@ -150,63 +173,140 @@ Istio will become the Envoy SDS listener if the socket is not created by SPIRE b EOF {{< /text >}} - Apply the configuration: +1. Apply the configuration: - {{< text bash >}} + {{< text syntax=bash snip_id=apply_istio_operator_configuration >}} $ istioctl install --skip-confirmation -f ./istio.yaml {{< /text >}} - This will share the `spiffe-csi-driver` with the Ingress Gateway and the sidecars that are going to be injected on workload pods, - granting them access to the SPIRE Agent's UNIX Domain Socket. +1. Check Ingress-gateway pod state: - This will also add an initContainer to the gateway that will wait for SPIRE to create the UNIX Domain Socket before starting the istio-proxy. If the SPIRE agent is not ready or has not been properly configured with the same socket path, the Ingress Gateway initContainer will wait forever. - -Check Ingress-gateway pod state: - -{{< text syntax=bash snip_id=none >}} -$ kubectl get pods -n istio-system -NAME READY STATUS RESTARTS AGE -istio-ingressgateway-5b45864fd4-lgrxs 0/1 Running 0 17s -istiod-989f54d9c-sg7sn 1/1 Running 0 23s -{{< /text >}} + {{< text syntax=bash snip_id=none >}} + $ kubectl get pods -n istio-system + NAME READY STATUS RESTARTS AGE + istio-ingressgateway-5b45864fd4-lgrxs 1/1 Running 0 17s + istiod-989f54d9c-sg7sn 1/1 Running 0 23s + {{< /text >}} -The Ingress-gateway pod and data plane containers will only reach `Ready` if a corresponding registration entry is created for them on the SPIRE Server. Then, -Envoy will be able to fetch cryptographic identities from SPIRE. -See [Register workloads](#register-workloads) to register entries for services in your mesh. + The Ingress-gateway pod is `Ready` since the corresponding registration entry is automatically created for it on the SPIRE Server. Envoy is able to fetch cryptographic identities from SPIRE. -## Register workloads +Note that `SPIRE Controller Manager` is used in the [quick start](#option-1:-quick-start) section. -This section describes the options available for registering workloads in a SPIRE Server. +### Option 2: Configuration for Manual Workload Registration with SPIRE -### Option 1: Registration using the SPIRE Controller Manager +1. [Download Istio release 1.14+](/docs/setup/getting-started/#download). -By deploying [SPIRE Controller Manager](https://github.com/spiffe/spire-controller-manager) -along with a SPIRE Server, new entries can be automatically registered for each new pod that matches the selector defined in a [ClusterSPIFFEID](https://github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md) custom resource. +1. After [deploying SPIRE](#install-spire) into your environment, and verifying that all deployments are in `Ready` state, install Istio with custom patches for the Ingress-gateway as well as for istio-proxy. -1. Create an example ClusterSPIFFEID: + Create Istio configuration: - {{< text syntax=bash snip_id=create_clusterspiffeid >}} - $ kubectl apply -f - <}} + $ cat < ./istio.yaml + apiVersion: install.istio.io/v1alpha1 + kind: IstioOperator metadata: - name: example + namespace: istio-system spec: - spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}" - podSelector: - matchLabels: - spiffe.io/spire-managed-identity: "true" + profile: default + meshConfig: + trustDomain: example.org + values: + global: + # This is used to customize the sidecar template + sidecarInjectorWebhook: + templates: + spire: | + spec: + containers: + - name: istio-proxy + volumeMounts: + - name: workload-socket + mountPath: /run/secrets/workload-spiffe-uds + readOnly: true + volumes: + - name: workload-socket + csi: + driver: "csi.spiffe.io" + readOnly: true + components: + ingressGateways: + - name: istio-ingressgateway + enabled: true + label: + istio: ingressgateway + k8s: + overlays: + - apiVersion: apps/v1 + kind: Deployment + name: istio-ingressgateway + patches: + - path: spec.template.spec.volumes.[name:workload-socket] + value: + name: workload-socket + csi: + driver: "csi.spiffe.io" + readOnly: true + - path: spec.template.spec.containers.[name:istio-proxy].volumeMounts.[name:workload-socket] + value: + name: workload-socket + mountPath: "/run/secrets/workload-spiffe-uds" + readOnly: true + - path: spec.template.spec.initContainers + value: + - name: wait-for-spire-socket + image: busybox:1.28 + volumeMounts: + - name: workload-socket + mountPath: /run/secrets/workload-spiffe-uds + readOnly: true + env: + - name: CHECK_FILE + value: /run/secrets/workload-spiffe-uds/socket + command: + - sh + - "-c" + - |- + echo "$(date -Iseconds)" Waiting for: ${CHECK_FILE} + while [[ ! -e ${CHECK_FILE} ]] ; do + echo "$(date -Iseconds)" File does not exist: ${CHECK_FILE} + sleep 15 + done + ls -l ${CHECK_FILE} EOF {{< /text >}} - The example ClusterSPIFFEID enables automatic workload registration for all workloads with the `spiffe.io/spire-managed-identity: "true"` label. For pods with this label, the values specified in the `spiffeIDTemplate` will be extracted to form the SPIFFE ID. +1. Apply the configuration: + + {{< text syntax=bash snip_id=none >}} + $ istioctl install --skip-confirmation -f ./istio.yaml + {{< /text >}} -1. Add the `spiffe.io/spire-managed-identity` label to the Ingress-gateway deployment to register the workload: +1. Check Ingress-gateway pod state: - {{< text syntax=bash snip_id=label_ingressgateway >}} - $ kubectl patch deployment istio-ingressgateway -n istio-system -p '{"spec":{"template":{"metadata":{"labels":{"spiffe.io/spire-managed-identity": "true"}}}}}' + {{< text syntax=bash snip_id=none >}} + $ kubectl get pods -n istio-system + NAME READY STATUS RESTARTS AGE + istio-ingressgateway-5b45864fd4-lgrxs 0/1 Running 0 20s + istiod-989f54d9c-sg7sn 1/1 Running 0 25s {{< /text >}} + The Ingress-gateway pod and data plane containers will only reach `Ready` if a corresponding registration entry is created for them on the SPIRE Server. Then, + Envoy will be able to fetch cryptographic identities from SPIRE. + See [Register workloads](#register-workloads) to register entries for services in your mesh. + +The Istio configuration shares the `spiffe-csi-driver` with the Ingress Gateway and the sidecars that are going to be injected on workload pods, +granting them access to the SPIRE Agent's UNIX Domain Socket. + +This configuration also adds an initContainer to the gateway that will wait for SPIRE to create the UNIX Domain Socket before starting the istio-proxy. If the SPIRE agent is not ready or has not been properly configured with the same socket path, the Ingress Gateway initContainer will wait forever. + +## Register workloads + +This section describes the options available for registering workloads in a SPIRE Server. + +### Option 1: Registration using the SPIRE Controller Manager + +New entries will be automatically registered for each new pod that matches the selector defined in a [ClusterSPIFFEID](https://github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md) custom resource. See [Configuration for Workload Registration with the SPIRE Controller Manager](#option-1:-configuration-for-workload-registration-with-the-spire-controller-manager) for the example ClusterSPIFFEID configuration. + 1. Deploy an example workload: {{< text syntax=bash snip_id=apply_sleep >}} diff --git a/content/en/docs/ops/integrations/spire/snips.sh b/content/en/docs/ops/integrations/spire/snips.sh index b56750514192f..c7a889bae556a 100644 --- a/content/en/docs/ops/integrations/spire/snips.sh +++ b/content/en/docs/ops/integrations/spire/snips.sh @@ -28,7 +28,21 @@ kubectl apply -f samples/security/spire/spire-quickstart.yaml socket_path = "/run/secrets/workload-spiffe-uds/socket" ENDSNIP -snip_define_istio_operator() { +snip_create_clusterspiffeid() { +kubectl apply -f - < ./istio.yaml apiVersion: install.istio.io/v1alpha1 kind: IstioOperator @@ -62,6 +76,7 @@ spec: enabled: true label: istio: ingressgateway + spiffe.io/spire-managed-identity: "true" k8s: overlays: - apiVersion: apps/v1 @@ -103,28 +118,85 @@ spec: EOF } -snip_install_istio_2() { +snip_apply_istio_operator_configuration() { istioctl install --set values.pilot.env.PILOT_ENABLE_CONFIG_DISTRIBUTION_TRACKING=true --skip-confirmation -f ./istio.yaml } -snip_create_clusterspiffeid() { -kubectl apply -f - < ./istio.yaml +apiVersion: install.istio.io/v1alpha1 +kind: IstioOperator metadata: - name: example + namespace: istio-system spec: - spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}" - podSelector: - matchLabels: - spiffe.io/spire-managed-identity: "true" + profile: default + meshConfig: + trustDomain: example.org + values: + global: + # This is used to customize the sidecar template + sidecarInjectorWebhook: + templates: + spire: | + spec: + containers: + - name: istio-proxy + volumeMounts: + - name: workload-socket + mountPath: /run/secrets/workload-spiffe-uds + readOnly: true + volumes: + - name: workload-socket + csi: + driver: "csi.spiffe.io" + readOnly: true + components: + ingressGateways: + - name: istio-ingressgateway + enabled: true + label: + istio: ingressgateway + k8s: + overlays: + - apiVersion: apps/v1 + kind: Deployment + name: istio-ingressgateway + patches: + - path: spec.template.spec.volumes.[name:workload-socket] + value: + name: workload-socket + csi: + driver: "csi.spiffe.io" + readOnly: true + - path: spec.template.spec.containers.[name:istio-proxy].volumeMounts.[name:workload-socket] + value: + name: workload-socket + mountPath: "/run/secrets/workload-spiffe-uds" + readOnly: true + - path: spec.template.spec.initContainers + value: + - name: wait-for-spire-socket + image: busybox:1.28 + volumeMounts: + - name: workload-socket + mountPath: /run/secrets/workload-spiffe-uds + readOnly: true + env: + - name: CHECK_FILE + value: /run/secrets/workload-spiffe-uds/socket + command: + - sh + - "-c" + - |- + echo "$(date -Iseconds)" Waiting for: ${CHECK_FILE} + while [[ ! -e ${CHECK_FILE} ]] ; do + echo "$(date -Iseconds)" File does not exist: ${CHECK_FILE} + sleep 15 + done + ls -l ${CHECK_FILE} EOF } -snip_label_ingressgateway() { -kubectl patch deployment istio-ingressgateway -n istio-system -p '{"spec":{"template":{"metadata":{"labels":{"spiffe.io/spire-managed-identity": "true"}}}}}' -} - snip_apply_sleep() { istioctl kube-inject --filename samples/security/spire/sleep-spire.yaml | kubectl apply -f - } From 2c3e0ec1fb528f3710dfd54b6202b21f707d8660 Mon Sep 17 00:00:00 2001 From: jaellio Date: Thu, 2 Mar 2023 10:35:23 -0800 Subject: [PATCH 8/9] Remove references to v1.14 and update required version to 1.14+ Signed-off-by: jaellio --- .../en/docs/ops/integrations/spire/index.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/content/en/docs/ops/integrations/spire/index.md b/content/en/docs/ops/integrations/spire/index.md index dc146009c6f60..e7844c82906a6 100644 --- a/content/en/docs/ops/integrations/spire/index.md +++ b/content/en/docs/ops/integrations/spire/index.md @@ -20,7 +20,7 @@ SPIRE's node attestation extends attestation to the physical or virtual hardware For a quick demo of how this SPIRE integration with Istio works, see [Integrating SPIRE as a CA through Envoy's SDS API]({{< github_tree >}}/samples/security/spire). {{< warning >}} -Note that this integration requires version 1.14 for both `istioctl` and the data plane. +Note that this integration requires version 1.14+ for both `istioctl` and the data plane. {{< /warning >}} The integration is compatible with Istio upgrades. @@ -93,9 +93,9 @@ A ClusterSPIFFEID must be applied prior to installing Istio in order for the Ing The example ClusterSPIFFEID enables automatic workload registration for all workloads with the `spiffe.io/spire-managed-identity: "true"` label. For pods with this label, the values specified in the `spiffeIDTemplate` will be extracted to form the SPIFFE ID. -1. [Download Istio release 1.14+](/docs/setup/getting-started/#download). +2. [Download the Istio release](/docs/setup/getting-started/#download). -1. Create the Istio configuration with custom patches for the Ingress-gateway and istio-proxy. The Ingress Gateway component includes the `spiffe.io/spire-managed-identity: "true"` label. +3. Create the Istio configuration with custom patches for the Ingress-gateway and istio-proxy. The Ingress Gateway component includes the `spiffe.io/spire-managed-identity: "true"` label. {{< text syntax=bash snip_id=define_istio_operator_for_auto_registration >}} $ cat < ./istio.yaml @@ -173,13 +173,13 @@ A ClusterSPIFFEID must be applied prior to installing Istio in order for the Ing EOF {{< /text >}} -1. Apply the configuration: +4. Apply the configuration: {{< text syntax=bash snip_id=apply_istio_operator_configuration >}} $ istioctl install --skip-confirmation -f ./istio.yaml {{< /text >}} -1. Check Ingress-gateway pod state: +5. Check Ingress-gateway pod state: {{< text syntax=bash snip_id=none >}} $ kubectl get pods -n istio-system @@ -194,9 +194,9 @@ Note that `SPIRE Controller Manager` is used in the [quick start](#option-1:-qui ### Option 2: Configuration for Manual Workload Registration with SPIRE -1. [Download Istio release 1.14+](/docs/setup/getting-started/#download). +1. [Download the Istio release](/docs/setup/getting-started/#download). -1. After [deploying SPIRE](#install-spire) into your environment, and verifying that all deployments are in `Ready` state, install Istio with custom patches for the Ingress-gateway as well as for istio-proxy. +2. After [deploying SPIRE](#install-spire) into your environment, and verifying that all deployments are in `Ready` state, install Istio with custom patches for the Ingress-gateway as well as for istio-proxy. Create Istio configuration: @@ -275,13 +275,13 @@ Note that `SPIRE Controller Manager` is used in the [quick start](#option-1:-qui EOF {{< /text >}} -1. Apply the configuration: +3. Apply the configuration: {{< text syntax=bash snip_id=none >}} $ istioctl install --skip-confirmation -f ./istio.yaml {{< /text >}} -1. Check Ingress-gateway pod state: +4. Check Ingress-gateway pod state: {{< text syntax=bash snip_id=none >}} $ kubectl get pods -n istio-system From b84b6863af8ff1c5367213ff9daed756fd4f7ae0 Mon Sep 17 00:00:00 2001 From: jaellio Date: Thu, 2 Mar 2023 13:08:08 -0800 Subject: [PATCH 9/9] Fix lint errors Signed-off-by: jaellio --- content/en/docs/ops/integrations/spire/index.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/content/en/docs/ops/integrations/spire/index.md b/content/en/docs/ops/integrations/spire/index.md index e7844c82906a6..274e4de9dde88 100644 --- a/content/en/docs/ops/integrations/spire/index.md +++ b/content/en/docs/ops/integrations/spire/index.md @@ -93,9 +93,9 @@ A ClusterSPIFFEID must be applied prior to installing Istio in order for the Ing The example ClusterSPIFFEID enables automatic workload registration for all workloads with the `spiffe.io/spire-managed-identity: "true"` label. For pods with this label, the values specified in the `spiffeIDTemplate` will be extracted to form the SPIFFE ID. -2. [Download the Istio release](/docs/setup/getting-started/#download). +1. [Download the Istio release](/docs/setup/getting-started/#download). -3. Create the Istio configuration with custom patches for the Ingress-gateway and istio-proxy. The Ingress Gateway component includes the `spiffe.io/spire-managed-identity: "true"` label. +1. Create the Istio configuration with custom patches for the Ingress-gateway and istio-proxy. The Ingress Gateway component includes the `spiffe.io/spire-managed-identity: "true"` label. {{< text syntax=bash snip_id=define_istio_operator_for_auto_registration >}} $ cat < ./istio.yaml @@ -173,13 +173,13 @@ A ClusterSPIFFEID must be applied prior to installing Istio in order for the Ing EOF {{< /text >}} -4. Apply the configuration: +1. Apply the configuration: {{< text syntax=bash snip_id=apply_istio_operator_configuration >}} $ istioctl install --skip-confirmation -f ./istio.yaml {{< /text >}} -5. Check Ingress-gateway pod state: +1. Check Ingress-gateway pod state: {{< text syntax=bash snip_id=none >}} $ kubectl get pods -n istio-system @@ -196,7 +196,7 @@ Note that `SPIRE Controller Manager` is used in the [quick start](#option-1:-qui 1. [Download the Istio release](/docs/setup/getting-started/#download). -2. After [deploying SPIRE](#install-spire) into your environment, and verifying that all deployments are in `Ready` state, install Istio with custom patches for the Ingress-gateway as well as for istio-proxy. +1. After [deploying SPIRE](#install-spire) into your environment, and verifying that all deployments are in `Ready` state, configure Istio with custom patches for the Ingress-gateway as well as for istio-proxy. Create Istio configuration: @@ -275,13 +275,13 @@ Note that `SPIRE Controller Manager` is used in the [quick start](#option-1:-qui EOF {{< /text >}} -3. Apply the configuration: +1. Apply the configuration: {{< text syntax=bash snip_id=none >}} $ istioctl install --skip-confirmation -f ./istio.yaml {{< /text >}} -4. Check Ingress-gateway pod state: +1. Check Ingress-gateway pod state: {{< text syntax=bash snip_id=none >}} $ kubectl get pods -n istio-system