[terraform/gcp] Create separate SA credentials for each dependency

[nandajavarma]

Description

Currently, in the GCP single cluster setup, we suggest the user to use the same service account for database, registry, etc. This PR splits the GKE terraform module to multiple smaller files and creates separate service accounts for each of the resource being creating including DNS, database, registry & storage and GKE cluster. This makes this module work very similar to the gitpod-gke-guide.

Related Issue(s)

Fixes #12609

How to test

Please follow the README of the GCP module.

Release Notes

NONE

Documentation

Werft options:

• /werft with-preview

[nandajavarma]

Converting to draft to fix a bug I encountered

[nandajavarma]

The issue was that the naming for service accounts started with numerals in test environments, this is not abiding by the pattern GCP accepts. Hence tests were failing. Force pushed the fix for it.

[adrienthebo]

Comments added, some questions might be out of scope as they may touch code that's being purely refactored.

In summary:

• We can probably snip out some repetitive code with a for_each loop (note that we need to use a for_each over iteration because terraform does silly things when iterating over arrays)
• We're passing the dns credentials as a local file; this looks like a hidden dependency.
• The service account and OAuth scopes seem broadly scoped for the workspaces; we may be able to trim this down (but this may be out of scope)

[adrienthebo]

Flagging this as a potential concern:

werft run github -j .werft/gke-installer-tests.yaml -a deps=external

[create-std-gke-cluster] STDERR: Activated service account credentials for: [[email protected]]
[create-std-gke-cluster] Fetching cluster endpoint and auth data.
[create-std-gke-cluster] ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=404, message=Not found: projects/sh-automated-tests/zones/europe-west1-d/clusters/gp-bb017-gcp.
[create-std-gke-cluster] No cluster named 'gp-bb017-gcp' in sh-automated-tests.
[create-std-gke-cluster] Activated service account credentials for: [[email protected]]
[create-std-gke-cluster] CommandException: No URLs matched: gs://nightly-tests/tf-state/bb017-gcp-kubeconfig

Edit: I tracked down the origin of the issue; it looks like our kubectl credential fetching tries a handful of methods and may emit an error if one of those methods failed. This appears to be a spurious failure.

[adrienthebo]

/hold for comment review, merge at your discretion

Noting this for later - reviewing this PR required having a moderate amount of self-hosted knowledge to understand the different moving parts and what configuration is required. When we've gotten automated testing to a fairly stable test let's bring in an outside perspective on running through this to identify the assumptions we're making and how customers might use this module.

Looks good otherwise! 🚢 at your discretion

[nandajavarma]

/unhold