Skip to content

[terraform/gcp] Create separate SA credentials for each dependency #12614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 14, 2022

Conversation

nandajavarma
Copy link
Contributor

@nandajavarma nandajavarma commented Sep 2, 2022

Description

Currently, in the GCP single cluster setup, we suggest the user to use the same service account for database, registry, etc. This PR splits the GKE terraform module to multiple smaller files and creates separate service accounts for each of the resource being creating including DNS, database, registry & storage and GKE cluster. This makes this module work very similar to the gitpod-gke-guide.

Related Issue(s)

Fixes #12609

How to test

Please follow the README of the GCP module.

Release Notes

NONE

Documentation

Werft options:

  • /werft with-preview

@nandajavarma
Copy link
Contributor Author

Converting to draft to fix a bug I encountered

@nandajavarma
Copy link
Contributor Author

The issue was that the naming for service accounts started with numerals in test environments, this is not abiding by the pattern GCP accepts. Hence tests were failing. Force pushed the fix for it.

@nandajavarma nandajavarma marked this pull request as ready for review September 5, 2022 12:07
Copy link
Contributor

@adrienthebo adrienthebo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments added, some questions might be out of scope as they may touch code that's being purely refactored.

In summary:

  • We can probably snip out some repetitive code with a for_each loop (note that we need to use a for_each over iteration because terraform does silly things when iterating over arrays)
  • We're passing the dns credentials as a local file; this looks like a hidden dependency.
  • The service account and OAuth scopes seem broadly scoped for the workspaces; we may be able to trim this down (but this may be out of scope)

@adrienthebo
Copy link
Contributor

adrienthebo commented Sep 13, 2022

Flagging this as a potential concern:

werft run github -j .werft/gke-installer-tests.yaml -a deps=external
[create-std-gke-cluster] STDERR: Activated service account credentials for: [[email protected]]
[create-std-gke-cluster] Fetching cluster endpoint and auth data.
[create-std-gke-cluster] ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=404, message=Not found: projects/sh-automated-tests/zones/europe-west1-d/clusters/gp-bb017-gcp.
[create-std-gke-cluster] No cluster named 'gp-bb017-gcp' in sh-automated-tests.
[create-std-gke-cluster] Activated service account credentials for: [[email protected]]
[create-std-gke-cluster] CommandException: No URLs matched: gs://nightly-tests/tf-state/bb017-gcp-kubeconfig

Edit: I tracked down the origin of the issue; it looks like our kubectl credential fetching tries a handful of methods and may emit an error if one of those methods failed. This appears to be a spurious failure.

Copy link
Contributor

@adrienthebo adrienthebo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/hold for comment review, merge at your discretion

Noting this for later - reviewing this PR required having a moderate amount of self-hosted knowledge to understand the different moving parts and what configuration is required. When we've gotten automated testing to a fairly stable test let's bring in an outside perspective on running through this to identify the assumptions we're making and how customers might use this module.

Looks good otherwise! 🚢 at your discretion

@nandajavarma
Copy link
Contributor Author

/unhold

@roboquat roboquat merged commit a56bd65 into main Sep 14, 2022
@roboquat roboquat deleted the nvn/fix-12609 branch September 14, 2022 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none size/XXL team: delivery Issue belongs to the self-hosted team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

terraform: Create separate service accounts in GKE module
3 participants