docs: Breaking down Pipelines architecture concepts #2753
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing touches🧪 Generate unit tests
Comment |
Co-authored-by: Josh Padnick <[email protected]>
Co-authored-by: Josh Padnick <[email protected]>
yhakbar
force-pushed
the
yousif/dev-1088-break-down-authenticating-to-the-cloud-to-avoid-assuming-aws
branch
from
ce4a816 to
99522a4
Compare
yhakbar
changed the base branch from
yousif/dev-1088-break-down-authenticating-to-the-cloud-to-avoid-assuming-aws
to
pipelines-v4
…ount Factory page
yhakbar
force-pushed
the
yousif/dev-1096-break-down-architecture-to-avoid-assuming-aws
branch
from
9405d3e to
81af9f6
Compare
…ure-to-avoid-assuming-aws
| @@ -0,0 +1,143 @@ | |||
| # Repository Topology | |||
|
|
|||
| Gruntwork Account Factory provides an opinionated (but flexible) repository structure that supports organizations as they scale their infrastructure management across multiple AWS accounts. This approach is designed to help teams graduate from managing a handful of accounts with difficulty to being able to conveniently manage hundreds of accounts, all while maintaining high standards for security, compliance, and developer productivity. | |||
There was a problem hiding this comment.
Gruntwork Account Factory provides an opinionated (but flexible) repository structure...
It might be worth clarifying what we mean by "provides" here. Is it more accurate to say that Gruntwork Account Factory assumes that you set up the following repo structure and is designed to work within that structure?
Alternatively, when I do my own docs update in a few weeks (hopefully soon), I plan to create a whole section on the AWS Platform Architecture. I'm wondering if the three-repo structure discussion belongs there, and then here we say that Account Factory assumes that structure?
If that resonates, then we should leave this as is for now, and I can just adjust that (via reviewable PR) later on.
|
|
||
| Think of `infrastructure-live-root` as your organization's infrastructure command center. This repository, built from the [infrastructure-live-root-template](https://github.com/gruntwork-io/infrastructure-live-root-template), is where your platform team manages the foundational elements that every other AWS account depends on, and where your Account Factory workflow lives. | ||
|
|
||
| This repository is the only repository with access to the management account, and is granted access to all AWS accounts for your platform team to provision infrastructure in as necessary. This is also where your platform team can provision new AWS accounts with consistent baselines whenever teams need them. You'll also manage critical organization-wide infrastructure like your AWS Landing Zone, central logging, and security services in this repository. |
There was a problem hiding this comment.
| This repository is the only repository with access to the management account, and is granted access to all AWS accounts for your platform team to provision infrastructure in as necessary. This is also where your platform team can provision new AWS accounts with consistent baselines whenever teams need them. You'll also manage critical organization-wide infrastructure like your AWS Landing Zone, central logging, and security services in this repository. | |
| This repository is the only repository with access to the AWS management account, and is granted access to all AWS accounts for your platform team to provision infrastructure in as necessary. This is also where your platform team can provision new AWS accounts with consistent baselines whenever teams need them. You'll also manage critical organization-wide infrastructure like your AWS Landing Zone, central logging, and security services in this repository. |
There was a problem hiding this comment.
...and is granted access to all AWS accounts...
Can you clarify what this means? Does it mean that that we recommend configuring an OIDC role for this repo that has access to all AWS accounts? Or something else?
|
|
||
| Access to this repository is intentionally restricted to your most trusted platform team members. Every other piece of infrastructure in your organization can trace back to the foundational services configured here. | ||
|
|
||
| ### `infrastructure-live-root` workflows |
There was a problem hiding this comment.
I guess we're using "workflows" in a GitHub Actions sense here? Can we use more general language to talk about GitLab as well? Or if you want to split that into a separate PR, at least make explicit that these are specific GitHub Actions workflows that will live in this repo.
|
|
||
| You can learn more about this in the ["Using the Account Factory Workflow" guide](/2.0/docs/accountfactory/guides/vend-aws-account). | ||
|
|
||
| ::: |
There was a problem hiding this comment.
It's worth clarifying what the end user workflow is:
- Launch the form
- Generate the JSON
- Pass it to the workflow
|
|
||
| ::: | ||
|
|
||
| - **Pipelines:** This is where Account Factory integrates with the Gruntwork Pipelines product to drive infrastructure changes via GitOps workflows. Every infrastructure change goes through a proper review process with pull requests, approvals, and controlled deployments. Your platform team gets the confidence of peer review while maintaining the ability to rapidly deploy critical infrastructure changes. |
There was a problem hiding this comment.
| - **Pipelines:** This is where Account Factory integrates with the Gruntwork Pipelines product to drive infrastructure changes via GitOps workflows. Every infrastructure change goes through a proper review process with pull requests, approvals, and controlled deployments. Your platform team gets the confidence of peer review while maintaining the ability to rapidly deploy critical infrastructure changes. | |
| - **Pipelines:** This is where Account Factory integrates with Gruntwork Pipelines to drive infrastructure changes via GitOps workflows. With Gruntwork Pipelines, every infrastructure change goes through a proper review process with pull requests, approvals, and controlled deployments. Your platform team gets the confidence of peer review while maintaining the ability to rapidly deploy critical infrastructure changes. |
|
|
||
| Unlike the infrastructure-live-root repository, this repository focuses on managing access control rather than defining infrastructure. You might grant write access to a broader group for managing access while maintaining tight control over the main branch. Encourage collaboration between platform teams and application engineers to review and refine access control continuously. | ||
|
|
||
| ## Token Strategy |
There was a problem hiding this comment.
Should we call this "CI/CD Token Strategy"?
|
|
||
| Refer to [Configuring OpenId Connect in Amazon Web Services](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services) for additional details. | ||
|
|
||
| ### Roles provisioned by DevOps Foundations |
There was a problem hiding this comment.
We've moved away from the "DevOps Foundations" term.
|
|
||
| ## Executor | ||
|
|
||
| The executor receives a pipeline action and an infrastructure change as input and executes the specified action on the change. For example, when responding to `AccountsAdded` events on merge, the executor may create a follow-up pull request in the `infrastructure-live-root` repository to include additional IaC code for baselining the newly added accounts. |
There was a problem hiding this comment.
It'd be helpful to clarify where these things run. For example, in the first paragraph, we say "The orchestrator determines..." But I think many users will wonder: "What is this orchestrator? Where does it run?"
Perhaps you could clarify that you define a core GitHub Actions workflow and then that calls other GitHub Actions workflows defined by Gruntwork which call a Go binary that we call the Pipelines binary. We could then further clarify that the orchestrator runs in a GitHub Actions workflow and calls the binary to figure out what to do.
There was a problem hiding this comment.
Oh, I realize now that you do clarify this in the architecture index. Might be worth referencing that directly.
| ## CI/CD pipelines | ||
|
|
||
| ## `infrastructure-live-root` | ||
| Due to the way in which Pipelines is designed, customers run the binary as part of their CI/CD pipelines (e.g. GitHub Actions, GitLab CI, etc.). As such, Gruntwork provides Out of the Box CI/CD configurations for supported platforms when customers sign up for Gruntwork Pipelines in addition to [Gruntwork Account Factory](https://docs.gruntwork.io/account-factory/overview). |
There was a problem hiding this comment.
| Due to the way in which Pipelines is designed, customers run the binary as part of their CI/CD pipelines (e.g. GitHub Actions, GitLab CI, etc.). As such, Gruntwork provides Out of the Box CI/CD configurations for supported platforms when customers sign up for Gruntwork Pipelines in addition to [Gruntwork Account Factory](https://docs.gruntwork.io/account-factory/overview). | |
| By design, customers run the binary as part of their CI/CD pipelines (e.g. GitHub Actions, GitLab CI, etc.). As such, Gruntwork provides Out of the Box CI/CD configurations for supported platforms when customers sign up for Gruntwork Pipelines in addition to [Gruntwork Account Factory](https://docs.gruntwork.io/account-factory/overview). |
| class GDP,GMU,GLMU gruntwork | ||
| class Binary runtime | ||
| class OIDC,Cloud cloud | ||
| ``` |
There was a problem hiding this comment.
Whoah, these diagrams are great! They clarify so much that I wonder if we should put them first, rather than at the bottom?
Breaking down Pipelines architecture concepts so that we aren't assuming AWS only.