[installer]: create readme

Author: Simon Emms

installer/README.md

@@ -0,0 +1,297 @@
+# Installer
+
+The best way to get started with Gitpod self-hosted
+
+> This guide assumes that a compatible infrastructure is in place. Please
+> refer to the [Gitpod docs](https://www.gitpod.io/docs/self-hosted/latest/installation)
+> for instructions on how to create the infrastructure.
+
+# Key Concepts
+
+The Installer is a complete replacement for our Helm charts. Over time,
+this had grown to be too complex to effectively support and was a barrier
+to entry for new users - the base config was many hundreds of lines long
+and did not have effective validation on it. By contrast, the Installer's
+config is < 50 lines long and can be fully validated before running.
+
+Conceptually, the process is now:
+1. generate a base config
+2. amend the config for your own use-case
+3. validate
+4. render the Kubernetes YAML
+5. `kubectl apply`
+
+# Quickstart
+
+## Create a gitpod-installer script
+
+```shell
+#!/bin/bash
+
+VERSION="${VERSION:-main.1844}" # Build ID taken from https://werft.gitpod-dev.com/
+
+docker run -it --rm \
+  -v $PWD:/gitpod \
+  -v $HOME/.kube:/root/.kube \
+  eu.gcr.io/gitpod-core-dev/build/installer:${VERSION} \
+  $@
+```
+
+Save this in your `$PATH` as `gitpod-installer` and make executable
+
+## Generate the base config
+
+```shell
+gitpod-installer init > config.yaml
+```
+
+## Validate
+
+```shell
+# Checks the validity of the configuration YAML
+gitpod-installer validate config --config /gitpod/config.yaml
+
+# Checks that your cluster is ready to install Gitpod
+gitpod-installer validate cluster --kubeconfig /root/.kube/config --config /gitpod/config.yaml
+```
+
+Any errors here must be fixed before deploying. See [Cluster Dependencies](#cluster-dependencies)
+and [Config](#config) for more details.
+
+## Render the YAML
+
+```shell
+gitpod-installer render --config /gitpod/config.yaml > gitpod.yaml
+```
+
+## Deploy
+
+```shell
+kubectl apply -f gitpod.yaml
+```
+
+After a few minutes, your Gitpod installation will be available on the
+specified `domain`.
+
+# What is installed
+
+- All Gitpod components
+- Container registry*
+- MySQL database*
+- Jaeger operator*
+- RabbitMQ
+- Minio object storage*
+
+\* By default, these dependencies are installed if the `inCluster` setting
+is `true`. External dependencies can be used in their place
+
+# Config
+
+> Not every parameter is discussed in this table, just ones that are likely
+> to need changing. The full config structure is available in [config.go](/installer/pkg/config/v1/config.go).
+
+| Property | Required | Description | Notes |
+| --- | --- | --- | --- |
+| `domain` | Y | The domain to deploy to | This will need to be changed on every deployment |
+| `kind` | Y | Installation type to run - for most users, this will be `Full` | Can be `Full`, `Meta` or `Workspace` |
+| `metadata.region` | Y | Location for your `objectStorage` provider | If using Minio, set to `local` |
+| `workspace.runtime.containerdRuntimeDir` | Y | The location of containerd on host machine | Common values are: <ul><li>`/run/containerd/io.containerd.runtime.v2.task/k8s.io` (K3s)</li><li>`/var/lib/containerd/io.containerd.runtime.v2.task/k8s.io` (AWS/GCP)</li><li>`/run/containerd/io.containerd.runtime.v1.linux/k8s.io` (Azure)</li><li>`/run/containerd/io.containerd.runtime.v1.linux/moby`</li></ul> |
+| `workspace.runtime.containerdSocket` | Y | The location of containerd socket on the host machine |
+| `workspace.runtime.fsShiftMethod` | Y | File system | Can be either `fuse` (fuse-overlayfs) or `shiftfs`. This depending upon your host OS/distribution. |
+
+## Auth Providers
+
+Gitpod must be connected to a Git provider. This can be done via the
+dashboard on first load, or by providing `authProviders` configuration.
+
+```yaml
+authProviders:
+  - id: Public-GitHub
+    host: github.com
+    type: GitHub
+    oauth:
+      clientId: xxx
+      clientSecret: xxx
+      callBackUrl: https://$DOMAIN/auth/github.com/callback
+      settingsUrl: xxx
+```
+
+## In-cluster vs External Dependencies
+
+Gitpod requires certain services for it to function correctly. The Installer
+provides all of these in-cluster, but they can be configured to use services
+external to the cluster.
+
+To use the in-cluster dependency, set `inCluster` to be `true`.
+
+## Container Registry
+
+```yaml
+containerRegistry:
+  inCluster: false
+  external:
+    url: <url of registry>
+    certificate:
+      kind: secret
+      name: container-registry-token
+```
+
+The `container-registry-token` secret must contain a `.dockerconfigjson`
+key - this can be created by using the `kubectl create secret docker-registry`
+[command](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-secret-by-providing-credentials-on-the-command-line).
+
+## Database
+
+Gitpod requires an instance of MySQL 5.7 for data storage.
+
+The default encryption keys are `[{"name":"general","version":1,"primary":true,"material":"4uGh1q8y2DYryJwrVMHs0kWXJlqvHWWt/KJuNi04edI="}]`
+
+### Google Cloud SQL Proxy
+
+If using a GCP SQL instance, a [Cloud SQL Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy)
+connection can be used.
+
+```yaml
+database:
+  inCluster: false
+  cloudSQL:
+    instance: <PROJECT_ID>:<REGION>:<INSTANCE>
+    serviceAccount:
+      kind: secret
+      name: cloudsql-token
+```
+
+The `cloudsql-token` secret must contain the following key/value pairs:
+ - `credentials.json` - GCP Service Account key with `roles/cloudsql.client` role
+ - `encryptionKeys` - database encryption key. Use default value as above if unsure
+ - `password` - database password
+ - `username` - database username
+
+### External Database
+
+For all other connections, use an external database configuration.
+
+```yaml
+database:
+  inCluster: false
+  external:
+    certificate:
+      kind: secret
+      name: database-token
+```
+
+The `database-token` secret must contain the following key/value pairs:
+- `encryptionKeys` - database encryption key. Use default value as above if unsure
+- `host` - IP or URL of the database
+- `password` - database password
+- `port` - database port, usually `3306`
+- `username` - database username
+
+## Object Storage
+
+Gitpod supports the following object storage providers:
+
+### GCP
+
+```yaml
+objectStorage:
+  inCluster: false
+  cloudStorage:
+    project: <PROJECT_ID>
+    serviceAccount:
+      kind: secret
+      name: gcp-storage-token
+```
+
+The `gcp-storage-token` secret must contain the following key/value pairs:
+- `service-account.json` - GCP Service Account key with `roles/storage.admin` and `roles/storage.objectAdmin` roles
+
+### Azure
+
+> Azure Blob Storage is not S3 compatible. This uses Minio Gateway to
+> provide an S3 interface
+
+```yaml
+objectStorage:
+  inCluster: true
+  azure:
+    certificate:
+      kind: secret
+      name: az-storage-token
+```
+
+The `az-storage-token` secret must contain the following key/value pairs:
+- `accountName` - the globally-unique storage account name
+- `accountKey` - access key for the storage account
+
+### S3
+
+> Documentation coming soon [#6772](https://github.com/gitpod-io/gitpod/issues/6772)
+
+# Cluster Dependencies
+
+In order for the deployment to work successfully, there are certain
+dependencies that need to be installed.
+
+## Kernel and Runtime
+
+Your Kubernetes nodes must have the Linux kernel v5.4.0 or above and
+have a containerd runtime.
+
+## Affinity Labels
+
+Your Kubernetes nodes must have the following labels applied to them:
+- `gitpod.io/workload_meta`
+- `gitpod.io/workload_ide`
+- `gitpod.io/workload_workspace_services`
+- `gitpod.io/workload_workspace_regular`
+- `gitpod.io/workload_workspace_headless`
+
+It is recommended to have a minimum of two node pools, grouping the `meta`
+and `ide` nodes together and the `workspace` nodes together.
+
+## TLS certificates
+
+It is a requirement that a certificate secret exists, named as per
+`certificate.name` in your config YAML with `tls.crt` and `tls.key`
+in the secret data. How this certificate is generated is entirely your
+choice - we suggest [Cert Manager](https://cert-manager.io/) for
+simplicity, however any certificate authority can be used by creating a
+Kubernetes secret.
+
+The certificate must be associated with the following domains (where
+`$DOMAIN` is the value in config `domain`):
+ - `$DOMAIN`
+ - `*.$DOMAIN`
+ - `*.ws.$DOMAIN`
+
+### Cert Manager
+
+Cert Manager **MUST** be installed to your cluster. In order to secure
+communication between the various components, the application creates
+internally which are created using the Cert Manager `Certificate` and
+`Issuer` Custom Resource Definitions.
+
+```shell
+helm repo add jetstack https://charts.jetstack.io
+helm repo update
+helm upgrade \
+    --atomic \
+    --cleanup-on-fail \
+    --create-namespace \
+    --install \
+    --namespace='cert-manager' \
+    --reset-values \
+    --set installCRDs=true \
+    --set 'extraArgs={--dns01-recursive-nameservers-only=true,--dns01-recursive-nameservers=8.8.8.8:53\,1.1.1.1:53}' \
+    --wait \
+    cert-manager \
+    jetstack/cert-manager
+```
+
+# Todo
+
+PRs/comments welcome
+
+- [ ] [Improve distribution of gitpod-installer binaries](https://github.com/gitpod-io/gitpod/issues/6766)
+- [ ] [Configure and document S3 object storage](https://github.com/gitpod-io/gitpod/issues/6772)