Proxy ConfigCat config in caddy

[geropl]

Currently, we're using ConfigCat client libraries from a wide variety of environments. All those clients periodically go to https://configcat.com/ and fetch the config from there directly.

Now Team IDE plans to/already uses ConfigCat to rollout features of with different lifecycles then the rest of Gitpod (delivered via marketplace to clients with different upgrade strategies vs. single version of Gitpod delivered to SaaS/Self-Hosted).

We see multiple problems with this:

• number of requests: We're on a certain plan with ConfigCat, and are close to/already exceeding their "number of requests" limits. So far this was not an issue, because we had close control over all clients, and could tweak the polling behavior. This changes when we use ConfigCat from clients with longer lifecycles that are not under our control.
• scaling to more clients: so far, we taught all clients to tell Self-Hosted and SaaS apart. It would be nice to centralize this behavior
• access to ConfigCat API: so far we had to build ConfigCat keys into multiple docker images (depending on client and runtime).

The proposed solution:

• installation-specific ConfigCat proixy: create a path in the caddy-powered proxy component that delivers the config.json file
• if there is not ConfigCat API key configured (Self-Hosted), this returns an empty JSON object (for now)
• if there is a ConfigCat API key (SaaS): go to ConfigCat at a certain interval, fetch the config.json from there and cache it locally to respond to all requests
• point all clients to this installation specific path

@mustard-mh @easyCZ

[geropl]

BTW here's an example of a Caddy plugin in our codebase:

gitpod/components/proxy/plugins/headlesslogdownload/headless_log_download.go

Line 42 in 242cc63

func (m HeadlessLogDownload) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error {

[easyCZ]

The URL for the config is https://cdn-global.configcat.com/configuration-files/<PUBLIC_API_KEY>/config_v5.json

I believe the client library understands etag and 302 responses which should be used in case the config has not changed since the last etag (etag is used as a hash of the contents more-less)