Cooldown-based eager marshaling

[jiahuif]

With #251, the marshaling of our OpenAPI spec is delayed until the first request, so that the spec is not re-marshaled every time a new CRD is added, which saves a lot of marshaling time. However, I've encountered the following frustration with our current implementation.

• The API server is idle after CRDs are added and before first request comes. Considering in a real-world scenario, the user usually start adding workload after a few minutes, there is usually plenty of time for the API server to "warm-up" before first request from the user.
• In a replicated/HA control plane, there are multiple instances of API server. The first request warms up only one of the instances. In other words, for N instances, the first N requests are "warm-up"s and slow.

This PR addresses the issues by adding a cooldown. If the spec is not updated within the cooldown, build the cache by marshaling the spec before the first request comes. This way, we can avoid having to frequently re-marshal the spec while keeping first requests fast.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jiahuif
To complete the pull request process, please assign sttts after the PR has been reviewed.
You can assign the PR to them by writing /assign @sttts in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jiahuif]

/assign @jpbetz @apelisse @DangerOnTheRanger

[apelisse]

I haven't read the code yet but thank you for the great explanation of the pull-request. I don't know if this is something we want for OpenAPI v2 since eventually we have to keep the feature enabled but we're hoping to decrease the number of users to 0, which means that the cache would NEVER have to be populated (that's the target goal). Now, we could do what you suggest for OpenAPI v3, but since it's split across group, we also don't know how often the groups are pulled vs refreshed. Do we have any specific data or reason to implement this?

[jpbetz]

cc @DangerOnTheRanger

This seems like a good idea (trying to proactively warm the cache so that the 1st request after an update is less likely to be high latency). I don't know exactly what the ideal cooldown time should be.

[k8s-ci-robot]

@jiahuif: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.