Skip to content

[release-4.19] CNTRLPLANE-1609: Backport StreamingCollectionEncoding for JSON and protobuf #2478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 8 commits into
base: release-4.19
Choose a base branch
from
Open

[release-4.19] CNTRLPLANE-1609: Backport StreamingCollectionEncoding for JSON and protobuf #2478

wants to merge 8 commits into from
+2,370 −34

Conversation

rh-roman
Copy link

@rh-roman rh-roman commented Sep 29, 2025 •
edited
Loading

Due to significant customer need for performance improvements related to API server pressure from large LIST requests, backporting the API streaming features that have been introduced in 1.33/1.34 related to kubernetes/enhancements#3157

@openshift-ci-robot openshift-ci-robot added the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Sep 29, 2025
@openshift-ci-robot
Copy link

@rh-roman: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@openshift-ci openshift-ci bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 29, 2025
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Sep 29, 2025

Hi @rh-roman. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci openshift-ci bot added the vendor-update Touching vendor dir or related files label Sep 29, 2025
@benluddy
Copy link

/ok-to-test

@openshift-ci openshift-ci bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 30, 2025
@openshift-ci-robot
Copy link

@rh-roman: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@benluddy
Copy link

benluddy commented Oct 2, 2025

/retitle [release-4.19] Backport StreamingCollectionEncoding for JSON and protobuf

@openshift-ci openshift-ci bot changed the title Backport StreamingCollectionEncoding for JSON and protobuf [release-4.19] Backport StreamingCollectionEncoding for JSON and protobuf Oct 2, 2025
@rh-roman
Copy link
Author

rh-roman commented Oct 2, 2025

/payload-job-with-prs periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview openshift/api#2517

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 2, 2025

@rh-roman: user rh-roman is not trusted for pull request #2478

@benluddy
Copy link

benluddy commented Oct 2, 2025

/payload-job-with-prs periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview openshift/api#2517

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 2, 2025

@benluddy: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@benluddy
Copy link

benluddy commented Oct 2, 2025

/payload-job-with-prs periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview openshift/api#2517

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 2, 2025

@benluddy: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/28fb66a0-9fca-11f0-9331-8f7b43dad041-0

@openshift-ci openshift-ci bot mentioned this pull request Oct 2, 2025
Open
@benluddy
Copy link

benluddy commented Oct 6, 2025

Out of curiosity: #2487

@rh-roman rh-roman force-pushed the backport-api-stream branch from 4cbb621 to 56688ad Compare October 8, 2025 19:14
@openshift-ci-robot
Copy link

@rh-roman: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

nkeert and others added 6 commits October 8, 2025 12:30
@nkeert @rh-roman
UPSTREAM: 130190: Add a test to validate deferredResponseWriteron mul…
e3767ef 
…tiple write calls

Signed-off-by: nkeert <[email protected]>
@z1cheng @rh-roman
UPSTREAM: 130281: Implement chunking for gzip encoder in deferredResp…
a2c4095 
…onseWriter

Signed-off-by: z1cheng <[email protected]>
@serathius @rh-roman
UPSTREAM: 130220: Add tests for encoding collections in JSON for KEP-…
805f25c 
…5116

Used test cases from:
* Original PR kubernetes#129334
* KEP https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/5116-streaming-response-encoding#unit-tests

For now testing current serializer implementation to show encoder
behavior and agree on set of tests. Having a separate PR should make review easier.
In separate PR will add the implementation for streaming that should
provide same response byte-to-byte.
@serathius @rh-roman
UPSTREAM: 129334: Streaming JSON encoder for List
c1f8b1f
@z1cheng @rh-roman
UPSTREAM: 130511: Implement tests for encoding collections in Proto
f9a4bd0 
Signed-off-by: z1cheng <[email protected]>
@serathius @rh-roman
UPSTREAM: 129407: Implement streaming proto encoding
f9d90d7
@rh-roman
UPSTREAM: <carry>: Use switch test to gofuzz
04467b1 
randfill isn't available in 4.19
@rh-roman rh-roman force-pushed the backport-api-stream branch from 56688ad to c40de39 Compare October 8, 2025 19:38
@openshift-ci-robot
Copy link

@rh-roman: the contents of this pull request could not be automatically validated.

The following commits are valid:

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

benluddy
benluddy reviewed Oct 8, 2025
View reviewed changes
@rh-roman
UPSTREAM: <carry>: Make streaming collection encoding off by default
cfbb772 
Require an explicit opt in for special handling of collection encoding.
@rh-roman rh-roman force-pushed the backport-api-stream branch from c40de39 to cfbb772 Compare October 8, 2025 19:58
@openshift-ci-robot
Copy link

@rh-roman: the contents of this pull request could not be automatically validated.

The following commits are valid:

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@benluddy
Copy link

benluddy commented Oct 8, 2025

/remove-label backports/unvalidated-commits
/approve

@openshift-ci openshift-ci bot removed the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Oct 8, 2025
@benluddy
Copy link

benluddy commented Oct 8, 2025 •
edited by openshift-ci bot
Loading

/retitle [release-4.19] CNTRLPLANE-1609: Backport StreamingCollectionEncoding for JSON and protobuf

@openshift-ci openshift-ci bot changed the title [release-4.19] Backport StreamingCollectionEncoding for JSON and protobuf [release-4.19] CNTRLPLANE-1609: Backport StreamingCollectionEncoding for JSON and protobuf Oct 8, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 8, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 8, 2025 •
edited by openshift-ci bot
Loading

@rh-roman: This pull request references CNTRLPLANE-1609 which is a valid jira issue.

In response to this:

Due to significant customer need for performance improvements related to API server pressure from large LIST requests, backporting the API streaming features that have been introduced in 1.33/1.34 related to kubernetes/enhancements#3157

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 8, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: benluddy, rh-roman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 8, 2025
@benluddy
Copy link

benluddy commented Oct 8, 2025

/label backport-risk-assessed

@openshift-ci openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Oct 8, 2025
@benluddy
Copy link

benluddy commented Oct 8, 2025

error: image "quay-proxy.ci.openshift.org/openshift/ci@sha256:57649f122962788acc752d82ea734e10c8fb750408969ff92d80a5d433d247aa" not found: manifest unknown: manifest unknown
Payload creation failure (attempt 5/5).

?

/retest-required

@benluddy
Copy link

benluddy commented Oct 9, 2025

/retest-required

@benluddy
Copy link

benluddy commented Oct 9, 2025

/test e2e-gcp

level=fatal msg=failed to fetch Cluster: failed to fetch dependency of "Cluster": failed to generate asset "Platform Quota Check": failed to load Quota for services: projects/XXXXXXXXXXXXXXXXXXXXXXXX/services/compute.googleapis.com, projects/XXXXXXXXXXXXXXXXXXXXXXXX/services/iam.googleapis.com: failed to load quota limits: googleapi: Error 503: The service is currently unavailable., backendError

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 9, 2025

@rh-roman: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bertinatto bertinatto Awaiting requested review from bertinatto

@deads2k deads2k Awaiting requested review from deads2k

1 more reviewer

@benluddy benluddy benluddy left review comments

Reviewers whose approvals may not affect merge requirements
Assignees

@kasturinarra kasturinarra

@zhouying7780 zhouying7780

@xingxingxia xingxingxia

@lyman9966 lyman9966

@duanwei33 duanwei33

@wangke19 wangke19

@gangwgr gangwgr

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. vendor-update Touching vendor dir or related files
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.