Fix out of range #35051
Fix out of range #35051
Conversation
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
Assigning reviewers: R: @Abacn for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
| // Throwing a runtime exception to break out of the RetryManager and signal | ||
| // to the Beam runner that the bundle should be retried, which will then | ||
| // allow an upstream DoFn to create a new stream. | ||
| throw new RuntimeException( |
There was a problem hiding this comment.
If I understood correctly it does not fix the error, but processing and re-throw an clearer error. It still rely on runner retry. Is that correct?
There was a problem hiding this comment.
yes, it fails the entire bundle and track the error counts. I hope this could clean up the wrong stream. Maybe we could just RetryType.DONT_RETRY?
There was a problem hiding this comment.
Does the error mean that the data in this stream has already been finalized to BQ? Could there be any data missing or duplicated?
...atform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiFlushAndFinalizeDoFn.java
Outdated
Show resolved
Hide resolved
...atform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiFlushAndFinalizeDoFn.java
Outdated
Show resolved
Hide resolved
...atform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiFlushAndFinalizeDoFn.java
Outdated
Show resolved
Hide resolved
| // Throwing a runtime exception to break out of the RetryManager and signal | ||
| // to the Beam runner that the bundle should be retried, which will then | ||
| // allow an upstream DoFn to create a new stream. | ||
| throw new RuntimeException( |
There was a problem hiding this comment.
Does the error mean that the data in this stream has already been finalized to BQ? Could there be any data missing or duplicated?
Fixes #30177
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.