changeToUseChangeFeedToFetchPkRanges #46810
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR enhances the partition key range fetching mechanism by switching from regular queries to change feed queries, improving diagnostics tracking, and modifying retry policy application.
Key changes:
- Switch to change feed approach for fetching partition key ranges to enable incremental updates
- Fix incorrect end time tracking in CosmosDiagnostics for PARTITION_KEY_RANGE_LOOK_UP operations
- Apply throttling retry policy per page instead of across all pages
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| InMemoryCollectionRoutingMap.java | Added change feed continuation token support and logging |
| CollectionRoutingMap.java | Extended interface to support change feed continuation tokens |
| RxPartitionKeyRangeCache.java | Refactored to use change feed for partition key range fetching with proper diagnostics timing |
| RxDocumentClientImpl.java | Modified to support change feed headers and per-request retry policy |
| Paginator.java | Added change feed flag parameter |
| DocumentProducer.java | Updated method call to include change feed flag |
| DefaultDocumentQueryExecutionContext.java | Updated method call to include change feed flag |
| Multiple test files | Updated method signatures and test data to accommodate new parameters |
...smos/src/main/java/com/azure/cosmos/implementation/routing/InMemoryCollectionRoutingMap.java
Outdated
Show resolved
Hide resolved
...re-cosmos/src/main/java/com/azure/cosmos/implementation/caches/RxPartitionKeyRangeCache.java
Show resolved
Hide resolved
...re-cosmos/src/main/java/com/azure/cosmos/implementation/caches/RxPartitionKeyRangeCache.java
Show resolved
Hide resolved
...re-cosmos/src/main/java/com/azure/cosmos/implementation/caches/RxPartitionKeyRangeCache.java
Show resolved
Hide resolved
...re-cosmos/src/main/java/com/azure/cosmos/implementation/caches/RxPartitionKeyRangeCache.java
Outdated
Show resolved
Hide resolved
…ntation/routing/InMemoryCollectionRoutingMap.java Co-authored-by: Copilot <[email protected]>
…ntation/caches/RxPartitionKeyRangeCache.java Co-authored-by: Copilot <[email protected]>
xinlian12
force-pushed
the
useChangeFeedToGetPkRanges
branch
from
e2eaac0 to
ee9cba9
Compare
|
/azp run java - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run java - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| Iterable<ImmutablePair<PartitionKeyRange, IServerIdentity>> ranges, String collectionUniqueId) { | ||
| Iterable<ImmutablePair<PartitionKeyRange, IServerIdentity>> ranges, | ||
| String collectionUniqueId, | ||
| String changeFeedNextIfNoneMatch) { |
There was a problem hiding this comment.
I don't get this - worth going throguh in a call. For PKRanges map we have to guarantee that it is always a complete set - how could this variable ever be false? We would have to always drain all changes and only try to update the map at a piint when no more changes are left? Otherwise we could corrupt the map?
There was a problem hiding this comment.
but this method is only called when we get all changes from backend
| DocumentClientRetryPolicy retryPolicy = this.resetSessionTokenRetryPolicy.getRequestPolicy(null); | ||
| retryPolicy.onBeforeSendRequest(request); | ||
| return ObservableHelper.inlineIfPossibleAsObs( | ||
| () -> readFeed(request) |
There was a problem hiding this comment.
q: what is the reason ChangeFeedFetcher isn't used?
There was a problem hiding this comment.
changeFeedFetcher is used for document fetches, it has too many document related logics, so I think a separate one for metadata will be much better
|
/azp run java - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Changes included: