-
Notifications
You must be signed in to change notification settings - Fork 281
Fixed CosmosSqlMetadataProvider concurrency issue. #2695
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes a concurrency issue in CosmosSqlMetadataProvider
by replacing a non-thread-safe dictionary with a ConcurrentDictionary
and adding argument null checks to its partition-key methods.
- Replaced
Dictionary<string, string>
withConcurrentDictionary<string, string>
- Added
ArgumentNullException.ThrowIfNull
guards inGetPartitionKeyPath
andSetPartitionKeyPath
- Switched from
TryAdd/assignment
to a singleAddOrUpdate
call inSetPartitionKeyPath
Comments suppressed due to low confidence (4)
src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:554
- Add unit tests that simulate concurrent calls to
SetPartitionKeyPath
andGetPartitionKeyPath
to verify the thread-safe behavior of the newConcurrentDictionary
implementation.
_partitionKeyPaths.AddOrUpdate("${database}/{container}", partitionKeyPath, (key, oldValue) => partitionKeyPath);
src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:554
- [nitpick] The key format
"${database}/{container}"
is used in multiple places; consider extracting it into a private helper method to avoid duplication and reduce the risk of mismatches.
_partitionKeyPaths.AddOrUpdate("${database}/{container}", partitionKeyPath, (key, oldValue) => partitionKeyPath);
src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:539
- Update the XML doc comments for
GetPartitionKeyPath
andSetPartitionKeyPath
to mention that they throwArgumentNullException
for null inputs, so consumers know about these new exceptions.
public string? GetPartitionKeyPath(string database, string container)
src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:540
- Introducing
ArgumentNullException
here changes the behavior ofGetPartitionKeyPath
(it now throws instead of returning null for null inputs). Confirm this breaking change is intended or document it in the public API.
ArgumentNullException.ThrowIfNull(database);
Hello @sajeetharan , I appreciate the team have lots of other work in progress already, but this fix looks pretty much good to go and should be a quick job to review thanks to the hard work of @michaelstaib. The problems this causes for us operationally are severe. We would be most grateful if you could bump this up the priority list and get it into a release very soon. Thank you! |
@michaelstaib Thanks @golfalot for sure , will track it with next release cc : @Aniruddh25 @sourabh1007 @JerryNixon |
/azp run |
Azure Pipelines successfully started running 6 pipeline(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thank you for identifying and fixing this!
/azp run |
Azure Pipelines successfully started running 6 pipeline(s). |
The CosmosSqlMetdataProvider uses a standard dictionary although the metadata provider is accessed and updated concurrently. This causes the issue in #2694. I have updated the implementation to use a ConcurrentDictionary. Fixes #2694 Co-authored-by: Aniruddh Munde <[email protected]>
The CosmosSqlMetdataProvider uses a standard dictionary although the metadata provider is accessed and updated concurrently. This causes the issue in #2694. I have updated the implementation to use a ConcurrentDictionary. Fixes #2694 Co-authored-by: Aniruddh Munde <[email protected]>
## Why make this change? This change is made in order to add all of the commits for milestone 1.5 into its respective branch. ## What is this change? This change cherry-picks all of the commits that were added after the first release candidate. Cherry-picked commits: - #2648 #2657 #2617 #2659 #2655 #2633 #2667 #2673 #2650 #2695 #2702 #2688 ## How was this tested? - [ ] Integration Tests - [ ] Unit Tests ## Sample Request(s) --------- Co-authored-by: Sezal Chug <[email protected]> Co-authored-by: sezalchug <[email protected]> Co-authored-by: Tommaso Stocchi <[email protected]> Co-authored-by: Aaron Powell <[email protected]> Co-authored-by: aaronburtle <[email protected]> Co-authored-by: Aniruddh Munde <[email protected]> Co-authored-by: Jerry Nixon <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Michael Staib <[email protected]> Co-authored-by: souvikghosh04 <[email protected]> Co-authored-by: Souvik Ghosh <[email protected]>
The CosmosSqlMetdataProvider uses a standard dictionary although the metadata provider is accessed and updated concurrently. This causes the issue in #2694. I have updated the implementation to use a ConcurrentDictionary.
Fixes #2694