Skip to content

Conversation

michaelstaib
Copy link
Collaborator

The CosmosSqlMetdataProvider uses a standard dictionary although the metadata provider is accessed and updated concurrently. This causes the issue in #2694. I have updated the implementation to use a ConcurrentDictionary.

Fixes #2694

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a concurrency issue in CosmosSqlMetadataProvider by replacing a non-thread-safe dictionary with a ConcurrentDictionary and adding argument null checks to its partition-key methods.

  • Replaced Dictionary<string, string> with ConcurrentDictionary<string, string>
  • Added ArgumentNullException.ThrowIfNull guards in GetPartitionKeyPath and SetPartitionKeyPath
  • Switched from TryAdd/assignment to a single AddOrUpdate call in SetPartitionKeyPath
Comments suppressed due to low confidence (4)

src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:554

  • Add unit tests that simulate concurrent calls to SetPartitionKeyPath and GetPartitionKeyPath to verify the thread-safe behavior of the new ConcurrentDictionary implementation.
_partitionKeyPaths.AddOrUpdate("${database}/{container}", partitionKeyPath, (key, oldValue) => partitionKeyPath);

src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:554

  • [nitpick] The key format "${database}/{container}" is used in multiple places; consider extracting it into a private helper method to avoid duplication and reduce the risk of mismatches.
_partitionKeyPaths.AddOrUpdate("${database}/{container}", partitionKeyPath, (key, oldValue) => partitionKeyPath);

src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:539

  • Update the XML doc comments for GetPartitionKeyPath and SetPartitionKeyPath to mention that they throw ArgumentNullException for null inputs, so consumers know about these new exceptions.
public string? GetPartitionKeyPath(string database, string container)

src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:540

  • Introducing ArgumentNullException here changes the behavior of GetPartitionKeyPath (it now throws instead of returning null for null inputs). Confirm this breaking change is intended or document it in the public API.
ArgumentNullException.ThrowIfNull(database);

@golfalot
Copy link

Hello @sajeetharan , I appreciate the team have lots of other work in progress already, but this fix looks pretty much good to go and should be a quick job to review thanks to the hard work of @michaelstaib.

The problems this causes for us operationally are severe. We would be most grateful if you could bump this up the priority list and get it into a release very soon. Thank you!

@sajeetharan
Copy link
Member

sajeetharan commented May 23, 2025

@michaelstaib Thanks @golfalot for sure , will track it with next release cc : @Aniruddh25 @sourabh1007 @JerryNixon

@Aniruddh25
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

Copy link
Collaborator

@Aniruddh25 Aniruddh25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for identifying and fixing this!

@Aniruddh25 Aniruddh25 added this to the 1.5 milestone May 28, 2025
@Aniruddh25
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@RubenCerna2079 RubenCerna2079 merged commit 70c48d7 into Azure:main May 28, 2025
11 checks passed
@Aniruddh25 Aniruddh25 mentioned this pull request May 29, 2025
2 tasks
RubenCerna2079 pushed a commit that referenced this pull request May 29, 2025
The CosmosSqlMetdataProvider uses a standard dictionary although the
metadata provider is accessed and updated concurrently. This causes the
issue in #2694. I have updated the implementation to use a
ConcurrentDictionary.

Fixes #2694

Co-authored-by: Aniruddh Munde <[email protected]>
RubenCerna2079 pushed a commit that referenced this pull request May 29, 2025
The CosmosSqlMetdataProvider uses a standard dictionary although the
metadata provider is accessed and updated concurrently. This causes the
issue in #2694. I have updated the implementation to use a
ConcurrentDictionary.

Fixes #2694

Co-authored-by: Aniruddh Munde <[email protected]>
aaronburtle added a commit that referenced this pull request May 29, 2025
## Why make this change?

This change is made in order to add all of the commits for milestone 1.5
into its respective branch.

## What is this change?

This change cherry-picks all of the commits that were added after the
first release candidate.
Cherry-picked commits:
 - #2648 
#2657
#2617 
#2659 
#2655 
#2633 
#2667 
#2673 
#2650 
#2695 
#2702 
#2688

## How was this tested?

- [ ] Integration Tests
- [ ] Unit Tests

## Sample Request(s)

---------

Co-authored-by: Sezal Chug <[email protected]>
Co-authored-by: sezalchug <[email protected]>
Co-authored-by: Tommaso Stocchi <[email protected]>
Co-authored-by: Aaron Powell <[email protected]>
Co-authored-by: aaronburtle <[email protected]>
Co-authored-by: Aniruddh Munde <[email protected]>
Co-authored-by: Jerry Nixon <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Michael Staib <[email protected]>
Co-authored-by: souvikghosh04 <[email protected]>
Co-authored-by: Souvik Ghosh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: unrecoverable The collection's state is no longer correct, some Entities return errors until DAB is restarted

6 participants