ref(spans): Pin redis sharding to partition index #93209
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
Scope: Backend
🔍 Existing Issues For ReviewYour pull request is modifying functions with the following pre-existing issues: 📄 File: src/sentry/spans/consumers/process/factory.py
Did you find this useful? React with a 👍 or 👎 |
untitaker
reviewed
Jun 10, 2025
Codecov ReportAttention: Patch coverage is ✅ All tests successful. No failed tests found.
Additional details and impacted files@@ Coverage Diff @@
## master #93209 +/- ##
==========================================
+ Coverage 85.31% 87.97% +2.65%
==========================================
Files 10256 10256
Lines 591639 591759 +120
Branches 22989 22986 -3
==========================================
+ Hits 504768 520593 +15825
+ Misses 86380 70675 -15705
Partials 491 491 |
github-actions
bot
added
the
Scope: Frontend
|
🚨 Warning: This pull request contains Frontend and Backend changes! It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently. Have questions? Please ask in the |
jan-auer
force-pushed
the
ref/spans-buffer-pin-redis
branch
from
655f4cb to
33a752a
Compare
untitaker
approved these changes
Jun 10, 2025
|
PR reverted: 4a34dac |
getsentry-bot
added a commit
that referenced
this pull request
Jun 10, 2025
This reverts commit 7cee132. Co-authored-by: jan-auer <[email protected]>
jan-auer
added a commit
that referenced
this pull request
Jun 10, 2025
* master: Revert "ref(spans): Pin redis sharding to partition index (#93209)" chore: add DemoModeGuardMiddleware logging (#92926) feat(tempest): Dynamically fetch ip addresses (#93012) fix(demo-mode): cleanup demo-mode sync options (#92918) ref(sdk): Migrate to `sentry_sdk.get_global_scope` (#93020) ref(sdk): Migrate to `get_isolation_scope` (#93019) ref(spans): Pin redis sharding to partition index (#93209)
andrewshie-sentry
pushed a commit
that referenced
this pull request
Jun 19, 2025
We observe performance degradation of the `process-spans` consumer with an increasing number of Redis shards. By using the partition index instead of the trace ID as sharding key, we pin each partition to exactly one Redis shard, which should reduce this effect. The downside is that a single hot partition can no longer be spread across multiple Redis shards. At the current time, we operate 4-8 partitions per shard, so we do not expect that this becomes an issue in the short time.
andrewshie-sentry
pushed a commit
that referenced
this pull request
Jun 19, 2025
This reverts commit 7cee132. Co-authored-by: jan-auer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We observe performance degradation of the
process-spansconsumer with anincreasing number of Redis shards. By using the partition index instead of the
trace ID as sharding key, we pin each partition to exactly one Redis shard,
which should reduce this effect.
The downside is that a single hot partition can no longer be spread across
multiple Redis shards. At the current time, we operate 4-8 partitions per shard,
so we do not expect that this becomes an issue in the short time.
If this proves successful, we should follow up with proper cellularization of
the buffer consumer.
Ref VIEPF-45