Skip to content

Fallback to the blocks queryable if vertical sharding is enabled #6764

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
May 29, 2025
Merged

Fallback to the blocks queryable if vertical sharding is enabled #6764

merged 10 commits into from
May 29, 2025

Conversation

alanprot
Copy link
Member

@alanprot alanprot commented May 28, 2025 •
edited
Loading

What this PR does:

Introduces several improvements to the Parquet implementation:

  • Adds fallback to the blocks store queryable when vertical sharding is enabled, as this is currently unsupported in the Parquet queryable.
  • Sorts blocks by timestamp in the converter to prioritize recent blocks during conversion
  • Adds support for configuring maxRowGroupSize and flag to enabled/disableon-disk buffer in the converter.
  • Randomizes tenant order during conversion to improve fairness.
  • Pull latest parquet common

Which issue(s) this PR fixes:
Part of #6712

Checklist

  • Tests updated
  • [NA] Documentation added
  • [NA] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@alanprot alanprot marked this pull request as ready for review May 28, 2025 00:06
@dosubot dosubot bot added component/querier storage/blocks Blocks storage engine labels May 28, 2025
yeya24
yeya24 reviewed May 28, 2025
View reviewed changes
@alanprot alanprot merged commit 469ae45 into cortexproject:master May 29, 2025
17 checks passed
@alanprot alanprot deleted the fix-parquet branch May 29, 2025 01:39
@CharlieTLe CharlieTLe mentioned this pull request Jul 26, 2025
Closed
@CharlieTLe CharlieTLe mentioned this pull request Aug 10, 2025
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danielblando danielblando danielblando approved these changes

@yeya24 yeya24 yeya24 approved these changes

Assignees
No one assigned
Labels
component/querier size/L storage/blocks Blocks storage engine
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alanprot @danielblando @yeya24