Skip to content

Always turn on ingester query streaming and mark its flag as deprecated #5817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 18, 2024
Merged

Always turn on ingester query streaming and mark its flag as deprecated #5817

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1d10a03
always turn on ingester query streaming and mark its flag as deprecated
yeya24 Mar 17, 2024
e90e355
lint
yeya24 Mar 17, 2024
18b1754
fix lint
yeya24 Mar 18, 2024
3f87e88
update changelog
yeya24 Mar 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
* [CHANGE] Ingester: Disable uploading compacted blocks and overlapping compaction in ingester. #5735
* [CHANGE] Distributor: Count the number of rate-limited samples in `distributor_samples_in_total`. #5714
* [CHANGE] Ruler: Remove `cortex_ruler_write_requests_total`, `cortex_ruler_write_requests_failed_total`, `cortex_ruler_queries_total`, `cortex_ruler_queries_failed_total`, and `cortex_ruler_query_seconds_total` metrics for the tenant when the ruler deletes the manager for the tenant. #5772
* [CHANGE] Querier: Mark `-querier.ingester-streaming` flag as deprecated. Now query ingester streaming is always enabled. #5817
* [FEATURE] Ingester: Add per-tenant new metric `cortex_ingester_tsdb_data_replay_duration_seconds`. #5477
* [FEATURE] Query Frontend/Scheduler: Add query priority support. #5605
* [FEATURE] Tracing: Add `kuberesolver` to resolve endpoints address with `kubernetes://` prefix as Kubernetes service. #5731
Expand Down
4 changes: 0 additions & 4 deletions docs/blocks-storage/querier.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,6 @@ querier:
# CLI flag: -querier.batch-iterators
[batch_iterators: <boolean> | default = true]

# Use streaming RPCs to query ingester.
# CLI flag: -querier.ingester-streaming
[ingester_streaming: <boolean> | default = true]

# Use streaming RPCs for metadata APIs from ingester.
# CLI flag: -querier.ingester-metadata-streaming
[ingester_metadata_streaming: <boolean> | default = false]
Expand Down
4 changes: 0 additions & 4 deletions docs/configuration/config-file-reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3596,10 +3596,6 @@ The `querier_config` configures the Cortex querier.
# CLI flag: -querier.batch-iterators
[batch_iterators: <boolean> | default = true]

# Use streaming RPCs to query ingester.
# CLI flag: -querier.ingester-streaming
[ingester_streaming: <boolean> | default = true]

# Use streaming RPCs for metadata APIs from ingester.
# CLI flag: -querier.ingester-metadata-streaming
[ingester_metadata_streaming: <boolean> | default = false]
Expand Down
191 changes: 79 additions & 112 deletions integration/querier_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,87 +27,70 @@ import (

func TestQuerierWithBlocksStorageRunningInMicroservicesMode(t *testing.T) {
tests := map[string]struct {
blocksShardingStrategy string // Empty means sharding is disabled.
tenantShardSize int
ingesterStreamingEnabled bool
indexCacheBackend string
chunkCacheBackend string
bucketIndexEnabled bool
blocksShardingStrategy string // Empty means sharding is disabled.
tenantShardSize int
indexCacheBackend string
chunkCacheBackend string
bucketIndexEnabled bool
}{
"blocks sharding disabled, ingester gRPC streaming disabled, memcached index cache": {
blocksShardingStrategy: "",
ingesterStreamingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
chunkCacheBackend: tsdb.CacheBackendMemcached,
"blocks sharding disabled, memcached index cache": {
blocksShardingStrategy: "",
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
chunkCacheBackend: tsdb.CacheBackendMemcached,
},
"blocks sharding disabled, ingester gRPC streaming disabled, multilevel index cache (inmemory, memcached)": {
blocksShardingStrategy: "",
ingesterStreamingEnabled: false,
indexCacheBackend: fmt.Sprintf("%v,%v", tsdb.IndexCacheBackendInMemory, tsdb.IndexCacheBackendMemcached),
chunkCacheBackend: tsdb.CacheBackendMemcached,
"blocks sharding disabled, multilevel index cache (inmemory, memcached)": {
blocksShardingStrategy: "",
indexCacheBackend: fmt.Sprintf("%v,%v", tsdb.IndexCacheBackendInMemory, tsdb.IndexCacheBackendMemcached),
chunkCacheBackend: tsdb.CacheBackendMemcached,
},
"blocks sharding disabled, ingester gRPC streaming disabled, redis index cache": {
blocksShardingStrategy: "",
ingesterStreamingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
chunkCacheBackend: tsdb.CacheBackendRedis,
"blocks sharding disabled, redis index cache": {
blocksShardingStrategy: "",
indexCacheBackend: tsdb.IndexCacheBackendRedis,
chunkCacheBackend: tsdb.CacheBackendRedis,
},
"blocks sharding disabled, ingester gRPC streaming disabled, multilevel index cache (inmemory, redis)": {
blocksShardingStrategy: "",
ingesterStreamingEnabled: false,
indexCacheBackend: fmt.Sprintf("%v,%v", tsdb.IndexCacheBackendInMemory, tsdb.IndexCacheBackendRedis),
chunkCacheBackend: tsdb.CacheBackendRedis,
"blocks sharding disabled, multilevel index cache (inmemory, redis)": {
blocksShardingStrategy: "",
indexCacheBackend: fmt.Sprintf("%v,%v", tsdb.IndexCacheBackendInMemory, tsdb.IndexCacheBackendRedis),
chunkCacheBackend: tsdb.CacheBackendRedis,
},
"blocks default sharding, ingester gRPC streaming disabled, inmemory index cache": {
blocksShardingStrategy: "default",
ingesterStreamingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
"blocks default sharding, inmemory index cache": {
blocksShardingStrategy: "default",
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
},
"blocks default sharding, ingester gRPC streaming enabled, inmemory index cache": {
blocksShardingStrategy: "default",
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
"blocks default sharding, memcached index cache": {
blocksShardingStrategy: "default",
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
chunkCacheBackend: tsdb.CacheBackendMemcached,
},
"blocks default sharding, ingester gRPC streaming enabled, memcached index cache": {
blocksShardingStrategy: "default",
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
chunkCacheBackend: tsdb.CacheBackendMemcached,
"blocks shuffle sharding, memcached index cache": {
blocksShardingStrategy: "shuffle-sharding",
tenantShardSize: 1,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
chunkCacheBackend: tsdb.CacheBackendMemcached,
},
"blocks shuffle sharding, ingester gRPC streaming enabled, memcached index cache": {
blocksShardingStrategy: "shuffle-sharding",
tenantShardSize: 1,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
chunkCacheBackend: tsdb.CacheBackendMemcached,
"blocks default sharding, inmemory index cache, bucket index enabled": {
blocksShardingStrategy: "default",
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
bucketIndexEnabled: true,
},
"blocks default sharding, ingester gRPC streaming enabled, inmemory index cache, bucket index enabled": {
blocksShardingStrategy: "default",
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
bucketIndexEnabled: true,
"blocks shuffle sharding, memcached index cache, bucket index enabled": {
blocksShardingStrategy: "shuffle-sharding",
tenantShardSize: 1,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
bucketIndexEnabled: true,
},
"blocks shuffle sharding, ingester gRPC streaming enabled, memcached index cache, bucket index enabled": {
blocksShardingStrategy: "shuffle-sharding",
tenantShardSize: 1,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
bucketIndexEnabled: true,
"blocks default sharding, redis index cache, bucket index enabled": {
blocksShardingStrategy: "default",
indexCacheBackend: tsdb.IndexCacheBackendRedis,
chunkCacheBackend: tsdb.CacheBackendRedis,
bucketIndexEnabled: true,
},
"blocks default sharding, ingester gRPC streaming enabled, redis index cache, bucket index enabled": {
blocksShardingStrategy: "default",
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
chunkCacheBackend: tsdb.CacheBackendRedis,
bucketIndexEnabled: true,
},
"blocks shuffle sharding, ingester gRPC streaming enabled, redis index cache, bucket index enabled": {
blocksShardingStrategy: "shuffle-sharding",
tenantShardSize: 1,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
chunkCacheBackend: tsdb.CacheBackendRedis,
bucketIndexEnabled: true,
"blocks shuffle sharding, redis index cache, bucket index enabled": {
blocksShardingStrategy: "shuffle-sharding",
tenantShardSize: 1,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
chunkCacheBackend: tsdb.CacheBackendRedis,
bucketIndexEnabled: true,
},
}

Expand All @@ -134,7 +117,6 @@ func TestQuerierWithBlocksStorageRunningInMicroservicesMode(t *testing.T) {
"-store-gateway.sharding-enabled": strconv.FormatBool(testCfg.blocksShardingStrategy != ""),
"-store-gateway.sharding-strategy": testCfg.blocksShardingStrategy,
"-store-gateway.tenant-shard-size": fmt.Sprintf("%d", testCfg.tenantShardSize),
"-querier.ingester-streaming": strconv.FormatBool(testCfg.ingesterStreamingEnabled),
"-querier.query-store-for-labels-enabled": "true",
"-querier.thanos-engine": strconv.FormatBool(thanosEngine),
"-blocks-storage.bucket-store.bucket-index.enabled": strconv.FormatBool(testCfg.bucketIndexEnabled),
Expand Down Expand Up @@ -319,52 +301,39 @@ func TestQuerierWithBlocksStorageRunningInMicroservicesMode(t *testing.T) {

func TestQuerierWithBlocksStorageRunningInSingleBinaryMode(t *testing.T) {
tests := map[string]struct {
blocksShardingEnabled bool
ingesterStreamingEnabled bool
indexCacheBackend string
bucketIndexEnabled bool
blocksShardingEnabled bool
indexCacheBackend string
bucketIndexEnabled bool
}{
"blocks sharding enabled, ingester gRPC streaming disabled, inmemory index cache": {
blocksShardingEnabled: true,
ingesterStreamingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
},
"blocks sharding enabled, ingester gRPC streaming enabled, inmemory index cache": {
blocksShardingEnabled: true,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
"blocks sharding enabled, inmemory index cache": {
blocksShardingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendInMemory,
},
"blocks sharding disabled, ingester gRPC streaming disabled, memcached index cache": {
blocksShardingEnabled: false,
ingesterStreamingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
"blocks sharding disabled, memcached index cache": {
blocksShardingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
},
"blocks sharding enabled, ingester gRPC streaming enabled, memcached index cache": {
blocksShardingEnabled: true,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
"blocks sharding enabled, memcached index cache": {
blocksShardingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
},
"blocks sharding enabled, ingester gRPC streaming enabled, memcached index cache, bucket index enabled": {
blocksShardingEnabled: true,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
bucketIndexEnabled: true,
"blocks sharding enabled, memcached index cache, bucket index enabled": {
blocksShardingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendMemcached,
bucketIndexEnabled: true,
},
"blocks sharding disabled, ingester gRPC streaming disabled, redis index cache": {
blocksShardingEnabled: false,
ingesterStreamingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
"blocks sharding disabled,redis index cache": {
blocksShardingEnabled: false,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
},
"blocks sharding enabled, ingester gRPC streaming enabled, redis index cache": {
blocksShardingEnabled: true,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
"blocks sharding enabled, redis index cache": {
blocksShardingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
},
"blocks sharding enabled, ingester gRPC streaming enabled, redis index cache, bucket index enabled": {
blocksShardingEnabled: true,
ingesterStreamingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
bucketIndexEnabled: true,
"blocks sharding enabled, redis index cache, bucket index enabled": {
blocksShardingEnabled: true,
indexCacheBackend: tsdb.IndexCacheBackendRedis,
bucketIndexEnabled: true,
},
}

Expand Down Expand Up @@ -398,7 +367,6 @@ func TestQuerierWithBlocksStorageRunningInSingleBinaryMode(t *testing.T) {
"-blocks-storage.tsdb.retention-period": ((blockRangePeriod * 2) - 1).String(),
"-blocks-storage.bucket-store.index-cache.backend": testCfg.indexCacheBackend,
"-blocks-storage.bucket-store.bucket-index.enabled": strconv.FormatBool(testCfg.bucketIndexEnabled),
"-querier.ingester-streaming": strconv.FormatBool(testCfg.ingesterStreamingEnabled),
"-querier.query-store-for-labels-enabled": "true",
"-querier.thanos-engine": strconv.FormatBool(thanosEngine),
// Ingester.
Expand Down Expand Up @@ -1041,7 +1009,6 @@ func TestQueryLimitsWithBlocksStorageRunningInMicroServices(t *testing.T) {
"-blocks-storage.tsdb.ship-interval": "1s",
"-blocks-storage.bucket-store.sync-interval": "1s",
"-blocks-storage.tsdb.retention-period": ((blockRangePeriod * 2) - 1).String(),
"-querier.ingester-streaming": "true",
"-querier.query-store-for-labels-enabled": "true",
"-querier.max-fetched-series-per-query": "3",
})
Expand Down
105 changes: 0 additions & 105 deletions pkg/chunk/chunk_test.go
View file
Open in desktop

This file was deleted.

Loading