Skip to content

Idea: improve large-step queries by storing rollup values in series index #1796

@bboreham

Description

@bboreham

[I'm not entirely sure this works: it should be useful in cases like rate(foo[24h]) where the time in square brackets is much larger than the chunk length]

We have an index mapping series to chunks in schema v9 and above.
Each chunk entry has the series ID, the chunk end time and the chunk ID.
We could add to this the start time, sum, count and increase* of the value, then in many cases we can skip reading and decoding the chunk.

It requires that Prometheus pass down (a) that it is computing an aggregation such as rate and (b) that the span is much longer than the chunk length.
I think (a) is in SelectParams.Func; (b) may be in SelectParams.Step but I haven't confirmed.

  • after resets, i.e. if the value goes to zero we keep incrementing from the previous value.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions