Poor Error Response for Max Query Length (Instant Queries)

[jtlisi]

When running an instant query with a length greater than the MaxQueryLength set in cortex the returned error is not helpful.

Error: Query failed after 5 retries.

When run on a range query an explicit message is returned showing the reason the query failed:

{
  "status": "error",
  "errorType": "execution",
  "error": "rpc error: code = Code(400) desc = invalid query, length > limit (789h0m0s > 720h0m0s)",
  "message": "rpc error: code = Code(400) desc = invalid query, length > limit (789h0m0s > 720h0m0s)"
}

Ideally the error should be the same for both range and instant queries.

[bboreham]

I hit the same problem after #1255 merged - underlying error was "rpc error: code = Code(413) desc = exceeded maximum number of series in a query"

It seems to me that an error in that category should not be retried, since there is no reason to suppose it will work better the next time.

[gouthamve]

Right, got to the bottom of this. https://github.com/cortexproject/cortex/blob/master/pkg/querier/ingester_streaming_queryable.go#L51-L53 this makes all the errors thrown by the ingesters, 500 errors cuz of this: https://github.com/prometheus/prometheus/blob/master/web/api/v1/api.go#L399-L400 which are then retried.

[rfratto]

@jtlisi Looks like this was fixed at some point. I ran rate(consul_http_request[40d]) against Cortex and both query and query_range returned the expected error:

{"status":"error","errorType":"execution","error":"rpc error: code = Code(400) desc = invalid query, length \u003e limit (961h0m0s \u003e 744h0m0s)"}

[jtlisi]

^