Add dynamic throughput configuration of writes on DynamoDB tables #318
Description
Right now user needs to deduct how much throughput user needs for Index storage using these static flags in table-manager:
f.Int64Var(&cfg.ProvisionedWriteThroughput, "dynamodb.periodic-table.write-throughput", 3000, "DynamoDB periodic tables write throughput")
f.Int64Var(&cfg.ProvisionedReadThroughput, "dynamodb.periodic-table.read-throughput", 300, "DynamoDB periodic tables read throughput")
f.Int64Var(&cfg.InactiveWriteThroughput, "dynamodb.periodic-table.inactive-write-throughput", 1, "DynamoDB periodic tables write throughput for inactive tables.")
f.Int64Var(&cfg.InactiveReadThroughput, "dynamodb.periodic-table.inactive-read-throughput", 300, "DynamoDB periodic tables read throughput for inactive tables")
Would be awesome to keep these flags to configure maximum read/write throughput and just update the active table dynamically with correct throughput based on some logic e.g:
- for the first iteration, it could be just naive dynamic table updates based on number of ingested samples for writes.
- for read I don't have an easy solution in mind, despite machine learning. But writes are the most important for now, we don't care so much about query latencies, so we can keep read throughput low, right?
That would save some costs spent on Dynamo, especially for non deterministic traffic on our clusters.