Retention-based Partition Dropping

[jcjones]

Determine which partitions are wholly outside of the configured retention_period for a given table and emit commands to drop them.

I'd just like to write for posterity that this is getting real hacky, even more so now with this branch rebased on top of the last year's worth of fixes.

To give just a hint of the deltas of the current version vs. what it was designed to do:

Partman gathers the Partition Table of each table, and attaches to each Partition the beginning and end positions, and then knowledge of how to derive the beginning and end timestamps. However, we know now that because MariaDB selects Partitions based on an incoming row matching any part of a partition, the partition offsets in the partition table are just guideposts. In reality, one must SELECT ... FROM table PARTITION(partition_name) ... to have actual truth of what the positions of the beginning and end of the partition. Deriving it from the table is dangerous, because it works until it doesn't, and that's where #64 comes from.

Partman also then assumed in the beginning that getting the start and end timestamps for a partition could come from the partition names, which leads to all these ALTER statements to keep the partition names roughly true. But that's hard to keep up with, and if we have a narrow set of partitions it ends up with giving very few data points from which to calculate a rate of change. But all the methods in Partman to calculate this stuff does so from the perspective of "calculate the rate of change of these partitions", but in reality we shouldn't care about the partitions at all. We should instead use the primary key, select the min, max, and then sample points between the two to get a better rate of change. Basically, we assumed in the beginning we could do this without earliest_utc_timestamp_query, but now that's basically mandatory. And we should use that.

And these decisions are baked into the myriad Partition types in types.py, which are all far too complicated.

Anyway, this thing needs a rewrite to become much simpler.

[kruti-s]

I added some suggestions about potentially overlooked errors and test cases