Versioning Tendermint Backup

[majecty]

Why we need versioning

We are using DB without versioning. If we change the structure of the saved data, we should migrate the old data with the new data format. We need a version number to check the data's structure.

Current database structure

There are six columns in our Database.

1. COL_STATE: saves state
2. COL_HEADERS: saves headers
3. COL_BODIES: saves bodies
4. COL_EXTRA: saves best block number, best proposal block number, tendermint backup, ...
5. COL_MEMPOOL: saves mempool items
6. COL_ERROR_HINT: saves error hint

Save the version number

I propose a versioning scheme. The scheme is saving the version as an integer in some key. For the Tendermint backup, save the data version number in the "tendermint-backup-version" key in the COL_EXTRA column. For the mempool backup, save the version in the "mempool-backup-version" key in the COL_MEMPOOL column.

Migration

We should implement migration code from version x to version x + 1. When CodeChain starts, it should check the save data's version. If the version is lower than the current version, it should run the migration code.

Pseudocode

function restore() {
  let version = get_saved_version();
  while version < current_version {
    migrate_db(from: version, to: version + 1)
    let version = get_saved_version();
  }

  //restore logic
}

[foriequal0]

RocksDB has a prefix iterator. If we make the key to be prefixed like version:tendermint-backup we should be able to query all old keys.

[majecty]

It's a good idea. How about adding version:xxx data in the COL_EXTRA column?

[majecty]

I found that mempool backup already uses a prefix like "item_." I'll use the version prefix as "version_" for consistency.

codechain/core/src/miner/backup.rs

Line 27 in e9bc48e

const PREFIX_ITEM: &[u8; PREFIX_SIZE] = b"item_";