mismatch between genesis state PUT and GET operations (dumpgenesis failure)

[holiman]

Wanted to check/repro #21881 , and found that something is wonky with custom genesis and/or custom datadirs:

[user@work ~]$ cat gen.json 
{
  "config": {
    "chainId": 12345,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip155Block": 0,
    "eip158Block": 0,
    "byzantiumBlock": 0,
    "constantinopleBlock": 0,
    "petersburgBlock": 0,
    "istanbulBlock": 0,
    "berlinBlock": 0,
    "clique": {
      "period": 5,
      "epoch": 30000
    }
  },
  "difficulty": "1",
  "gasLimit": "30000",
  "extradata": "0x00000000000000000000000000000000000000000000000000000000000000009Ccf0fC294d629719ecEF4570632Cd1FdEE02c770000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  "alloc": {
    "DE9aD14212772d071c65BE3E84c390EA462735B6": { "balance": "1606938044258990275541962092341162602522202993782792835301376" },
    "6f095002ad09006dda6c5fecdc4e58550caab34e": { "balance": "1606938044258990275541962092341162602522202993782792835301376" },
    "9Ccf0fC294d629719ecEF4570632Cd1FdEE02c77": { "balance": "1606938044258990275541962092341162602522202993782792835301376" }
  }
}
[user@work ~]$ geth --datadir=/tmp/foobar  init gen.json 
INFO [02-21|10:19:31.392] Maximum peer count                       ETH=50 LES=0 total=50
INFO [02-21|10:19:31.407] Set global gas cap                       cap=50,000,000
INFO [02-21|10:19:31.410] Using leveldb as the backing database 
INFO [02-21|10:19:31.410] Allocated cache and file handles         database=/tmp/foobar/geth/chaindata cache=16.00MiB handles=16
INFO [02-21|10:19:31.412] Using LevelDB as the backing database 
INFO [02-21|10:19:31.413] Opened ancient database                  database=/tmp/foobar/geth/chaindata/ancient/chain readonly=false
INFO [02-21|10:19:31.413] Writing custom genesis block 
INFO [02-21|10:19:31.413] Persisted trie from memory database      nodes=4 size=639.00B time="29.427µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [02-21|10:19:31.413] Successfully wrote genesis state         database=chaindata                                hash=170f6a..b53b17
INFO [02-21|10:19:31.413] Using leveldb as the backing database 
INFO [02-21|10:19:31.413] Allocated cache and file handles         database=/tmp/foobar/geth/lightchaindata          cache=16.00MiB handles=16
INFO [02-21|10:19:31.414] Using LevelDB as the backing database 
INFO [02-21|10:19:31.414] Opened ancient database                  database=/tmp/foobar/geth/lightchaindata/ancient/chain readonly=false
INFO [02-21|10:19:31.414] Writing custom genesis block 
INFO [02-21|10:19:31.414] Persisted trie from memory database      nodes=4 size=639.00B time="23.348µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [02-21|10:19:31.415] Successfully wrote genesis state         database=lightchaindata                           hash=170f6a..b53b17
[user@work ~]$ geth --datadir=/tmp/foobar  dumpgenesis
INFO [02-21|10:19:36.009] Maximum peer count                       ETH=50 LES=0 total=50
INFO [02-21|10:19:36.019] Set global gas cap                       cap=50,000,000
INFO [02-21|10:19:36.022] Using leveldb as the backing database 
INFO [02-21|10:19:36.022] Allocated cache and file handles         database=/tmp/foobar/geth/chaindata cache=16.00MiB handles=16 readonly=true
INFO [02-21|10:19:36.026] Using LevelDB as the backing database 
Fatal: failed to read genesis: genesis state missing from db

[holiman]

Ah, so this is a bit weird. We write the geneiss spec here: https://github.com/ethereum/go-ethereum/blob/master/core/genesis.go#L169

	root, err := statedb.Commit(false)
	if err != nil {
		return err
	}
	// Commit newly generated states into disk if it's not empty.
	if root != types.EmptyRootHash {
		if err := triedb.Commit(root, true); err != nil {
			return err
		}
	}
	// Marshal the genesis state specification and persist.
	blob, err := json.Marshal(ga)
	if err != nil {
		return err
	}
	rawdb.WriteGenesisStateSpec(db, root, blob)

That is, the we use the trie root as key for the genesis state.

But dumpgenesis doesn't use the trie root has, but the block hash

func ReadGenesis(db ethdb.Database) (*Genesis, error) {
	var genesis Genesis
	stored := rawdb.ReadCanonicalHash(db, 0)
	if (stored == common.Hash{}) {
		return nil, fmt.Errorf("invalid genesis hash in database: %x", stored)
	}
	blob := rawdb.ReadGenesisStateSpec(db, stored)

And, even worse, in blockchain, under certain conditions, we try to look up the genesis state, by blockhash, not root hash:

					if beyondRoot || newHeadBlock.NumberU64() == 0 {
						if newHeadBlock.NumberU64() == 0 {
							// Recommit the genesis state into disk in case the rewinding destination
							// is genesis block and the relevant state is gone. In the future this
							// rewinding destination can be the earliest block stored in the chain
							// if the historical chain pruning is enabled. In that case the logic
							// needs to be improved here.
							if !bc.HasState(bc.genesisBlock.Root()) {
								if err := CommitGenesisState(bc.db, bc.triedb, bc.genesisBlock.Hash()); err != nil {
									log.Crit("Failed to commit genesis state", "err", err)
								}

[holiman]

We need to decide what key to use (blockhash or root hash), and be consistent about it.

[holiman]

Bug introduced here: https://github.com/ethereum/go-ethereum/pull/25473/files#diff-1272116434e5549f278a940767af52ed2d1abc91c44fa04ec48d8835a3774036R132

Affects v1.11.1 v1.11.0 v1.10.26 v1.10.25 v1.10.24 v1.10.23 v1.10.22