mandatory network upgrades #640

ceyonur · 2023-05-04T15:30:23Z

Why this should be merged

We should be able to specify network upgrades in the code, without expecting them to be in upgrade bytes.

Closes: #638

How this works

Removes network uprades from UpgradeConfig and move them to specifiable variables for networks (fuji + mainnet + local)

How this was tested

Locally tested

How is this documented

No need for documentation

richardpringle

LGTM
(not leaving a full review as I am not super familiar with the code)

…et-evm into mandatory-network-upgrades

aaronbuchwald · 2023-05-25T18:25:15Z

params/network_upgrades.go

 	return nil
 }
+
+func GetNetworkUpgrades(networkID uint32) NetworkUpgrades {


If we needed to add a network upgrade where the timestamp was set by each instance of Subnet-EVM as opposed to being hardcoded in Subnet-EVM for official Avalanche network-wide upgrades, how would we do that?

There are 2 different options:
1- Allow these timestamps to be overriden by genesis/external upgrades
2- Allow separate optional network upgrades.

There could be some problems allowing overrides for avalanche network-wide upgrades. Like AvalancheGo versions, a critical bug etc etc.

I can revert changes that removes specifiable network upgrades, and convert them to "Optional" network upgrades. In any case we should keep this mandatory network upgrade notion in the code, without expecting to be activated by genesis/upgrade files.

Added optional network upgrades

aaronbuchwald · 2023-06-27T02:16:06Z

plugin/evm/vm.go

+	// NetworkID of EVM is equal to the ChainID
+	// But this is different than Avalanche's NetworkID
+	// which represents the Avalanche network the EVM is running on
+	// like Fuji, Mainnet, Local, etc.


what's the reason for this comment?

the networkID is not necessarily the same as the chainID in the EVM, but some tooling incorrectly assumes it is and adds incorrect checks to verify this which caused us to follow this convention so that the tooling would work correctly 😅

Could we either remove this comment or clarify this point?

aaronbuchwald · 2023-06-27T02:18:23Z

plugin/evm/vm_test.go

+	// use modified genesis to set SubnetEVM timestamp to 1
+	genesisJsonModifiedSubnetEVM := strings.Replace(genesisJSONSubnetEVM, "\"subnetEVMTimestamp\":0", "\"subnetEVMTimestamp\":1", 1)


This seems like something we could easily break if we changed the string formatting slightly.

Could we alter this format to unmarshal the JSON, update the value, and re-marshal it to pass it in as we do in other places?

I see, so the expected behavior is that mandatory upgrades are overridden completely based on the Avalanche network enabled network upgrades.

nit: can we adopt the convention of using backtick (``) to quote strings containing "?

I see, so the expected behavior is that mandatory upgrades are overridden completely based on the Avalanche network enabled network upgrades.

True.

I changed the test.

aaronbuchwald · 2023-06-27T02:51:44Z

params/network_upgrades.go

+	return nil
+}

+func (m *MandatoryNetworkUpgrades) MandatoryForkOrder() []fork {


Can we un-export this, so that we do not have an exported function returning an internal type?

aaronbuchwald · 2023-06-27T02:52:16Z

params/network_upgrades.go

+func (n *OptionalNetworkUpgrades) OptionalForkOrder() []fork {
+	return []fork{}
+}


Same comment here on returning an internal type from an exported function

darioush · 2023-06-27T03:24:08Z

plugin/evm/vm_test.go

+	// use modified genesis to set SubnetEVM timestamp to 1
+	genesisJsonModifiedSubnetEVM := strings.Replace(genesisJSONSubnetEVM, "\"subnetEVMTimestamp\":0", "\"subnetEVMTimestamp\":1", 1)


nit: can we adopt the convention of using backtick (``) to quote strings containing "?

darioush · 2023-06-27T03:25:12Z

plugin/evm/vm_test.go

+
+	defer func() {
+		if err := vm.Shutdown(context.Background()); err != nil {
+			t.Fatal(err)


nit: can use require here.

I used this to keep the convention with other tests, but yea I think we can change it.

We should definitely modify these tests to use require. Created an issue #701

darioush · 2023-06-27T03:31:35Z

plugin/evm/vm_upgrade_bytes_test.go

 	assert.Equal(t, signedTx1.Hash(), txs[0].Hash())
 }

-func TestVMUpgradeBytesNetworkUpgrades(t *testing.T) {


Can we keep these tests and modify them to be with optional upgrades instead?
Since the optional upgrades should still function with upgrade bytes?

We don't currently have any actual upgrade to test out. I commented out a working case and added a TODO.

…s#640) * remove network upgrades from upgrade configs and enforce them in vm init * fix tests * remove unused var * remove extra shutdown * copy test chain config when modify * use SimulatedTestChainConfig for simulated tests * copy modified test chain config * add optional upgrades * Add tests * fix comments * use unmarshal in test * unexport functions * add commented out test * add TODO comment

* Rename test_ to step_ for clarity (ava-labs#636) * Codeowners fix (specific dir overrides global) (ava-labs#639) * rename state upgrade test (ava-labs#641) * plugin/evm: fix godoc typo (ava-labs#647) Signed-off-by: Gyuho Lee <[email protected]> * plugin/evm: remove unncessary tx.Hash op in map iter (ava-labs#648) Signed-off-by: Gyuho Lee <[email protected]> * deployExampleRewardManager.ts hardhat script fixed (ava-labs#650) * Migrate HardHat tests to DS-tests (Solidity tests) (ava-labs#601) * Increase trie dirty default cache size to 512MB (ava-labs#656) * export gingkgo suites (ava-labs#659) * export gingkgo suites * remove precompiles description from ping tests * update package.json * fix url in package json * change package name * Publish npm module for contract-examples (ava-labs#663) * export gingkgo suites * remove precompiles description from ping tests * update package.json * fix url in package json * change package name * prepare for module publishing * use npm ci * preload ethers * nits * Fix broken path to docs in README. (ava-labs#665) * Fix contract examples package json (ava-labs#672) * rename contract-examples to contracts & fix pack issue * allow custom CMD to be run on blockchains for e2e tests * move default hardhat test to utils * fix comment * add more comments * change folder reference * Bump got and ethereum-waffle in /contracts (ava-labs#680) Removes [got](https://github.com/sindresorhus/got). It's no longer used after updating ancestor dependency [ethereum-waffle](https://github.com/EthWorks/Waffle). These dependencies need to be updated together. Removes `got` Updates `ethereum-waffle` from 3.4.4 to 4.0.10 - [Release notes](https://github.com/EthWorks/Waffle/releases) - [Commits](https://github.com/EthWorks/Waffle/compare/[email protected]@4.0.10) --- updated-dependencies: - dependency-name: got dependency-type: indirect - dependency-name: ethereum-waffle dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump flat and hardhat in /contracts (ava-labs#679) Bumps [flat](https://github.com/hughsk/flat) to 5.0.2 and updates ancestor dependency [hardhat](https://github.com/nomiclabs/hardhat). These dependencies need to be updated together. Updates `flat` from 4.1.1 to 5.0.2 - [Commits](hughsk/flat@4.1.1...5.0.2) Updates `hardhat` from 2.6.1 to 2.15.0 - [Release notes](https://github.com/nomiclabs/hardhat/releases) - [Commits](https://github.com/nomiclabs/hardhat/compare/[email protected]) --- updated-dependencies: - dependency-name: flat dependency-type: indirect - dependency-name: hardhat dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix core benchmarks (ava-labs#697) * remove unused line in test (ava-labs#695) * bump version to v0.5.2 (ava-labs#681) * bump version to v0.5.2 * downgrade avalanchego to 1.10.2 * update compatibility * Remove unused ethereum-waffle and ethereumjs-tx pkgs (ava-labs#685) * move state upgrade related functions to own file (ava-labs#684) * Update load test to use ANR for network setup (ava-labs#579) * Update load test to use ANR for network setup * Remove ToWebsocketURI that relies on strings feature added in go1.20 * Update load test to use ENV var from CI if populated * Move env var logic into NewDefaultANRConfig * Add init functions and clean up comments * Remove unnecessary helper * Remove unneeded cmd * Cleanup * Address PR comments * Move 2 min timeout for SetupNetwork to the start of the function * Turn list of txs into tx sequence (ava-labs#683) * Turn list of txs into tx sequence * Add agent comments * Remove unused code and address comments * add some more comments * address more comments' * lint * Remove :::warning --------- Co-authored-by: Anusha <[email protected]> * Minor simulator fixes (ava-labs#703) * add benchmark to template (ava-labs#705) * add benchmark to template * fix comment * Add metrics for gas price / usage (ava-labs#700) Co-authored-by: aaronbuchwald <[email protected]> * added config setting to prune warpdb (ava-labs#702) * added config setting to prune warpdb * refactored warp prune uint test * message * fixed up nits * fixed nits * fixed nits * Update plugin/evm/config.go * Add wrapped validator state for primary to subnet warp messages (ava-labs#708) * Add wrapped validator state for primary to subnet warp message verification * Address PR comment * upload network runner artifact after e2e test run (ava-labs#709) * Differentiate mandatory/optional Subnet-EVM Network Upgrades (ava-labs#640) * remove network upgrades from upgrade configs and enforce them in vm init * fix tests * remove unused var * remove extra shutdown * copy test chain config when modify * use SimulatedTestChainConfig for simulated tests * copy modified test chain config * add optional upgrades * Add tests * fix comments * use unmarshal in test * unexport functions * add commented out test * add TODO comment * add blockchain.Stop as defer to test (ava-labs#714) Co-authered-by: najeal <[email protected]> Co-authored-by: Ceyhun Onur <[email protected]> * Add support for predicate gas into IntrinsicGas (ava-labs#713) * Add support for predicate gas * Address review * Fix test and add test for PredicateGas * Address comments * Add signature aggregation helpers (ava-labs#711) * Add signature aggregation helpers * Fix * Move WarpQuorumDenominator into new file * Address comments * fix var rename * Improve comments * Address comments * Add predicate packing helper (ava-labs#710) * Add predicate packing helper * move predicate code and add readme * fix moved import * fix merge * Run ginkgo precompile tests in parallel (ava-labs#712) Use SynchronizedBeforeSuite to create the node and all the subnet that are needed to run tests. Pass the blockchainID to each work processor and run the tests in parallel to reduce runtime * Fix BlockContext comment (ava-labs#731) * precompile accepter should take logIdx (ava-labs#728) * precompile accepter should take logIdx * minimize mock * copyright yr * add comments * Improve tests (ava-labs#733) Improve tests based on feedback from ava-labs#712 * Fix precompile test gen template (ava-labs#737) * fix Benchmark * fix spelling/references in Benchmark * Clean up gossip (ava-labs#744) * Remove call from precompile (ava-labs#748) * Fix nil types tmpl (ava-labs#751) * add more nil types and fix int types * readd hash type * remove convertToNil * Sync coreth v0.12.4 rc0 (ava-labs#694) * apply diff to rc0 * replace coreth with subnet-evm * go mod tidy * remove must commit * remove apricots * replace coreth with subnet-evm * rename banff and cortina * test memory db * add dUpgrade * fix errors * sync with v0.12.4-rc0 * more diffs * change templates * readd preparePredicateSlots * run goimports * readd tx allow list check to txpool * Fix tests * readd subnet evm check for tests * fix tests * add gas limit & coinbase fakers * Fix timestamp types * check error from newblockchain * fix gas limit in simulated genesis * downgrade ci to go 1.19 * gofmt files * rebump go to 1.20 * bump anr to latest * bump anr to latest * prepare predicate storage slots in Subnet-EVM * uncomment tests * change order of airdrop memory cleanup * add utils import * Coreth 0.12.4 x apply (ava-labs#720) Co-authored-by: Darioush Jalali <[email protected]> * move instruction sets * revert unpack changes * reduce diffs * move genesis verify to vm (ava-labs#753) * move genesis verify to vm * fix comment * set a correct default from genesis * fix log * revert change to vm.go --------- Co-authored-by: Darioush Jalali <[email protected]> * avoid adding/removing GasLimits in tests (ava-labs#755) * move genesis verify to vm * fix comment * set a correct default from genesis * fix log * revert change to vm.go * avoid adding/removing GasLimits in tests * nit --------- Co-authored-by: Ceyhun Onur <[email protected]> --------- Co-authored-by: Darioush Jalali <[email protected]> * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release (ava-labs#757) * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release * bump anr version to v1.7.1 * Migrate ABI unpack fix (ava-labs#750) * change sprintf verb and use values (ava-labs#760) * Generalize precompile test suite (ava-labs#759) * restore old functions, generalize paths in sync subnet creation * add subnet suit struct and methods * move constants * fix reviews * add comment * Update subnet.go Signed-off-by: Ceyhun Onur <[email protected]> --------- Signed-off-by: Ceyhun Onur <[email protected]> * Add warp contract implementation (ava-labs#718) * Add warp contract implementation * Cleanup predicate test * Fix new function signature * Replace invalid fuzz test with unit test * Add chain config to enable warp API for warp e2e test * remove unused var * Add experimental warning and move warp precompile to x/ package * fix warning label * Fix warning * vm test nits * Improve sendWarpMessenger sol comment * more vm warp test nits * Move warp params into params package * More vm warp test nits * Address more PR comments * Remove triggerTx2 * Add check for expected topics from sendWarpMessage log * Fix config test * Fix incorrect replace * remove unnecessary echo * Address comments * Address comments * Address PR comments * Improve comments * Convert [32]byte type to common.Hash * Add base cost for getVerifiedWarpMessage * fix require equal type check * Fix updated awm message format * Update warp message format * Move IWarpMessenger.sol to interfaces/ * Add newline to warp genesis * uncomment assertion * Fix broken links in README * Add metrics to simulator (ava-labs#706) * add tps * get block build metrics * Add issuance time, confirmed time, issued-> acceptance time, verify time * Log times at end of batch * cleaner * address comments * remove unused code block * avoids taking len of channel * nits * pass in stringID * move to loader * remove unused field * revert file back * cleaner * lint * make it work for ws or for rpc * protect * endpoint * no return on defer * sep to a funciton * have blockchainidstr passed in * typo * pass in metrics through config * address comments * address more comments and edit err policy of metrics functions * add more logging to load_test * typo * better check * fix endpoints * typo: * individual * histogram * address feedback: * remove metrics from default * address comments * simplify time metrics * better explanation * address comments * address comments * cleanup * more cleanup * rename vars for clarity * ws * cleanup * address comments * ws * expose metrics add flag * fix blocking issue of http server and gracefully stop it * cleanup * use constant * add issuance to confirmation metrics * ws * simplify metrics server * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release (ava-labs#757) * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release * bump anr version to v1.7.1 * handle control c * print out output * clean up * clean up * remove go routines to close client * address comments * memory leak * fix * print * e2e test: connect to appropriate chain (ava-labs#771) * Bump avalanchego dep and update version for next release (ava-labs#770) * Bump avalanchego dep and update version for next release * Update cache sizes * Update flag in run script to replace staking-enabled=false * Update block builder to enforce predicate one tx at a time (ava-labs#773) * Update block builder to enforce predicate one tx at a time * remove dead code * Update miner/worker.go Co-authored-by: Darioush Jalali <[email protected]> Signed-off-by: aaronbuchwald <[email protected]> --------- Signed-off-by: aaronbuchwald <[email protected]> Co-authored-by: Darioush Jalali <[email protected]> * update iocorders address * remove state-sync-enabled from chain config * 1s block ticker * flush hubbleFeed when len > 512 * fix compilation err --------- Signed-off-by: Gyuho Lee <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]> Signed-off-by: aaronbuchwald <[email protected]> Co-authored-by: Richard Pringle <[email protected]> Co-authored-by: Darioush Jalali <[email protected]> Co-authored-by: Ceyhun Onur <[email protected]> Co-authored-by: Gyuho Lee <[email protected]> Co-authored-by: İzzet Emre Demir <[email protected]> Co-authored-by: aaronbuchwald <[email protected]> Co-authored-by: Gabriel Cardona <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anusha <[email protected]> Co-authored-by: Cesar <[email protected]> Co-authored-by: morrisettjohn <[email protected]> Co-authored-by: nathan haim <[email protected]> Co-authored-by: Martin Eckardt <[email protected]> Co-authored-by: rodrigo <[email protected]> Co-authored-by: Anusha <[email protected]>

remove network upgrades from upgrade configs and enforce them in vm init

41bc6f9

ceyonur added subnet-upgrades network upgrade labels May 4, 2023

ceyonur self-assigned this May 4, 2023

Merge branch 'master' into mandatory-network-upgrades

5a8277e

richardpringle reviewed May 4, 2023

View reviewed changes

ceyonur added 8 commits May 5, 2023 15:46

fix tests

57ae650

Merge branch 'mandatory-network-upgrades' of github.com:ava-labs/subn…

e296ee0

…et-evm into mandatory-network-upgrades

remove unused var

c7c42af

remove extra shutdown

0f3b91e

copy test chain config when modify

a5d1b00

use SimulatedTestChainConfig for simulated tests

5f0b046

copy modified test chain config

975b74e

Merge branch 'master' into mandatory-network-upgrades

28ed33a

ceyonur marked this pull request as ready for review May 23, 2023 15:11

ceyonur requested review from darioush and aaronbuchwald as code owners May 23, 2023 15:11

aaronbuchwald reviewed May 25, 2023

View reviewed changes

ceyonur marked this pull request as draft June 12, 2023 19:13

ceyonur added 4 commits June 13, 2023 18:49

Merge branch 'master' into mandatory-network-upgrades

0fb40f4

add optional upgrades

584498e

Add tests

053b72c

Merge branch 'master' into mandatory-network-upgrades

297474c

ceyonur marked this pull request as ready for review June 18, 2023 14:15

aaronbuchwald reviewed Jun 27, 2023

View reviewed changes

aaronbuchwald approved these changes Jun 27, 2023

View reviewed changes

darioush reviewed Jun 27, 2023

View reviewed changes

ceyonur added 5 commits June 27, 2023 13:11

fix comments

11e649e

use unmarshal in test

fe3d9f8

unexport functions

8173f2c

add commented out test

2565137

add TODO comment

d130ce0

darioush approved these changes Jun 28, 2023

View reviewed changes

aaronbuchwald approved these changes Jul 3, 2023

View reviewed changes

Merge branch 'master' into mandatory-network-upgrades

74f82e6

aaronbuchwald merged commit 9abfac7 into master Jul 3, 2023

aaronbuchwald deleted the mandatory-network-upgrades branch July 3, 2023 14:21

		// use modified genesis to set SubnetEVM timestamp to 1
		genesisJsonModifiedSubnetEVM := strings.Replace(genesisJSONSubnetEVM, "\"subnetEVMTimestamp\":0", "\"subnetEVMTimestamp\":1", 1)

mandatory network upgrades #640

mandatory network upgrades #640

Uh oh!

Conversation

ceyonur commented May 4, 2023

Why this should be merged

How this works

How this was tested

How is this documented

Uh oh!

richardpringle left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!