Update load test to use ANR for network setup #579

aaronbuchwald · 2023-03-17T02:13:08Z

Why this should be merged

This PR splits out the e2e test setup using the ANR that will be used for the warp e2e tests. This is intended to reduce the size of the actual Warp PR.

How this works

This PR copies the style of https://github.com/ava-labs/hypersdk/blob/warp-messaging/examples/tokenvm/tests/e2e/e2e_test.go to use the ANR to set up a subnet with BLS keys registered on the P-Chain.

It adds the following

thin wrapper for ANR to minimize how much of the ANR interface is exposed to the test logic itself
adds a minimal wrapper of the ANR as a new command to set up an empty default network, a network with two subnets, and stop the network runner

How this was tested

Tested locally to ensure the load test still works and the wrapped ANR commands work correclty.

How is this documented

This PR should add documentation to the README explaining how to use the new command before getting merged.

aaronbuchwald · 2023-03-17T14:40:13Z

This PR is blocked on ava-labs/avalanche-network-runner#467, which will be used to create new subnets with validators that have registered their BLS keys on the P-Chain.

tests/utils/runner/runner.go

ceyonur · 2023-06-19T13:08:52Z

tests/utils/runner/runner.go

+func NewDefaultANRConfig() ANRConfig {
+	defaultConfig := ANRConfig{
+		LogLevel:            "info",
+		AvalancheGoExecPath: os.ExpandEnv("$GOPATH/src/github.com/ava-labs/avalanchego/build/avalanchego"),
+		PluginDir:           os.ExpandEnv("$GOPATH/src/github.com/ava-labs/avalanchego/build/plugins"),
+		GlobalNodeConfig: `{
+			"log-display-level":"info",
+			"proposervm-use-current-height":true,
+			"throttler-inbound-validator-alloc-size":"107374182",
+			"throttler-inbound-node-max-processing-msgs":"100000",
+			"throttler-inbound-bandwidth-refill-rate":"1073741824",
+			"throttler-inbound-bandwidth-max-burst-size":"1073741824",
+			"throttler-inbound-cpu-validator-alloc":"100000",
+			"throttler-inbound-disk-validator-alloc":"10737418240000",
+			"throttler-outbound-validator-alloc-size":"107374182"
+		}`,
+	}
+	// If AVALANCHEGO_BUILD_PATH is populated, override location set by GOPATH
+	if envBuildPath, exists := os.LookupEnv("AVALANCHEGO_BUILD_PATH"); exists {
+		defaultConfig.AvalancheGoExecPath = fmt.Sprintf("%s/avalanchego", envBuildPath)
+		defaultConfig.PluginDir = fmt.Sprintf("%s/plugins", envBuildPath)
+	}
+	return defaultConfig
+}


can we make this a var?

It currently uses this function so that it can expand environment variables when it's called. We could do the same in init(), but since we use environment variables here, I'd prefer to expand them when the function is called.

ceyonur · 2023-06-19T13:09:25Z

tests/utils/runner/runner.go

+
+	n.anrServer, err = runner_server.New(
+		runner_server.Config{
+			Port:                ":12352",


should these be a part of config?

Ya I think it's fine for the port to be a part of the config. Also, this is using the avalanche-network-runner code, and I think we should use the current interface rather than try to push style changes to that dependency unless we're doing a larger re-write.

Overall, I added this here to try and keep the use of ANR as well-separated from our actual tests as possible.

ceyonur · 2023-06-19T13:14:33Z

tests/utils/runner/runner.go

+}
+
+// init starts the ANR server and client if they are not yet populated
+func (n *NetworkManager) init() error {


can we rename this to start() (or similar) to avoid any possible confusion with built-in init function

I don't think this should be easily confused with the built-in init() function and this follows the convention used in avalanchego to use init() to ensure that it is set up correctly before accessing the client/server on the network manager itself.

ceyonur · 2023-06-19T13:19:48Z

tests/utils/runner/runner.go

can we rename the file to network_manager?

anusha-ctrl · 2023-06-20T14:41:32Z

tests/utils/runner/runner.go

+		LogLevel:            "info",
+		AvalancheGoExecPath: os.ExpandEnv("$GOPATH/src/github.com/ava-labs/avalanchego/build/avalanchego"),
+		PluginDir:           os.ExpandEnv("$GOPATH/src/github.com/ava-labs/avalanchego/build/plugins"),
+		GlobalNodeConfig: `{


if the user doesn't have these set correctly, we should let them know? $GOPATH/src/github.com/ava-labs/avalanchego/build/plugins"

I think this is OK as a default. This assumption already exists in our scripts/.

I'll add a comment to this function stating which environment variables will be used in this function and need to be set as a result.

anusha-ctrl · 2023-06-20T15:22:27Z

tests/utils/runner/runner.go

+			time.Sleep(1 * time.Second)
+			continue
+		}
+		// TODO: clean this up


this logic seems weird. if err != nil we will continuously wait, when would we enter the second code block?

Good catch, gonna remove the second code block

aaronbuchwald · 2023-06-27T17:21:15Z

tests/utils/runner/runner.go

+			"throttler-inbound-validator-alloc-size":"107374182",
+			"throttler-inbound-node-max-processing-msgs":"100000",
+			"throttler-inbound-bandwidth-refill-rate":"1073741824",
+			"throttler-inbound-bandwidth-max-burst-size":"1073741824",
+			"throttler-inbound-cpu-validator-alloc":"100000",
+			"throttler-inbound-disk-validator-alloc":"10737418240000",
+			"throttler-outbound-validator-alloc-size":"107374182"


remove unneeded params

* Update load test to use ANR for network setup * Remove ToWebsocketURI that relies on strings feature added in go1.20 * Update load test to use ENV var from CI if populated * Move env var logic into NewDefaultANRConfig * Add init functions and clean up comments * Remove unnecessary helper * Remove unneeded cmd * Cleanup * Address PR comments * Move 2 min timeout for SetupNetwork to the start of the function

* Rename test_ to step_ for clarity (ava-labs#636) * Codeowners fix (specific dir overrides global) (ava-labs#639) * rename state upgrade test (ava-labs#641) * plugin/evm: fix godoc typo (ava-labs#647) Signed-off-by: Gyuho Lee <[email protected]> * plugin/evm: remove unncessary tx.Hash op in map iter (ava-labs#648) Signed-off-by: Gyuho Lee <[email protected]> * deployExampleRewardManager.ts hardhat script fixed (ava-labs#650) * Migrate HardHat tests to DS-tests (Solidity tests) (ava-labs#601) * Increase trie dirty default cache size to 512MB (ava-labs#656) * export gingkgo suites (ava-labs#659) * export gingkgo suites * remove precompiles description from ping tests * update package.json * fix url in package json * change package name * Publish npm module for contract-examples (ava-labs#663) * export gingkgo suites * remove precompiles description from ping tests * update package.json * fix url in package json * change package name * prepare for module publishing * use npm ci * preload ethers * nits * Fix broken path to docs in README. (ava-labs#665) * Fix contract examples package json (ava-labs#672) * rename contract-examples to contracts & fix pack issue * allow custom CMD to be run on blockchains for e2e tests * move default hardhat test to utils * fix comment * add more comments * change folder reference * Bump got and ethereum-waffle in /contracts (ava-labs#680) Removes [got](https://github.com/sindresorhus/got). It's no longer used after updating ancestor dependency [ethereum-waffle](https://github.com/EthWorks/Waffle). These dependencies need to be updated together. Removes `got` Updates `ethereum-waffle` from 3.4.4 to 4.0.10 - [Release notes](https://github.com/EthWorks/Waffle/releases) - [Commits](https://github.com/EthWorks/Waffle/compare/[email protected]@4.0.10) --- updated-dependencies: - dependency-name: got dependency-type: indirect - dependency-name: ethereum-waffle dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump flat and hardhat in /contracts (ava-labs#679) Bumps [flat](https://github.com/hughsk/flat) to 5.0.2 and updates ancestor dependency [hardhat](https://github.com/nomiclabs/hardhat). These dependencies need to be updated together. Updates `flat` from 4.1.1 to 5.0.2 - [Commits](hughsk/flat@4.1.1...5.0.2) Updates `hardhat` from 2.6.1 to 2.15.0 - [Release notes](https://github.com/nomiclabs/hardhat/releases) - [Commits](https://github.com/nomiclabs/hardhat/compare/[email protected]) --- updated-dependencies: - dependency-name: flat dependency-type: indirect - dependency-name: hardhat dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix core benchmarks (ava-labs#697) * remove unused line in test (ava-labs#695) * bump version to v0.5.2 (ava-labs#681) * bump version to v0.5.2 * downgrade avalanchego to 1.10.2 * update compatibility * Remove unused ethereum-waffle and ethereumjs-tx pkgs (ava-labs#685) * move state upgrade related functions to own file (ava-labs#684) * Update load test to use ANR for network setup (ava-labs#579) * Update load test to use ANR for network setup * Remove ToWebsocketURI that relies on strings feature added in go1.20 * Update load test to use ENV var from CI if populated * Move env var logic into NewDefaultANRConfig * Add init functions and clean up comments * Remove unnecessary helper * Remove unneeded cmd * Cleanup * Address PR comments * Move 2 min timeout for SetupNetwork to the start of the function * Turn list of txs into tx sequence (ava-labs#683) * Turn list of txs into tx sequence * Add agent comments * Remove unused code and address comments * add some more comments * address more comments' * lint * Remove :::warning --------- Co-authored-by: Anusha <[email protected]> * Minor simulator fixes (ava-labs#703) * add benchmark to template (ava-labs#705) * add benchmark to template * fix comment * Add metrics for gas price / usage (ava-labs#700) Co-authored-by: aaronbuchwald <[email protected]> * added config setting to prune warpdb (ava-labs#702) * added config setting to prune warpdb * refactored warp prune uint test * message * fixed up nits * fixed nits * fixed nits * Update plugin/evm/config.go * Add wrapped validator state for primary to subnet warp messages (ava-labs#708) * Add wrapped validator state for primary to subnet warp message verification * Address PR comment * upload network runner artifact after e2e test run (ava-labs#709) * Differentiate mandatory/optional Subnet-EVM Network Upgrades (ava-labs#640) * remove network upgrades from upgrade configs and enforce them in vm init * fix tests * remove unused var * remove extra shutdown * copy test chain config when modify * use SimulatedTestChainConfig for simulated tests * copy modified test chain config * add optional upgrades * Add tests * fix comments * use unmarshal in test * unexport functions * add commented out test * add TODO comment * add blockchain.Stop as defer to test (ava-labs#714) Co-authered-by: najeal <[email protected]> Co-authored-by: Ceyhun Onur <[email protected]> * Add support for predicate gas into IntrinsicGas (ava-labs#713) * Add support for predicate gas * Address review * Fix test and add test for PredicateGas * Address comments * Add signature aggregation helpers (ava-labs#711) * Add signature aggregation helpers * Fix * Move WarpQuorumDenominator into new file * Address comments * fix var rename * Improve comments * Address comments * Add predicate packing helper (ava-labs#710) * Add predicate packing helper * move predicate code and add readme * fix moved import * fix merge * Run ginkgo precompile tests in parallel (ava-labs#712) Use SynchronizedBeforeSuite to create the node and all the subnet that are needed to run tests. Pass the blockchainID to each work processor and run the tests in parallel to reduce runtime * Fix BlockContext comment (ava-labs#731) * precompile accepter should take logIdx (ava-labs#728) * precompile accepter should take logIdx * minimize mock * copyright yr * add comments * Improve tests (ava-labs#733) Improve tests based on feedback from ava-labs#712 * Fix precompile test gen template (ava-labs#737) * fix Benchmark * fix spelling/references in Benchmark * Clean up gossip (ava-labs#744) * Remove call from precompile (ava-labs#748) * Fix nil types tmpl (ava-labs#751) * add more nil types and fix int types * readd hash type * remove convertToNil * Sync coreth v0.12.4 rc0 (ava-labs#694) * apply diff to rc0 * replace coreth with subnet-evm * go mod tidy * remove must commit * remove apricots * replace coreth with subnet-evm * rename banff and cortina * test memory db * add dUpgrade * fix errors * sync with v0.12.4-rc0 * more diffs * change templates * readd preparePredicateSlots * run goimports * readd tx allow list check to txpool * Fix tests * readd subnet evm check for tests * fix tests * add gas limit & coinbase fakers * Fix timestamp types * check error from newblockchain * fix gas limit in simulated genesis * downgrade ci to go 1.19 * gofmt files * rebump go to 1.20 * bump anr to latest * bump anr to latest * prepare predicate storage slots in Subnet-EVM * uncomment tests * change order of airdrop memory cleanup * add utils import * Coreth 0.12.4 x apply (ava-labs#720) Co-authored-by: Darioush Jalali <[email protected]> * move instruction sets * revert unpack changes * reduce diffs * move genesis verify to vm (ava-labs#753) * move genesis verify to vm * fix comment * set a correct default from genesis * fix log * revert change to vm.go --------- Co-authored-by: Darioush Jalali <[email protected]> * avoid adding/removing GasLimits in tests (ava-labs#755) * move genesis verify to vm * fix comment * set a correct default from genesis * fix log * revert change to vm.go * avoid adding/removing GasLimits in tests * nit --------- Co-authored-by: Ceyhun Onur <[email protected]> --------- Co-authored-by: Darioush Jalali <[email protected]> * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release (ava-labs#757) * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release * bump anr version to v1.7.1 * Migrate ABI unpack fix (ava-labs#750) * change sprintf verb and use values (ava-labs#760) * Generalize precompile test suite (ava-labs#759) * restore old functions, generalize paths in sync subnet creation * add subnet suit struct and methods * move constants * fix reviews * add comment * Update subnet.go Signed-off-by: Ceyhun Onur <[email protected]> --------- Signed-off-by: Ceyhun Onur <[email protected]> * Add warp contract implementation (ava-labs#718) * Add warp contract implementation * Cleanup predicate test * Fix new function signature * Replace invalid fuzz test with unit test * Add chain config to enable warp API for warp e2e test * remove unused var * Add experimental warning and move warp precompile to x/ package * fix warning label * Fix warning * vm test nits * Improve sendWarpMessenger sol comment * more vm warp test nits * Move warp params into params package * More vm warp test nits * Address more PR comments * Remove triggerTx2 * Add check for expected topics from sendWarpMessage log * Fix config test * Fix incorrect replace * remove unnecessary echo * Address comments * Address comments * Address PR comments * Improve comments * Convert [32]byte type to common.Hash * Add base cost for getVerifiedWarpMessage * fix require equal type check * Fix updated awm message format * Update warp message format * Move IWarpMessenger.sol to interfaces/ * Add newline to warp genesis * uncomment assertion * Fix broken links in README * Add metrics to simulator (ava-labs#706) * add tps * get block build metrics * Add issuance time, confirmed time, issued-> acceptance time, verify time * Log times at end of batch * cleaner * address comments * remove unused code block * avoids taking len of channel * nits * pass in stringID * move to loader * remove unused field * revert file back * cleaner * lint * make it work for ws or for rpc * protect * endpoint * no return on defer * sep to a funciton * have blockchainidstr passed in * typo * pass in metrics through config * address comments * address more comments and edit err policy of metrics functions * add more logging to load_test * typo * better check * fix endpoints * typo: * individual * histogram * address feedback: * remove metrics from default * address comments * simplify time metrics * better explanation * address comments * address comments * cleanup * more cleanup * rename vars for clarity * ws * cleanup * address comments * ws * expose metrics add flag * fix blocking issue of http server and gracefully stop it * cleanup * use constant * add issuance to confirmation metrics * ws * simplify metrics server * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release (ava-labs#757) * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release * bump anr version to v1.7.1 * handle control c * print out output * clean up * clean up * remove go routines to close client * address comments * memory leak * fix * print * e2e test: connect to appropriate chain (ava-labs#771) * Bump avalanchego dep and update version for next release (ava-labs#770) * Bump avalanchego dep and update version for next release * Update cache sizes * Update flag in run script to replace staking-enabled=false * Update block builder to enforce predicate one tx at a time (ava-labs#773) * Update block builder to enforce predicate one tx at a time * remove dead code * Update miner/worker.go Co-authored-by: Darioush Jalali <[email protected]> Signed-off-by: aaronbuchwald <[email protected]> --------- Signed-off-by: aaronbuchwald <[email protected]> Co-authored-by: Darioush Jalali <[email protected]> * update iocorders address * remove state-sync-enabled from chain config * 1s block ticker * flush hubbleFeed when len > 512 * fix compilation err --------- Signed-off-by: Gyuho Lee <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]> Signed-off-by: aaronbuchwald <[email protected]> Co-authored-by: Richard Pringle <[email protected]> Co-authored-by: Darioush Jalali <[email protected]> Co-authored-by: Ceyhun Onur <[email protected]> Co-authored-by: Gyuho Lee <[email protected]> Co-authored-by: İzzet Emre Demir <[email protected]> Co-authored-by: aaronbuchwald <[email protected]> Co-authored-by: Gabriel Cardona <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anusha <[email protected]> Co-authored-by: Cesar <[email protected]> Co-authored-by: morrisettjohn <[email protected]> Co-authored-by: nathan haim <[email protected]> Co-authored-by: Martin Eckardt <[email protected]> Co-authored-by: rodrigo <[email protected]> Co-authored-by: Anusha <[email protected]>

Update load test to use ANR for network setup

c28db5b

aaronbuchwald self-assigned this Mar 17, 2023

aaronbuchwald added the ci label Mar 17, 2023

aaronbuchwald added 2 commits March 17, 2023 10:36

Merge branch 'master' into setup-e2e-helpers

b4b9f13

Remove ToWebsocketURI that relies on strings feature added in go1.20

c840d44

aaronbuchwald added the blocked Blocked by another issue or pull request label Mar 17, 2023

aaronbuchwald added 2 commits March 17, 2023 10:47

Update load test to use ENV var from CI if populated

51f58c0

Move env var logic into NewDefaultANRConfig

1ccdea0

aaronbuchwald commented Mar 17, 2023

View reviewed changes

tests/utils/runner/runner.go Show resolved Hide resolved

aaronbuchwald mentioned this pull request Mar 23, 2023

Warp e2e #586

Closed

14 tasks

aaronbuchwald added 6 commits March 23, 2023 12:41

Add init functions and clean up comments

a954205

Merge branch 'master' into setup-e2e-helpers

0991581

Merge branch 'master' into setup-e2e-helpers

6aff801

Remove unnecessary helper

0f5cbe8

Remove unneeded cmd

dfecbea

Cleanup

e37ec0f

aaronbuchwald marked this pull request as ready for review June 17, 2023 20:59

aaronbuchwald requested review from ceyonur and darioush as code owners June 17, 2023 20:59

ceyonur reviewed Jun 19, 2023

View reviewed changes

anusha-ctrl reviewed Jun 20, 2023

View reviewed changes

aaronbuchwald commented Jun 27, 2023

View reviewed changes

aaronbuchwald added 3 commits June 27, 2023 13:35

Address PR comments

b5238b8

Move 2 min timeout for SetupNetwork to the start of the function

279a350

Merge branch 'master' into setup-e2e-helpers

988d0b9

darioush approved these changes Jun 27, 2023

View reviewed changes

aaronbuchwald removed the blocked Blocked by another issue or pull request label Jun 27, 2023

aaronbuchwald merged commit 0fc8ad1 into master Jun 27, 2023

aaronbuchwald deleted the setup-e2e-helpers branch June 27, 2023 18:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update load test to use ANR for network setup #579

Update load test to use ANR for network setup #579

Uh oh!

aaronbuchwald commented Mar 17, 2023

Uh oh!

aaronbuchwald commented Mar 17, 2023

Uh oh!

Uh oh!

ceyonur Jun 19, 2023

Uh oh!

aaronbuchwald Jun 27, 2023

Uh oh!

ceyonur Jun 19, 2023

Uh oh!

aaronbuchwald Jun 27, 2023

Uh oh!

ceyonur Jun 19, 2023

Uh oh!

aaronbuchwald Jun 27, 2023

Uh oh!

ceyonur Jun 19, 2023

Uh oh!

aaronbuchwald Jun 27, 2023

Uh oh!

anusha-ctrl Jun 20, 2023

Uh oh!

aaronbuchwald Jun 27, 2023

Uh oh!

anusha-ctrl Jun 20, 2023

Uh oh!

aaronbuchwald Jun 27, 2023

Uh oh!

aaronbuchwald Jun 27, 2023

Uh oh!

Uh oh!

Update load test to use ANR for network setup #579

Update load test to use ANR for network setup #579

Uh oh!

Conversation

aaronbuchwald commented Mar 17, 2023

Why this should be merged

How this works

How this was tested

How is this documented

Uh oh!

aaronbuchwald commented Mar 17, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!