test(systemtests): add appside mempool e2e test #580

cloudgray · 2025-09-02T11:16:30Z

Description

Summary

This PR implements comprehensive end-to-end testing for appside mempool functionality, adding robust test infrastructure and extensive test cases to verify mempool behavior across different transaction types and node configurations.

Closes #581

System Tests Directory Structure

clients/ - Multi-client support infrastructure
- Ethereum client integration for EVM transaction handling
- Cosmos client integration for native Cosmos transaction support
mempool/ - Dedicated mempool test suite implementation
- Comprehensive test cases using structured test suite pattern
- Deterministic testing with nonce and baseFee tracking

Test Mechanism

2.1 Persistent Node Architecture

Single node startup for multiple test cases execution
evmd nodes remain running throughout entire test suite
Test suite maintains deterministic state by tracking:
- Account nonces across all test cases
- BaseFee progression for optimal gas pricing

2.2 Common Verification Framework

Action-based test structure: Each test case broken into smaller
verification steps
Inter-test validation: Between every test case, suite performs:
- Verification that expected pending transactions are actually pending
  (or committed)
- Confirmation that expected queued transactions are properly queued
  on target nodes only
Final state verification: After all actions complete, suite waits for
pending transactions to commit

Test Coverage Checklist

Multi-node transaction broadcasting - All test cases execute with transactions broadcast across multiple nodes
Ethereum-only transaction scenarios:
- Queued transactions remain unbroadcast until promotion conditions met
- Pending transactions broadcast immediately (including queue-to-pending promotions)
- Same account transactions sent to different nodes handled correctly
Cosmos-only transaction scenarios - Native Cosmos transaction mempool behavior
Mixed transaction scenarios - Ethereum + Cosmos transactions broadcasted together
Transaction type mixing - LegacyFeeTx and DynamicFeeTx combined scenarios

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

tackled an existing issue or discussed with a team member
left instructions on how to review the changes
targeted the main branch

tests/systemtests/clients/ethclient.go

tests/systemtests/suite/query.go

…s=0stake

aljo242 · 2025-09-10T21:38:39Z

mempool/mempool.go

-	if m.HasEventBus() {
+	if m.eventBus != nil {


why did we make this change? seems out of scope for this

Oh, It seems that when I merge conflict from main branch, something wrong happens.
I'll rollback that changes.

fixed! (d089213)

vladjdk

Overall, lovely test setup! I think this will make it significantly easier to write E2E tests in the future, which is something I applaud you for doing.

I left a few comments, mostly regarding the separation of Pending and Committed as end-states, as transactions get gossiped differently in the consensus phase (commit) vs. the mempool phase (unconfirmed).

vladjdk · 2025-09-12T18:19:29Z

tests/systemtests/clients/types.go

+	Queued  map[string]map[string]*RPCTransaction `json:"queued"`
+}
+
+type RPCTransaction struct {


We have this type here. Let's use that instead of copying.

vladjdk · 2025-09-12T18:20:00Z

tests/systemtests/clients/types.go

+	PrivKey       *ethsecp256k1.PrivKey
+}
+
+type TxPoolResult struct {


Ditto, same file as the RPCTransaction.

vladjdk · 2025-09-12T18:22:40Z

tests/systemtests/go.mod

 	cosmossdk.io/systemtests v1.3.0
-	github.com/ethereum/go-ethereum v1.15.5
+	github.com/cometbft/cometbft/v2 v2.0.0-rc1


Why are we using V2 and RCs here?

vladjdk · 2025-09-12T18:24:10Z

tests/systemtests/mempool/interface.go

+type TestSuite interface {
+	// Test Lifecycle
+	BeforeEach(t *testing.T)
+	AfterEach(t *testing.T)


Suggested change

AfterEach(t *testing.T)

AfterEachCase(t *testing.T)

vladjdk · 2025-09-12T18:24:21Z

tests/systemtests/mempool/interface.go

+	// Test Lifecycle
+	BeforeEach(t *testing.T)
+	AfterEach(t *testing.T)
+	JustAfterEach(t *testing.T)


Suggested change

JustAfterEach(t *testing.T)

AfterEachAction(t *testing.T)

vladjdk · 2025-09-12T19:12:05Z

tests/systemtests/mempool/test_replacement.go

+				func(s TestSuite) {
+					tx1, err := s.SendCosmosTx(t, s.Node(0), "acc0", 0, s.BaseFeeX2(), big.NewInt(1))
+					require.NoError(t, err, "failed to send tx")
+					_, err = s.SendEthTx(t, s.Node(1), "acc0", 0, s.BaseFee(), nil)


vladjdk · 2025-09-12T19:12:48Z

tests/systemtests/mempool/test_replacement.go

+				func(s TestSuite) {
+					_, err := s.SendEthTx(t, s.Node(0), "acc0", 0, s.BaseFee(), nil)
+					require.NoError(t, err, "failed to send tx")
+					tx2, err := s.SendCosmosTx(t, s.Node(1), "acc0", 0, s.BaseFeeX2(), big.NewInt(1))


vladjdk · 2025-09-12T19:12:55Z

tests/systemtests/mempool/test_replacement.go

+				func(s TestSuite) {
+					tx1, err := s.SendEthTx(t, s.Node(0), "acc0", 0, s.BaseFeeX2(), big.NewInt(1))
+					require.NoError(t, err, "failed to send tx")
+					_, err = s.SendCosmosTx(t, s.Node(1), "acc0", 0, s.BaseFee(), nil)


vladjdk · 2025-09-12T19:13:58Z

tests/systemtests/mempool/test_replacement.go

+			},
+		},
+		{
+			name: "legacy should always not tx replace dynamic fee tx",


Suggested change

name: "legacy should always not tx replace dynamic fee tx",

name: "legacy should never replace dynamic fee tx",

vladjdk · 2025-09-12T19:17:42Z

tests/systemtests/suite/query.go

+}
+
+// CheckPendingOrCommitted checks if the given tx is either pending or committed within the timeout duration
+func (s *SystemTestSuite) CheckPendingOrCommitted(


Let's do CheckPending and CheckCommitted as two separate calls as they function differently in Comet.

cloudgray added 28 commits August 28, 2025 21:39

test(mempool): setup e2e test suite for mempool testing

6e76883

WIP

8b68d43

test(mempool): refactor test

7ae8b16

test(mempool): refactor system test suite

6700051

test(systemtest): refactor mempool test

3681758

Merge branch 'main' into teest/appside-mempool-e2e

b41028a

Merge branch 'main' into teest/appside-mempool-e2e

d8368f1

test(systemtests): add cosmosClient

87ec45e

test(systemtests): refactor mempool test

51cb60f

test(systemtests): refactor mempool test

2d41eeb

test(systemtests): refactor mempool test

a4d24e2

test(systemtests): add txpool content verification to mempool tests

f552c38

test(systemtests): fix mempool test suite

173541c

test(systemtests): fix mempool test suite

adc5533

test(systemtests): fix mempool tests

6a3b03e

test(systemtests): fix mempool test suite

6d8d240

test(systemtests): fix mempool test suite

14ef6a9

test(systemtests): fix mempool test suite

9fbe143

test(systemtests): add mempool test cases

48de3af

test(systemtests): add comments

e33650a

test(systemtests): chore: enable replacement tests

a4fe745

chore: modify names of test cases

25e65d4

test(systemtests): enhance helper functions

8f5d4a4

test(systemtests): fix mempool test

06fc10a

chore(systemtests): change test name and add readme.md

ac74377

Merge branch 'main' into teest/appside-mempool-e2e

230e422

chore: update CHANGELOG.md

de46b7f

fix(proto): restore removed data field during merge

bf6db75

github-advanced-security bot found potential problems Sep 9, 2025

View reviewed changes

tests/systemtests/clients/ethclient.go Fixed Show fixed Hide fixed

tests/systemtests/clients/ethclient.go Fixed Show fixed Hide fixed

tests/systemtests/suite/query.go Fixed Show fixed Hide fixed

tests/systemtests/suite/query.go Fixed Show fixed Hide fixed

test(systemtests): refactor

2ecece2

cloudgray added 4 commits September 9, 2025 14:41

test(systemtests): fix map iteration

58d1e7d

chore(systemtests): fix build tag

e6085fd

test(systemtests): add test case for mempool

1c55fb9

test(systemtests): fix map iteration

a55ad3d

cloudgray marked this pull request as ready for review September 9, 2025 08:13

cloudgray requested review from a team as code owners September 9, 2025 08:13

cloudgray requested review from vladjdk and aljo242 September 9, 2025 08:13

cloudgray and others added 3 commits September 10, 2025 14:18

test(systemtests): refactor and add test case for --minimum-gas-price…

d139dd7

…s=0stake

test(systemtests): enhance validation

ffbb84d

Merge branch 'main' into teest/appside-mempool-e2e

57a8d10

aljo242 reviewed Sep 10, 2025

View reviewed changes

cloudgray added 4 commits September 11, 2025 13:26

fix: restore removed changes from merge

d089213

fix: restore removed changes from merge

f4fe8af

fix: restore removed changes from merge

88d36bc

fix: restore removed changes from merge

95c6dfd

vladjdk requested changes Sep 12, 2025

View reviewed changes

	name: "legacy should always not tx replace dynamic fee tx",
	name: "legacy should never replace dynamic fee tx",

test(systemtests): add appside mempool e2e test #580

Are you sure you want to change the base?

test(systemtests): add appside mempool e2e test #580

Conversation

cloudgray commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Summary

System Tests Directory Structure

Test Mechanism

Test Coverage Checklist

Author Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vladjdk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cloudgray commented Sep 2, 2025 •

edited

Loading