Skip to content

eth/downloader: fix race condition in tests #22140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 9, 2021

Conversation

ziogaschr
Copy link
Contributor

Running this cmd:

go test -timeout 420s -count 1 -race github.com/ethereum/go-ethereum/eth/downloader

was giving race condition issues.

This PR fixes the minor race condition in the tests.

@holiman
Copy link
Contributor

holiman commented Jan 7, 2021

Hm. I don't see how this fixes any race -- do you have any more context, e.g. from the race detector?

@ziogaschr
Copy link
Contributor Author

ziogaschr commented Jan 7, 2021

Sorry @holiman that I haven't added the initial error.

The actual point where the race is being occurred is https://github.com/ethereum/go-ethereum/blob/master/eth/downloader/downloader.go#L387-L389

Traceback from master branch follows:

➜ go test -timeout 420s -count 1 -race github.com/ethereum/go-ethereum/eth/downloader
==================
WARNING: DATA RACE
Write at 0x00c00836e738 by goroutine 1179:
  github.com/ethereum/go-ethereum/eth/downloader.(*queue).Reset()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/queue.go:178 +0x924
  github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:401 +0x1d9
  github.com/ethereum/go-ethereum/eth/downloader.(*downloadTester).sync()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:111 +0x2c4
  github.com/ethereum/go-ethereum/eth/downloader.testThrottling.func2()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:570 +0x64

Previous read at 0x00c00836e738 by goroutine 104:
  github.com/ethereum/go-ethereum/eth/downloader.testThrottling()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:590 +0x478
  github.com/ethereum/go-ethereum/eth/downloader.TestThrottling65Full()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:550 +0x49
  testing.tRunner()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1123 +0x202

Goroutine 1179 (running) created at:
  github.com/ethereum/go-ethereum/eth/downloader.testThrottling()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:569 +0x2b5
  github.com/ethereum/go-ethereum/eth/downloader.TestThrottling65Full()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:550 +0x49
  testing.tRunner()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1123 +0x202

Goroutine 104 (running) created at:
  testing.(*T).Run()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1168 +0x5bb
  testing.runTests.func1()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1439 +0xa6
  testing.tRunner()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1123 +0x202
  testing.runTests()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1437 +0x612
  testing.(*M).Run()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1345 +0x3b3
  main.main()
      _testmain.go:247 +0x236
==================
--- FAIL: TestThrottling65Fast (8.13s)
    testing.go:1038: race detected during execution of test
==================
WARNING: DATA RACE
Write at 0x00c0093a6918 by goroutine 1131:
  github.com/ethereum/go-ethereum/eth/downloader.(*queue).Reset()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/queue.go:178 +0x924
  github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:401 +0x1d9
  github.com/ethereum/go-ethereum/eth/downloader.(*downloadTester).sync()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:111 +0x2c4
  github.com/ethereum/go-ethereum/eth/downloader.testThrottling.func2()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:570 +0x64

Previous read at 0x00c0093a6918 by goroutine 35:
  github.com/ethereum/go-ethereum/eth/downloader.testThrottling()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:590 +0x478
  github.com/ethereum/go-ethereum/eth/downloader.TestThrottling64Full()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:548 +0x49
  testing.tRunner()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1123 +0x202

Goroutine 1131 (running) created at:
  github.com/ethereum/go-ethereum/eth/downloader.testThrottling()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:569 +0x2b5
  github.com/ethereum/go-ethereum/eth/downloader.TestThrottling64Full()
      /Users/ziogaschr/Sites/go/src/github.com/ethereum/go-ethereum/eth/downloader/downloader_test.go:548 +0x49
  testing.tRunner()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1123 +0x202

Goroutine 35 (running) created at:
  testing.(*T).Run()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1168 +0x5bb
  testing.runTests.func1()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1439 +0xa6
  testing.tRunner()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1123 +0x202
  testing.runTests()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1437 +0x612
  testing.(*M).Run()
      /usr/local/Cellar/go/1.15.5/libexec/src/testing/testing.go:1345 +0x3b3
  main.main()
      _testmain.go:247 +0x236
==================
--- FAIL: TestForkedSync64Fast (6.90s)
    testing.go:1038: race detected during execution of test
--- FAIL: TestThrottling65Full (6.10s)
    testing.go:1038: race detected during execution of test
--- FAIL: TestThrottling64Full (5.19s)
    testing.go:1038: race detected during execution of test
--- FAIL: TestDeliverHeadersHang64Fast (223.78s)
    testing.go:1038: race detected during execution of test
--- FAIL: TestDeliverHeadersHang64Full (224.19s)
    testing.go:1038: race detected during execution of test
--- FAIL: TestDeliverHeadersHang65Full (227.13s)
    testing.go:1038: race detected during execution of test
--- FAIL: TestDeliverHeadersHang65Light (233.50s)
    testing.go:1038: race detected during execution of test
--- FAIL: TestDeliverHeadersHang65Fast (233.54s)
    testing.go:1038: race detected during execution of test
FAIL
FAIL    github.com/ethereum/go-ethereum/eth/downloader  284.633s
FAIL

@fjl fjl changed the title downloader: fix race condition in tests eth/downloader: fix race condition in tests Jan 7, 2021
@ziogaschr
Copy link
Contributor Author

ziogaschr commented Jan 8, 2021

The CI's failing tests doesn't seem to be related to this change.

@holiman
Copy link
Contributor

holiman commented Jan 8, 2021

Based on the output from that race detector, it seems to me that this would be a more accurate fix:

diff --git a/eth/downloader/downloader_test.go b/eth/downloader/downloader_test.go
index 6578275d0c..5de1ef3f8e 100644
--- a/eth/downloader/downloader_test.go
+++ b/eth/downloader/downloader_test.go
@@ -584,14 +584,15 @@ func testThrottling(t *testing.T, protocol uint, mode SyncMode) {
 			time.Sleep(25 * time.Millisecond)
 
 			tester.lock.Lock()
+			tester.downloader.queue.lock.Lock()
+			tester.downloader.queue.resultCache.lock.Lock()
 			{
-				tester.downloader.queue.resultCache.lock.Lock()
 				cached = tester.downloader.queue.resultCache.countCompleted()
-				tester.downloader.queue.resultCache.lock.Unlock()
 				frozen = int(atomic.LoadUint32(&blocked))
 				retrieved = len(tester.ownBlocks)
-
 			}
+			tester.downloader.queue.resultCache.lock.Unlock()
+			tester.downloader.queue.lock.Unlock()
 			tester.lock.Unlock()
 
 			if cached == blockCacheMaxItems ||

@ziogaschr
Copy link
Contributor Author

Thanks @holiman.
I tried fixing in this place, but I missed the queue lock tester.downloader.queue.lock.Lock(). I added your fix.

I am wondering if you agree that it makes sense to also leave my initial commit, as it mimics how the downloader.New() is being used from within the actual code. Let me know if you prefer that I remove it.

@holiman
Copy link
Contributor

holiman commented Jan 8, 2021

I am wondering if you agree that it makes sense to also leave my initial commit

I don't know, I don't really see any reason for it, but I can be convinced... Does not allocating the sync bloom make the tests run any faster? I would probably prefer to leave the bulk of the code untouched if there's no really solid reason to change it

@ziogaschr
Copy link
Contributor Author

No, I don't have a solid reason for it. I will revert the commit and feel free to squash merge it :)
Thanks

Copy link
Contributor

@holiman holiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@holiman holiman added this to the 1.10.0 milestone Jan 9, 2021
@holiman holiman merged commit 89030ec into ethereum:master Jan 9, 2021
@ziogaschr ziogaschr deleted the fix/downloader-tests-race branch January 11, 2021 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants