Closed
Description
As briefly discussed here: https://twitter.com/_rsc/status/956888213314068481
I don't see why Go shouldn't cache the results of -coverprofile
when running tests, as test coverage shouldn't vary from run to run, given the same set of arguments.
What version of Go are you using (go version
)?
go version go1.10rc1 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env
)?
$GOOS="darwin"
$GOARCH="amd64"
What did you do?
Call go test -coverprofile=coverprofile.out ./...
multiple times.
What did you expect to see?
Test results cached between runs.
$ go test -coverprofile=coverprofile.out ./...
ok github.com/bbrks/tmp 2.893s coverage: 46.1% of statements
$ go test -coverprofile=coverprofile.out ./...
ok github.com/bbrks/tmp (cached)
What did you see instead?
Test results were not cached between runs.
$ go test -coverprofile=coverprofile.out ./...
ok github.com/bbrks/tmp 2.893s coverage: 46.1% of statements
$ go test -coverprofile=coverprofile.out ./...
ok github.com/bbrks/tmp 2.893s coverage: 46.1% of statements
Activity
[-]Coverage profile should be cached with tests[/-][+]cmd/go: coverage profile should be cached with tests[/+]ianlancetaylor commentedon Jan 26, 2018
CC @rsc
gopherbot commentedon Jan 30, 2018
Change https://golang.org/cl/90955 mentions this issue:
cmd/go: coverage profile use cache if the set of arguements equals
fgrosse commentedon Aug 6, 2018
I was hoping this change would make it into go1.11 but it seems its not working yet in
go1.11beta3
. Looking at the CL it seems it got stuck due to merge conflicts? Any chance we can pick up this issue and still include it in v11?bbrks commentedon Aug 6, 2018
@fgrosse Looks like it's being targeted for 1.12 now. I don't think it will be rescheduled this close to 1.11's release.
fgrosse commentedon Aug 6, 2018
nightlyone commentedon Jan 24, 2019
Is this still planned for Go1.12 ? I guess not...
82 remaining items
chudilka1 commentedon Feb 21, 2025
Right to a bull's eye from @ryancurrah
The very same case, one of our pipelines ran up to 45 mins w/o caching, and up to 10 with it. So, it is not a minor feature, it is underevaluated here.
ryancurrah commentedon Feb 27, 2025
CL 610564 got a thumbs up. Waiting on another review.
ryancurrah commentedon Mar 5, 2025
@thanm sorry to bug you again. I had to rebase my change because there was a conflict in the docs. I ordered the flags alphabetically which requires your re-review. http://go-review.googlesource.com/c/go/+/610564/16/src/cmd/go/alldocs.go
CL: https://go-review.googlesource.com/c/go/+/610564
thanm commentedon Mar 5, 2025
@ryancurrah I'll take a look. Thank you for your persistence in getting this CL moved forward, I believe it will be a valuable change to make.
cmd/go: cache coverage profile with tests
ruoibmt commentedon Apr 17, 2025
@thanm On March 6, this modification was merged to master. Do you intend to update it to the upcoming minor version of 1.23 or 1.24?
thanm commentedon Apr 18, 2025
It is not my intent to back-port this change. The general policy on back-porting is that the fix needs to address a serious problem (compiler or runtime crash, computing incorrect results, that sort of thing). Performance improvements typically don't meet those criteria. I could nominate the bug for back-porting, but I would be surprised if the release team approved it.
make test
erigontech/erigon#14710make test
erigontech/erigon#14691hihoak commentedon Jun 6, 2025
Hello guys,
Thanks a lot for resolving this issue.
I started testing this feature and encountered a problem that occurs when I use the
go test
command with the-coverpkg
flag.When coverage is collected with the
-coverpkg
flag and test result caching is enabled, the generated report may include lines that no longer exist. This happens because:Each test package attempts to collect coverage for all packages matching the
-coverpkg
pattern.If the test result is loaded from the cache, the coverage data may be outdated—especially if the test package does not directly or indirectly depend on the modified code, leaving the cache uninvalidated.
for example we have project with layout
Project layout
Files Content
some_func.go
sub.go
sum.go
some_func_test.go
sub_test.go
Coverage result of this tests
Change a bit
sub.go
sub.go
Coverage result after change
Conclusion
so there are now 2 versions of
sub.go:3.24
line in one report andproj/sub/sub.go:3.24,4.22 1 0
doesn't existas a result it can breaks up coverage tools to generate cobertura coverage
ryancurrah commentedon Jun 6, 2025
Can you provide the
go test
commands your running to get these results.hihoak commentedon Jun 6, 2025
first run
second run
ryancurrah commentedon Jun 6, 2025
My immediate thought here is to disable coverage caching when
-coverpkg
is used. Otherwise we will need to figure out how to determine if any of the packages found incoverpkg
changed. Which I'm not sure how to do. If none of these options sound good we should revert this CL.hihoak commentedon Jun 6, 2025
All of these "phantom" lines always have 0 hits; otherwise, they would be invalidated (since they would have been imported by modified code).
This fact can be used to ignore them in tools that generate reports.
That said, I agree, this behavior does seem unintuitive and non-obvious at first glance