-
Notifications
You must be signed in to change notification settings - Fork 18k
cmd/go: go test -cover & go test -coverprofile should always output a coverage #24570
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Change https://golang.org/cl/115095 mentions this issue: |
#25492 is about the case where there are no statements to cover (and thus the percentage is undefined). That's materially different from the case here, where |
My reading of this issue is that |
That's the current issue title, but that behavior seems clearly wrong if there are no statements to cover: (I guess we could output |
I like the idea of a consistent output that would be created by just implementing this issue. Another approach is, as in #25492, for all statements it is true that they are covered by the tests. So one could argue for 100% which is obviously mathematically incorrect. I am also the creator of the CL. Happy to implement a solution that results from the discussion :) |
This should fix test failures that were being caused by golang/go#24570, which caused test binaries to be built even when there are no test files. The poor interaction there arose from the setting of flags in our init() function, it seems.
This should fix test failures that were being caused by golang/go#24570, which caused test binaries to be built even when there are no test files. The poor interaction there arose from the setting of flags in our init() function, it seems.
Change https://golang.org/cl/122518 mentions this issue: |
Original CL description: When using test -cover or -coverprofile the output for "no test files" is the same format as for "no tests to run". Reverting because this CL changed cmd/go to build test binaries for packages that have no tests, leading to extra work and confusion. Updates #24570 Fixes #25789 Fixes #26157 Fixes #26242 Change-Id: Ibab1307d39dfaec0de9359d6d96706e3910c8efd Reviewed-on: https://go-review.googlesource.com/122518 Run-TryBot: Ian Lance Taylor <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Russ Cox <[email protected]>
CL 115095 was reverted, so reopening this issue. |
Any news here? |
One can use
Without
With
|
Using So, say package However, if we have a package
Additionally, it is misleading to say that because |
I'm not sure this is true. Here is the code from the example above:
as you can see nothing is calling |
I don’t know exactly what is different, but even with
While adding just a single Meanwhile, if I add a reference to
|
Change https://go.dev/cl/495452 mentions this issue: |
Change https://go.dev/cl/495446 mentions this issue: |
Change https://go.dev/cl/495447 mentions this issue: |
Introduce a new mode of execution for instrumenting packages that have no test files. Instead of just skipping packages with no test files (during "go test -cover" runs), the go command will invoke cmd/cover on the package passing in an option in the config file indicating that it should emit a coverage meta-data file directly for the package (if the package has no functions, an empty file is emitted). Note that this CL doesn't actually wire up this functionality in the Go command, that will come in a later patch. Updates #27261. Updates #58770 Updates #24570. Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest Change-Id: I01e8a3edb62441698c7246596e4bacbd966591c3 Reviewed-on: https://go-review.googlesource.com/c/go/+/495446 Reviewed-by: Bryan Mills <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>
This patch fixes some problems with how "go test -cover" was handling tests involving A) multiple package tests and B) multiple packages matched by "-coverpkg". In such scenarios the expectation is that the percent statements covered metric for each package needs to be relative to all statements in all packages matched by the -coverpkg arg (this aspect of the reporting here was broken as part of GOEXPERIMENT=coverageredesign). The new scheme works as follows. If -coverpkg is in effect and is matching multiple packages, and we have multiple test targets, then: - each time a package is built for coverage, capture a meta-data file fragment corresponding to just the meta-data for that package. - create a new "writeCoverMeta" action, and interpose it between the build actions for the covered packages and the run actions. The "writeCoverMeta" action at runtime will emit a file "metafiles.txt" containing a table mapping each covered package (by import path) to its corresponding meta-data file fragment. - pass in the "metafiles.txt" file to each run action, so that when the test finishes running it will have an accurate picture of _all_ covered packages, permitting it to calculate the correct percentage. Concrete example: suppose we have a top level directory with three package subdirs, "a", "b", and "c", and from the top level, a user runs "go test -coverpkg=./... ./...". This will result in (roughly) the following action graph: build("a") build("b") build("c") | | | link("a.test") link("b.test") link("c.test") | | | run("a.test") run("b.test") run("c.test") | | | print print print With the new scheme, the action graph is augmented with a writeCoverMeta action and additional dependence edges to form build("a") build("b") build("c") | \ / | / | | v v | / | | writecovmeta<-|-------------+ | | ||| | | | ||\ | | link("a.test")/\ \ link("b.test") link("c.test") | / \ +-|--------------+ | | / \ | \ | | v v | v | run("a.test") run("b.test") run("c.test") | | | print print print A note on init functions: prior to GOEXPERIMENT=coverageredesign the "-coverpkg=..." flag was implemented by force-importing all packages matched by "-coverpkg" into each instrumented package. This meant that for the example above, when executing "a.test", the init function for package "c" would fire even if package "a" did not ordinarily import package "c". The new implementation does not do this sort of forced importing, meaning that the coverage percentages can be slightly different between 1.21 and 1.19 if there are user-written init funcs. Fixes #58770. Updates #24570. Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest Change-Id: I7749ed205dce81b96ad7f74ab98bc1e90e377302 Reviewed-on: https://go-review.googlesource.com/c/go/+/495452 Reviewed-by: Bryan Mills <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>
@thanm The commit d1cb5c0 cause regression which makes "go tool cover -html=cover.out -o cover.html" return an error if a package called "internal" in root module does not have test files. This can be reproduced by running "make" on these Go repository: https://github.com/shuLhan/share/ Using go tip 1e69040 (one commit before d1cb5c0),
Using go tip d1cb5c0,
Using go tip 638e0d3,
|
Thanks for the report. Could you possibly open another issue for this? Thanks. |
Update: I will go ahead and file an issue for this, I think I see what the problem is. |
#63356 filed, please follow along on that issue if you want to track the fix. |
Change https://go.dev/cl/532555 mentions this issue: |
As of CL 495447 we now synthesize coverage data (including coverage profiles) for packages that have no tests, if they are included in a "go test -cover" run. The code that set up the "run" actions for such tests wasn't setting the objdir for the action, which meant that the coverage profile temp file fragment ("_cover_.out") was being created in the dir where the test was run, and in addition the same fragment could be written to by more than one package (which could lead to a corrupted file). This CL updates the code to properly set the objdir, and to create the dir when needed. Updates #24570. Fixes #63356. Change-Id: Iffe131cf50f07ce91085b816a039308e0ca84776 Reviewed-on: https://go-review.googlesource.com/c/go/+/532555 Reviewed-by: Russ Cox <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>
This change is a bit unfortunate, because with <go1.22 one could run This used to be consistent, where "no test files" regardless of -cover and regardless of function bodies literally meant just that: "no test files". But now if I have function bodies and run |
I can see your point, however there are other easy ways of determining that a package has no tests, e.g. If on the other hand I run
to identify code that isn't covered, I would argue that what most folks want is to see all of their uncovered functions, not just the functions in subdirectories where someone wrote tests. |
Resolves sourcenetwork#672 **Description:** Ensures the empty packages aren't left out (because previously packages with 0 tests won't be included in the coverage). This should help avoid seeing the random coverage drops we see in PRs that are actually introducing tests. __Moreover__ - Removes the make rule `make test:coverage-quick`. - Change make rule `make test:coverage-full` to `make test:coverage` - Adds the ability to also provide `path=...` to `make test:coverage` like we had for `make test:coverage-html`. - Account for integration test coverage for the rule `make test:coverage-html`. __Limitation__ Using `-coverpkg` does not actually help if the package is never called in any test at all. `-coverpkg` is a cover-up that allows packages from a different package to provide coverage for this package. So, say package `a` has no tests, and thus no coverage. But package `b` has tests, and those tests call into `a`. As a result, if you start using a `-coverpkg` that covers both packages, the tests from package `b` will report its coverage of package `a` as coverage for `a`. However, if we have a package `c` that is never called by either `a` or `b` then that package will still be left out in the dry, even if it would match the `-coverpkg` given. Reference: golang/go#24570
Description
Now in 1.10 when
go test -cover
supports multiple packages, I would expect it to print out a percentage for all packages (including those missing tests).And for
go test -coverprofile
, I would expect all packages to be included in the calculated total.Currently only packages that have at least one test (can be a
*_test.go
with only thepackage
declaration) is included, seepkg2
below.What version of Go are you using (
go version
)?go version go1.10 linux/amd64
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (
go env
)?Linux, amd64
What did you do?
go test ./... -cover
go test ./... -coverprofile cover.out; go tool cover -func cover.out
What did you expect to see?
What did you see instead?
The text was updated successfully, but these errors were encountered: