Skip to content

Profiling report with branch coverage for templates is misleading #111743

Open
@stma247

Description

@stma247
Contributor

The export mode of tool "llvm-cov" for format "lcov" does not combine the branch counts of C++ template functions instantiations.

A preceding discussion describes the issue in detail:
https://discourse.llvm.org/t/profiling-report-with-branch-coverage-for-templates-is-misleading/82323

Activity

stma247

stma247 commented on Oct 9, 2024

@stma247
ContributorAuthor

I am already working on a solution. The PR will follow soon.

chapuni

chapuni commented on Nov 21, 2024

@chapuni
Contributor

Thanks for working on it.
Could it be applied to text and html as well?

evodius96

evodius96 commented on Nov 21, 2024

@evodius96
Contributor

Thanks for working on it. Could it be applied to text and html as well?

This is only really for LCOV exporting. The text and html from llvm-cov show summaries show the right information.

chapuni

chapuni commented on Nov 21, 2024

@chapuni
Contributor

From llvm/test/tools/llvm-cov/branch-templates.cpp

   12|       |template<typename T>
   13|      3|int func(T x) {
   14|      3|  if(x)       // CHECK: |  Branch ([[@LINE]]:6): [True: 0, False: 1]
  ------------------
  |  Branch (14:6): [True: 0, False: 1]
  |  Branch (14:6): [True: 1, False: 0]
  |  Branch (14:6): [True: 0, False: 1]
  ------------------
   15|      1|    return 0; // CHECK: |  Branch ([[@LINE-1]]:6): [True: 1, False: 0]
   16|      2|  else        // CHECK: |  Branch ([[@LINE-2]]:6): [True: 0, False: 1]
   17|      2|    return 1;
   18|      0|  int j = 1;
   19|      0|}

I'll investigate further later.

evodius96

evodius96 commented on Nov 21, 2024

@evodius96
Contributor

Hi @chapuni

Sorry, I didn't elaborate -- this is primarily an effect of the summary report. With llvm-cov show the summary report correctly merges the data across the corresponding instantiations showing the union of the data, whereas with llvm-cov export --format=lcov it was not correctly rendering the union with BRDA. As a result, LCOV was not displaying the summary correctly.

The source-level report that you show shows the individual branch coverage information across the instantiations, but the report summary for this should have merged this data and ought to show 100% coverage for the condition.

19 remaining items

bartlettroscoe

bartlettroscoe commented on Apr 8, 2025

@bartlettroscoe

Thanks for working on it. Could it be applied to text and html as well?

This is only really for LCOV exporting. The text and html from llvm-cov show summaries show the right information.

Yes, but I will argue you still want to have the overall coverage metrics for the template function, in addition to its specific instantiations. The way llvm-cov show, for example, displays coverage information for template functions with lots of instantiations is about worthless (you can't see the forest for the trees). I will post an issue about how to fix this shortly (i.e. wrapping details for the instantiations in <details> ... </details> blocks). And for that, we will need combined branch metrics for the template function across all instantiations.

Update: I posted:

describing the problem with the way that llvm-cov show generates HTML coverage files and how to easily fix the problem.

But we would need to compute the cumulative branch coverage metrics for a template function over all of its instantaitions in order to implement #134927.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @chapuni@bartlettroscoe@EugeneZelenko@evodius96@stma247

        Issue actions

          Profiling report with branch coverage for templates is misleading · Issue #111743 · llvm/llvm-project