Coverage inaccuracy

[bramtayl]

Apologies if there's already a corresponding open issue for this. I'm running into coverage issues. The coverage reports for

https://github.com/bramtayl/JuliennedArrays.jl

and especially https://github.com/bramtayl/LightQuery.jl

list several lines of code as uncovered, and several lines of code are considered to be not code. As far as I can tell, the true coverage statistic for both packages should be 100%, that is, if I comment out any line of code and replace it with error(), tests will fail.

[ViralBShah]

Close?

[bramtayl]

I don't think this is solved. Ref #44455 (comment)

[brenhinkeller]

Is this an issue for Julia or an issue for codecov.io?

[vtjnash]

For julia. Usually it is constant-folded values that get evaluated, but not run so they end up not getting counted

[dhanak]

I ran into the same issue with my library InlineDispatch.jl. There are 7 lines (4 here and 3 here) reported as uncovered code lines, whereas they are part of a docstring. There's also a line in the body of the macro which is shown as uncovered, but it is most definitely covered, otherwise the tests wouldn't work (here)

This results in the coverage being reported as 58%, whereas it should be 100%.

Note that this is not a codecov issue, the same anomaly can be observed in the cov files generated by Pkg.test(coverage = true).

[MilesCranmer]

This popped up on my Google search so I just wanted to document my workaround. There is a (fairly new) directive in Coverage.jl to disable coverage for some code:

To exclude specific code blocks, surround the section with COV_EXCL_START and COV_EXCL_STOP comments:

# COV_EXCL_START
foo() = nothing
# COV_EXCL_STOP

To exclude a single line, add a comment with COV_EXCL_LINE:

const a = 1 # COV_EXCL_LINE

[maleadt]

a (fairly new) directive in Coverage.jl

This was added in JuliaCI/CoverageTools.jl#35, which is almost 5 years old, so I wouldn't call it fairly new.

[MilesCranmer]

Oops! Newly discovered by me