Is it possible to specify a directory to exclude from coverage?

[abe545]

Hi, we would love to start using coverlet to generate code coverage, but we really don't want to include code that gets automatically generated in our coverage results. Since we generate it during CI, it wouldn't be feasible for us to apply attributes to every class that we want to exclude from coverage. We do, however, have a common folder in which the code gets generated.

Is there a flag we can pass to exclude the files in said folder from coverage?

[ryanmvt]

We're also using gRPC code generation which makes the attribute-based exclusion feature unrealistic. A configuration or flag based solution would really help out here.

[abe545]

@tonerdo the syntax suggested in #55 looks like it could be a winner - if that were a globbing pattern (and we'd have to support multiple globs). My suggestion would be a semicolon separated list of globs (probably have to quote it)

/p:ExcludeFromCoverage="some/**/glob;c:\some\complex rooted path\*.glob"

I think it would need to support both absolute and relative paths. Thoughts?

[tonerdo]

I totally agree! My only thought is whether to limit this feature to just whole assemblies or to extend it to allow specifying namespaces/classes/methods and how that will work.

[MarcoRossignoli]

In my scenario could be very useful also to have an "include" switch(a lot of auxiliary class i don't test). I like the idea of json config file(in addition) because we can address both. Another concern is the case with a lot of include/exclude, command line could become hard to use and i don't like the idea to organize code for coverage, again this isn't possible on existing code. My idea is to chose the code i want to test with minimum % of coverage or fail for new or existing code.

[abe545]

@tonerdo I think the easiest would be path based. This will cover the majority of the scenarios, and devs can use the attribute method to exclude methods still. I don't see a scenario where I'd want to cover most code generated code, but not one particular method. At the very least, I'd expect such code to be generated in its own file.

@MarcoRossignoli I think the ability to both include and exclude is valuable. I think if we help out with implementing an msbuild switch, it would be easy to include an msbuild xml file in your project (via project includes), which could define these in a file for you.

[ryanmvt]

@abe545 Path-based meaning everything inside the directory specified in the path, as well as all sub-directories? That idea would be flexible enough to encompass generated code perfectly. It also allows you to target large swaths of code or down to a single file at a time.

@tonerdo I don't think assembly-only based exclusions would be flexible enough. It would potentially force the developer to create additional projects specifically to exclude portions of code from code coverage right?

Even the config I would think would end up path-based too? What do you guys think?

[abe545]

@ryanmvt yes... except I was thinking to support globbing patterns. If you don't specify any wildcards, it would indeed exclude the directory. That way, you could even do something like **/*test*.cs and exclude all files with the word test in the filename.

[tonerdo]

@abe545 @ryanmvt all good ideas. Here's what my final thoughts are:

• Path based ignore with support for globbing patterns
• Keep ExcludeFromCoverage attribute for fine grained control over ignoring classes/methods

Is this a good setup?

[abe545]

@tonerdo sounds good to me

[tonerdo]

Awesome! Now who's gonna open that PR 😄?

[abe545]

My coworker @ido-namely has volunteered, if nobody else wants it.

[ryanmvt]

@tonerdo sounds good to me as well! Thank you for the volunteer @abe545