Skip to content

Coverage for binary tests #1489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
4 tasks
LesnyRumcajs opened this issue Mar 21, 2022 · 2 comments
Closed
4 tasks

Coverage for binary tests #1489

LesnyRumcajs opened this issue Mar 21, 2022 · 2 comments
Labels
CI & Testing CLI Priority: 4 - Low Limited impact and can be implemented at any time

Comments

@LesnyRumcajs
Copy link
Member

Issue summary

Coverage reports are missing runs where the forest binary is executed directly, e.g. cli tests. Some attempts were made with adding --follow-exec to tarpaulin (perhaps the executed binary was not the instrumented one) or creating explicitly an instrumented binary (which was severely timing out, >20mins).

Task summary

  • Investigate how we can make binary tests with coverage check (most likely it should be one binary for most cases, except where some fancy features are used)
  • Implement the change

Acceptance Criteria

  • main is at least partially covered
  • coverage step does not take significantly more time than before

Other information and links

assert-rs/assert_cmd#57
https://docs.rs/escargot/latest/escargot/
xd009642/tarpaulin#105

An attempt at building instrumented binary once for all tests

lazy_static!(
    static ref FOREST_BINARY: CargoRun = {
    escargot::CargoBuild::new()
        .bin("forest")
        .current_release()
        .current_target()
        .run()
        .unwrap()
    };
);

// then use with: let cmd = Command::new(FOREST_BINARY.path())
@LesnyRumcajs LesnyRumcajs added Priority: 4 - Low Limited impact and can be implemented at any time CLI CI & Testing labels Mar 21, 2022
@lemmih
Copy link
Collaborator

lemmih commented Jun 21, 2022

This might be solvable once #1558 lands.

@lemmih
Copy link
Collaborator

lemmih commented Jul 11, 2022

cargo llvm-cov magically does the right thing. We have good coverage and it doesn't significantly impact testing time.

@lemmih lemmih closed this as completed Jul 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI & Testing CLI Priority: 4 - Low Limited impact and can be implemented at any time
Projects
None yet
Development

No branches or pull requests

2 participants