-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Open
Labels
C-feature-requestCategory: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`Command-testS-blocked-externalStatus: ❌ blocked on something out of the direct control of the Cargo project, e.g., upstream fixStatus: ❌ blocked on something out of the direct control of the Cargo project, e.g., upstream fix
Description
Problem
When a crate has no unit tests (via test = false
in its manifest), I would expect cargo to never ever try to run unit tests for that crate.
And yet it turns out that cargo test --lib
will run the unit tests.
Steps
- Run
cargo test --lib --bins --examples --tests --benches
(in an attempt to run "everything except for doc tests"), or justcargo test --lib
- Note how even if the library has unit tests are disabled, cargo will run them
Running unittests src/lib.rs (build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/core-04ceb2ced8c36de5)
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 7.54µs
Possible Solution(s)
--lib
should just be a filter, removing from the usual test set everything that's not a "library unit test". It should not add things that were not present to begin with.
Notes
This is related to but different from #10936
Version
No response
Metadata
Metadata
Assignees
Labels
C-feature-requestCategory: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`Command-testS-blocked-externalStatus: ❌ blocked on something out of the direct control of the Cargo project, e.g., upstream fixStatus: ❌ blocked on something out of the direct control of the Cargo project, e.g., upstream fix
Type
Projects
Status
No status
Milestone
Relationships
Development
Select code repository
Activity
RalfJung commentedon Mar 29, 2024
Something similar seems to happen with benchmarks. With a manifest like this
Running
cargo test
will only run the first benchmark, but runningcargo test --lib --bins --examples --tests --benches
will run both -- even though it's not a test and shouldn't be run as a test.weihanglo commentedon Mar 29, 2024
This seems to be covered by rust-lang/testing-devex-team#1> Once we have a new JSON output format, Cargo could get more controls over tests than relying on libtest.
RalfJung commentedon Mar 29, 2024
I don't think this issue has anything to do with handling the output from the tests. It's all about which tests cargo decides to run.
weihanglo commentedon Mar 29, 2024
I see. Sorry I understood it wrong.
--lib
is a flag for user to explicitly requests tests in library target to be run, so that is expected. So does other flags like--test
,--bench
, etc.test = false
controls only the default set of tests being run. See the target selection section.weihanglo commentedon Mar 29, 2024
The proposed solution can only be solved with rust-lang/testing-devex-team#1, from my understanding, because Cargo needs to run libtest to know if there is any unit test defined.
RalfJung commentedon Mar 29, 2024
That's very surprising... but it explains a lot, thanks.
Auto merge of #3430 - RalfJung:doc, r=RalfJung
Auto merge of rust-lang#3430 - RalfJung:doc, r=RalfJung