Skip to content

Leverage the --workspace flag when running cargo command against cargo itself #11987

@weihanglo

Description

@weihanglo
Member

Problem

After #11851 Cargo becomes a Cargo workspace. That means we can dogfood ourselves finally 🎉.

We left some TODOs in that pull request, one of them are making --workspace usable for cargo itself. That is, making cargo build and other subcommands happy with --workspace flag, instead of using -p flag everywhere in our CI workflow yaml.

Some workspace members are platform-dependant inside cargo workspace, i.e., cargo-credential-gnome-secret and friends. We don't really have a good programming way to get the list other than cargo metadata | jq .workspace_members[] -r | cut -d' ' -f1. As a consequence, we expands the list manually in a ugly way:

- run: cargo test -p cargo-test-support
- run: cargo test -p cargo-platform
- run: cargo test -p cargo-util
- run: cargo test -p home
- run: cargo test -p mdman
- run: cargo build -p cargo-credential-1password
- run: cargo build -p cargo-credential-gnome-secret
if: matrix.os == 'ubuntu-latest'
- run: cargo build -p cargo-credential-macos-keychain
if: matrix.os == 'macos-latest'
- run: cargo build -p cargo-credential-wincred
if: matrix.os == 'windows-latest'

We want to make running command like cargo test --workspace as easy as possible to reduce the friction in CI also in contributor experience.

Proposed Solution

In the long term, we could look into #5220 and also the comment here #11987 (comment).

An alternative workaround in the short term would be making those subcrates and dependencies platform specific via conditional compilation. For example, dependencies could be listed in target."cfg(not(windows))".dependencies section. And have a lib.rs doing nothing but delegating to unix/lib.rs or windows/lib.rs with #[cfg(…)], and one of them also does nothing.

Notes

I think at least the follow commands should be able to run with --workspace for cargo itself.

  • cargo build/cargo check
    cargo test
    cargo doc
    cargo clippy (this is blocked on RFC: Add a [lints] table to Cargo.toml rfcs#3389, so we can more easily propagate our clippy settings)

Activity

added
C-feature-requestCategory: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`
on Apr 17, 2023
epage

epage commented on Apr 17, 2023

@epage
Contributor

I feel like I'm missing the distinction between #11526 and #5220, they look like duplicates to me. I also feel that conceptually, conditional workspace members are a bad idea and both should be rejected. As the syntax is laid out, I read it as cargo resolving differently depending on the platform which means the lockfile is no longer cross-platform.

For me, I'd like a variant of #9406 which is instead package.required-target which behaves like <build-target>.required-features in that we automatically skip those packages if we aren't building for the right target. We could also skip resolving any target-specific dependencies that do not match package.required-target or package.forced-target, helping in cases like #7058 (and the related minimal lockfile issues).

#9406 and required-target remind me of rust-lang/rfcs#3374 and required-features.

That said, I see this as short-term and long-term. Longer-term, we should do something like this but we likely don't want to wait on design and stablizxation, so we should do conditional builds in the short term.

added and removed
C-feature-requestCategory: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`
on Apr 17, 2023
weihanglo

weihanglo commented on Apr 17, 2023

@weihanglo
MemberAuthor

Totally valid. I was waiting for a response from #11526 but no. Closed.

And yep, clarified the short-term and long-term fixes in the PR descriptions. Thanks for the reply!

added a commit that references this issue on Apr 18, 2023
added
S-acceptedStatus: Issue or feature is accepted, and has a team member available to help mentor or review
A-infrastructureArea: infrastructure around the cargo repo, ci, releases, etc.
and removed on Apr 24, 2023
heisen-li

heisen-li commented on Apr 26, 2023

@heisen-li
Contributor

I'll try to solve the cargo test --workspace for cargo itself.

NanthR

NanthR commented on Jun 11, 2023

@NanthR

Hey, I wanted to work on this. I just had a couple of questions before I get started.

package.required-target which behaves like <build-target>.required-features

When you say target, this is talking about platform targets, correct?

weihanglo

weihanglo commented on Jun 11, 2023

@weihanglo
MemberAuthor

When you say target, this is talking about platform targets, correct?

Yes. Please read relevant issues to track their progress.

This pull request is more like blocked on other pull requests or features. For exammple, I don't think there is an easy way to skip cargo-credential-gnome-secret without explicit exclusion, like cargo test --workspace --exclude "cargo-credential-gnome-secret". I don't think there is much we can do at this moment. Maybe only focus on reducing CI yaml? Anyway, feel free to drop your idea.

NanthR

NanthR commented on Jun 11, 2023

@NanthR

Ok. Thanks. I was just thinking of adding an option to the package field to set the required platforms and having that be the filter of whether a package needs to be built. I read through the previous suggestions, and from my understanding, this is what seems to be the expected solution?

weihanglo

weihanglo commented on Jun 11, 2023

@weihanglo
MemberAuthor

Thanks for being interested in this! However required-targets is just one of the proposed solutions out there. We haven't decided on a proper way to solve this issue.

Note that this is labeled as A-infrastructure, meaning that it focuses on making cargo the project infra better with existing functionality in cargo. If we want to resolve this issue with new features, please discuss in their own issues respectively, or open a new issue for a new feature proposal.

Sorry for the confusion 😔.

added
S-needs-designStatus: Needs someone to work further on the design for the feature or fix. NOT YET accepted.
and removed
S-acceptedStatus: Issue or feature is accepted, and has a team member available to help mentor or review
on Jun 11, 2023
added 3 commits that reference this issue on Jul 11, 2023
weihanglo

weihanglo commented on Nov 27, 2023

@weihanglo
MemberAuthor

This should be resolved by [lints] stabilization and conditional compilation for cargo-credential-* crates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-building-cargo-itselfArea: issues with building cargoA-infrastructureArea: infrastructure around the cargo repo, ci, releases, etc.A-testing-cargo-itselfArea: cargo's testsC-enhancementCategory: enhancementE-mediumExperience: MediumS-needs-designStatus: Needs someone to work further on the design for the feature or fix. NOT YET accepted.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @epage@weihanglo@heisen-li@NanthR

        Issue actions

          Leverage the `--workspace` flag when running cargo command against cargo itself · Issue #11987 · rust-lang/cargo