Skip to content

cloud versioning: external outputs #8411

Closed as not planned
Closed as not planned
@dberenbaum

Description

@dberenbaum

For a cloud bucket/container with versioning turned on, external outputs can work like cloud-versioned remotes to simplify the workflow. No external cache is needed since a cloud-versioned external output path can serve as its own remote/cache. DVC should only need to collect the version info to store in .dvc or dvc.lock. This also can make external outputs much more performant because no copies or md5s are needed.

Question: Should dvc checkout update the current versions of external outputs? This should probably be configurable, but it can be left for a follow-up task.

Out of scope: Isolation of external output paths from race conditions and conflicts when multiple users are writing to the same paths. Let's improve the current functionality before worrying about this, which we could address either with something like #3920 or adding locks on the cloud.

Metadata

Metadata

Assignees

No one assigned

    Labels

    p2-mediumMedium priority, should be done, but less important

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions