Skip to content

is it possible to run more than one dvc pipeline in the same dvc context? #2851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
aabramovrepo opened this issue Nov 26, 2019 · 4 comments
Closed
Labels
triage Needs to be triaged

Comments

@aabramovrepo
Copy link

Hi,

I'm trying to run multiple dvc pipelines (trainings) on the same data in the same dvc context. The only difference is .dvc config file. I'm getting the following error message:

ERROR: cannot perform the cmd since DVC is busy and locked. Please retry the cmd later.

Is it somehow possible to start multiple dvc processes at the same time?

Thanks!

Best,
Alexey

@ghost ghost added the triage Needs to be triaged label Nov 26, 2019
@efiop
Copy link
Contributor

efiop commented Nov 26, 2019

Hi @aabramovrepo !

The issue is that dvc is not able to run multiple instances at the same time #755 . It is a known issue and we are working on a solution in #2584 , which should hopefully be done soon-ish. As a workaround, you could try re-running individual long-running commands by-hand and then dvc commit-ing the result.

I'll close this for now as a duplicate of #755

@efiop efiop closed this as completed Nov 26, 2019
@aabramovrepo
Copy link
Author

@efiop: thanks for your response, we are waiting then...

@free-soellingeraj
Copy link

Is this resolved?

@efiop
Copy link
Contributor

efiop commented Mar 2, 2021

@free-soellingeraj We are releasing locks since #2584 Are you running into any particular issues?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage Needs to be triaged
Projects
None yet
Development

No branches or pull requests

3 participants