run: add --outs-persist and --outs-persist-no-cache options #1759
Conversation
pared
changed the title
pared
force-pushed
the
1214
branch
3 times, most recently
from
e6c32eb to
321c661
Compare
pared
changed the title
pared
force-pushed
the
1214
branch
2 times, most recently
from
6b705dd to
5f9c12a
Compare
pared
changed the title
dvc/stage.py
Outdated
| @@ -260,7 +260,10 @@ def remove_outs(self, ignore_remove=False): | |||
| Used mainly for `dvc remove --outs` | |||
| """ | |||
| for out in self.outs: | |||
| out.remove(ignore_remove=ignore_remove) | |||
| if out.persist: | |||
| self.repo.unprotect(out.path) | |||
There was a problem hiding this comment.
This will lead to a bug in non-local out case. What we need to do is implement remote.unprotect and out.unprotect(that would call remote.unprotect). And also use remote.unprotect in repo.unprotect.
dvc/remote/local.py
Outdated
| @@ -743,3 +744,6 @@ def _log_missing_caches(self, checksum_info_dict): | |||
| "nor on remote. Missing cache files: {}".format(missing_desc) | |||
| ) | |||
| logger.warning(msg) | |||
|
|
|||
| def unportect(self, target): | |||
| unprotect(target) | |||
There was a problem hiding this comment.
This is backwards. Move repo.unprotect logic to RemoteLOCAL.unprotect and use it.
There was a problem hiding this comment.
Please move logic from repo.unprotect() to RemoteLOCAL.unprotect() and use RemoteLOCAL.unprotect in repo.unprotect().
dvc/remote/local.py
Outdated
| @@ -743,3 +744,6 @@ def _log_missing_caches(self, checksum_info_dict): | |||
| "nor on remote. Missing cache files: {}".format(missing_desc) | |||
| ) | |||
| logger.warning(msg) | |||
|
|
|||
| def unportect(self, target): | |||
| unprotect(target) | |||
There was a problem hiding this comment.
Please move logic from repo.unprotect() to RemoteLOCAL.unprotect() and use RemoteLOCAL.unprotect in repo.unprotect().
pared
force-pushed
the
1214
branch
2 times, most recently
from
b4f511d to
4a3d0e6
Compare
|
|
||
| def unprotect_outs(self): | ||
| for out in self.outs: | ||
| if out.scheme != "local" or not out.exists: |
There was a problem hiding this comment.
Previously we were simply ignoring an output if it didn't exist, but now unprotect() will raise an exception. This will break dvc repro when output doesn't exist.
There was a problem hiding this comment.
This functionality still exists, I moved it to output base class.
There was a problem hiding this comment.
Ah, indeed! Sorry, didn't catch that 🙂
efiop
changed the title
|
I just realize this was never documented 😋 iterative/dvc.org/issues/2027 |
|
There was a PR, we didn't merge it to keep this option hidden for now. |
|
Wait I was wrong. This is documented in https://dvc.org/doc/command-reference/run#options, it's just missing from the dvc.yaml doc (explain well what "persist" means). |
Fixes #1214