specify remote when adding file to dvc repo

[christian-steinmeyer]

Currently, one can add a file to dvc via dvc add some.file which will create a tracking file some.file.dvc. In this tracking file, one can add remote: some_remote to the fields, to tell dvc which remote to synchronize with (cf. docs). I would like to be able to do that from the command line per some argument to dvc add. The command already has argument -r/--remote (although currently only allowed in combination with --to-remote. It'd be great if it could be used for this purpose.

What I'm suggesting in code:
dvc add test.txt --remote foo should yield a file with this content

outs:
- md5: <SOME_HASH>
  size: <SOME_SIZE>
  path: test.txt
  remote: foo

[dberenbaum]

@christian-steinmeyer Do you have any interest in trying to contribute it?

[christian-steinmeyer]

I've had a first look in how the arguments are parsed and where the add method of the repo is defined, but I dont know what mechanism creates the actual files and how their content is defined.

[dberenbaum]

I think it should work similarly to annotation fields like desc that persist unless a new value is explicitly passed, but remote should probably be separate since it's more than an annotation.

The annotation info is added here AFAIK:

dvc/dvc/repo/add.py

Line 269 in 787c72e

out_obj.annot.update(**kwargs)

@iterative/dvc WDYT?

[daavoo]

I think it should be better passed to:

dvc/dvc/repo/add.py

Lines 259 to 266 in 787c72e

	stage = repo.stage.create(
	single_stage=True,
	validate=False,
	fname=fname or path,
	wdir=wdir,
	outs=[out],
	external=external,
	)

And pass it all the way down to Output init class:

dvc/dvc/output.py

Lines 316 to 335 in 787c72e

	def __init__( # noqa: PLR0913
	self,
	stage,
	path,
	info=None,
	cache=True,
	metric=False,
	plot=False,
	persist=False,
	checkpoint=False,
	desc=None,
	type=None, # noqa: A002, pylint: disable=redefined-builtin
	labels=None,
	meta=None,
	remote=None,
	repo=None,
	fs_config=None,
	files: Optional[List[Dict[str, Any]]] = None,
	push: bool = True,
	):

[dberenbaum]

Sounds good, @daavoo. My only point about annotations was that if someone does dvc add --remote myremote data and later commits some changes with dvc add data, data.dvc should keep the remote: myremote field.

[daavoo]

My only point about annotations was that if someone does dvc add --remote myremote data and later commits some changes with dvc add data, data.dvc should keep the remote: myremote field.

I am a little lost without actually going into the code, but shouldn't remote be also preserved in that scenario?

[daavoo]

My only point about annotations was that if someone does dvc add --remote myremote data and later commits some changes with dvc add data, data.dvc should keep the remote: myremote field.

I am a little lost without actually going into the code, but shouldn't remote be also preserved in that scenario?

I think it is preserved and it's tested here:

dvc/tests/func/test_add.py

Lines 846 to 880 in b814531

	def test_add_preserve_fields(tmp_dir, dvc):
	text = textwrap.dedent(
	"""\
	# top comment
	desc: top desc
	outs:
	- path: foo # out comment
	desc: out desc
	type: mytype
	labels:
	- label1
	- label2
	remote: testremote
	meta: some metadata
	"""
	)
	tmp_dir.gen("foo.dvc", text)
	tmp_dir.dvc_gen("foo", "foo")
	assert (tmp_dir / "foo.dvc").read_text() == textwrap.dedent(
	"""\
	# top comment
	desc: top desc
	outs:
	- path: foo # out comment
	desc: out desc
	type: mytype
	labels:
	- label1
	- label2
	remote: testremote
	md5: acbd18db4cc2f85cedef654fccc4a4d8
	size: 3
	meta: some metadata
	"""
	)

[kshitiz305]

Hi So shall we add this?

[christian-steinmeyer]

I got as far as this 😅

Index: dvc/repo/add.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/dvc/repo/add.py b/dvc/repo/add.py
--- a/dvc/repo/add.py	(revision b814531c84071823d83194af6f91d5718fc26976)
+++ b/dvc/repo/add.py	(date 1682085738793)
@@ -72,9 +72,7 @@
             invalid_opt = "--external option"
     else:
         message = "{option} can't be used without --to-remote"
-        if kwargs.get("remote"):
-            invalid_opt = "--remote"
-        elif kwargs.get("jobs"):
+        if kwargs.get("jobs"):
             invalid_opt = "--jobs"
 
     if invalid_opt is not None:
@@ -255,6 +253,7 @@
         path, wdir, out = resolve_paths(
             repo, target, always_local=transfer and not kwargs.get("out")
         )
+        remote = kwargs.get("remote")
 
         stage = repo.stage.create(
             single_stage=True,
@@ -263,6 +262,7 @@
             wdir=wdir,
             outs=[out],
             external=external,
+            remote=remote
         )
 
         out_obj = stage.outs[0]

Not sure where it would need to go next.

[kshitiz305]

have you pushed these changes on a branch ?

[christian-steinmeyer]

No, but feel free to just apply the git patch I posted