import: git sparse-checkout

[casperdcl]

dvc import https://some/git/repo/ some_file should probably do a sparse checkout.

Currently looks like it does a full clone:

dvc/dvc/stage.py

Line 941 in 05cc023

self.deps[0].download(self.outs[0])

would call

dvc/dvc/dependency/repo.py

Lines 73 to 81 in 05cc023

	def download(self, to):
	with self._make_repo() as repo:
	if self.def_repo.get(self.PARAM_REV_LOCK) is None:
	self.def_repo[self.PARAM_REV_LOCK] = repo.scm.get_rev()
	if hasattr(repo, "cache"):
	repo.cache.local.cache_dir = self.repo.cache.local.cache_dir
	repo.pull_to(self.def_path, to.path_info)

On a related note, is the repo locally cached by default

dvc/dvc/dependency/repo.py

Lines 78 to 79 in 05cc023

	if hasattr(repo, "cache"):
	repo.cache.local.cache_dir = self.repo.cache.local.cache_dir

so that dvc update won't re-clone?

[shcheklein]

@Suor you should have more info about his I think?

[efiop]

Related to #3473

[Suor]

Local caches only used within one dvc execution or one python session.

Sparse checkout or at least shallow checkouts might be used, this will complicate out external_repo() abstraction, including its caching. But looks like there is no way around.

[casperdcl]

I think we had a discussion with @efiop and @shcheklein about which is more important to fix first (persistent repo cache or sparse checkout) and there was no clear outcome. Will open a new issue for persistent repo cache...

[skshetry]

We don't have a persistent repo checkout, but we do shallow clone these days. Closing...