dvc: dvc_walk() and related things are broken by design

[Suor]

A few things:

• when we collect stages in a branch dvcignore is not used at all, i.e. we go into .dvc and other ignored dirs
• dvc_walk() and walk_files() always walk the working dir, but might accept dvcignore built from a branch. The results will make no sense.

We may approach this from two sides:

• wrap dvcignore into a tree, i.e. dvctree = TreeWithIgnores(some_tree); dvctree.walk(...).
• check that we only use dvcignore built from working dir if we os.walk().

[efiop]

when we collect stages in a branch dvcignore is not used at all, i.e. we go into .dvc and other ignored dirs

It is used https://github.com/iterative/dvc/blob/master/dvc/repo/__init__.py#L401 . Could you clarify?

[Suor]

Not used in GitTree.walk()

dvc/dvc/scm/git/tree.py

Lines 139 to 152 in ee70822

	def walk(self, top, topdown=True, dvcignore=None):
	"""Directory tree generator.
	See `os.walk` for the docs. Differences:
	- no support for symlinks
	- it could raise exceptions, there is no onerror argument
	"""
	tree = self.git_object_by_path(top)
	if tree is None:
	raise IOError(errno.ENOENT, "No such file")
	for x in self._walk(tree, topdown):
	yield x

[efiop]

@Suor Good catch! CC @pared

@Suor, what do you mean by a tree?

[efiop]

@MrOutis trees that we use. E.g. dvc/dvc/scm/git/tree.py

thanks, @efiop

[pared]

Confirmed, dvcignore has not been properly working for branches since day 1.
#1820