PermissionError on file which should be in .dvcignore.

[dreaquil]

Bug Report

Getting a permission denied when I try to DVC add a file that is in my dvc ignore.
This is the .dvcignore file:

*/*/.thumbnail/
*/*/.exports/

Last few errors in traceback:

File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/stage/__init__.py", line 413, in commit
    out.commit()
  File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/output/base.py", line 281, in commit
    self.cache.save(self.path_info, self.cache.tree, self.info)
  File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/remote/base.py", line 1154, in save
    return self._save(path_info, tree, hash_, save_link, **kwargs)
  File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/remote/base.py", line 1161, in _save
    return self._save_dir(path_info, tree, hash_, save_link, **kwargs)
  File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/remote/base.py", line 1133, in _save_dir
    entry_info, tree, entry_hash, save_link=False, **kwargs
  File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/remote/base.py", line 1070, in _save_file
    self.tree.move(path_info, cache_info, mode=self.CACHE_MODE)
  File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/remote/local.py", line 138, in move
    move(from_info, to_info, mode=mode)
  File "/home/andrea/projects/myproject/.venv/lib/python3.6/site-packages/dvc/utils/fs.py", line 103, in move
    shutil.move(src, tmp)
  File "/usr/lib/python3.6/shutil.py", line 565, in move
    os.unlink(src)

Final Error:

PermissionError: [Errno 13] Permission denied: '/home/andrea/projects/coco-annotator/datasets/myproject/datasets/RoIDatasetValidv4/.exports/coco-1593013621.885437.json'

Please provide information about your setup

Output of dvc version:

Python version: 3.6.9
Platform: Linux-5.3.0-51-generic-x86_64-with-Ubuntu-18.04-bionic
Binary: False
Package: pip
Supported remotes: azure, http, https, ssh
Cache: reflink - not supported, hardlink - supported, symlink - supported
Filesystem type (cache directory): ('ext4', '/dev/nvme0n1p3')
Repo: dvc, git
Filesystem type (workspace): ('ext4', '/dev/nvme0n1p3')

Additional Information (if any):

If applicable, please also provide a --verbose output of the command, eg: dvc add --verbose.

[pared]

So, seems like we introduced a bug in 652f5ab.
Following script:

#!/bin/bash

rm -rf repo
mkdir repo

pushd repo
git init --quiet && dvc init --quiet

set -ex
mkdir -p data/.ignore_dir 
echo some_content >> data/1
echo ignored >> data/.ignore_dir/file
echo '**/.ignore_dir/' >> .dvcignore

dvc add data
rm -rf data
dvc checkout 
ls -a data
ls -a data/.ignore_dir

Shows that we stopped ignoring .ignore_dir in this change:
652f5ab result:

+ ls -a data
.  ..  1  .ignore_dir
+ ls -a data/.ignore_dir
.  ..  file

and commit before:

+ ls -a data
.  ..  1
+ ls -a data/.ignore_dir
ls: cannot access 'data/.ignore_dir': No such file or directory

This might be a separate issue, keeping it here until I know more about the original issue.

[pared]

To summarize:

• @dreaquil wanted to ignore particular directories, because files inside had a permission that would not allow him add the data to dvc
• so original issue is not about problem with adding the dir, but dvcignore not ignoring targeted dir

Currently, @dreaquil pattern does not match. Latest version that seems to be working is 1.0.0a11.
Fixing script written in my previous comment should fix the issue.

[pared]

Another problem with @dreaquil setup was that folder containing images was owned by root. Which led to some unexpected behavior.