import: importing directories output is ugly, confusing, not informative

[shcheklein]

To reproduce:

mkdir test-import-ui
git init
dvc init
dvc import https://github.com/iterative/example-get-started data/features

While it's running:

Multi-Threaded: is blinking multiple times (cc @casperdcl ).

Importing 'data/features (https://github.com/iterative/example-get-started)' -> 'features'
Multi-Threaded:
49%|████▉ |../../../../private/var/folders/_1/dxrf7_f15sn4r01jvqvr5b6h0000gn/T/tmp64drrk0idvc-erepo/data/features/train.pkl 5636096/11549662 [00:08<00:15, 374kB/s]

If we have a few separate steps (querying something, calculating checksums, downloading, etc) - it would be helpful to show clearly the message of what's happening. It might be helpful to see that this stage is done, instead of just removing everything. Otherwise it's all blinking too fast.

We must avoid exposing temp path (handled in #2605).

When it's done:

Importing 'data/features (https://github.com/iterative/example-get-started)' -> 'features'
Output 'features' didn't change. Skipping saving.

To track the changes with git, run:

	git add .gitignore features.dvc

Output 'features' didn't change. Skipping saving. - what does it mean? Is it done successfully? What has just happened?

Todo here or use multiple PRs:

• Make sure that Multi-Threaded is not used anymore
• Add a meaningful message around progress bar that clarifies what's going on here (calculating hashes, downloading data, etc)
• Do not use temp file paths (handled separately in dvc get/import shows temporary paths #2605 per @Suor comment below)
• Remove skipping saving message at the end. Make sure we don't break any other commands.
• (discuss) Show some meaningful message at the end.

[kurianbenoy]

Can I work on this?

[Suor]

@kurianbenoy take a bite if you wish, imports are not famous for having straightforward code though)

The temp path issue was discussed here #2605, which is also tricky.

[kurianbenoy]

@Suor @efiof, I couldn't find the file where the temp file path is currently being located.

[Suor]

@kurianbenoy If you are specifically going after tmp paths then it has its separate issue #2605. Let's discuss it there. It says about dvc get but that issue applies to imports too.

[shcheklein]

@kurianbenoy @Suor have updated the ticket description to reflect that #2605 is handling temp paths issue. thanks @Suor .

[kurianbenoy]

@shcheklein @Suor, I got a bit busy with other works. Will catch up with this issue after that

[pared]

Related: #2839

[dberenbaum]

It looks to me like all these issues have now been addressed. Outputs from 2.9.5.dev11+g30fa3fc41:

$ dvc import --rev cats-dogs-v1 [email protected]:iterative/dataset-registry.git use-cases/cats-dogs
Importing 'use-cases/cats-dogs ([email protected]:iterative/dataset-registry.git)' -> 'cats-dogs'

To track the changes with git, run:

    git add cats-dogs.dvc .gitignore

To enable auto staging, run:

        dvc config core.autostage true
$ dvc update cats-dogs.dvc
'cats-dogs.dvc' didn't change, skipping
# Remove revisions from cats-dogs.dvc
$ dvc update cats-dogs.dvc
Importing 'use-cases/cats-dogs ([email protected]:iterative/dataset-registry.git)' -> 'cats-dogs'

Closing this.