Weekly Digest (3 May, 2020 - 10 May, 2020)

[weekly-digest]

Here's the Weekly Digest for iterative/dvc:

---

ISSUES

Last week 47 issues were created.
Of these, 29 issues have been closed and 18 issues are still open.

OPEN ISSUES

💚 #3769 Tab autocomplete when adding dirs not ignoring case even though it is set to in my .inputrc, by george-gca
💚 #3767 [WIP] pipeline file: disallow punctuation characters in stage name, by skshetry
💚 #3766 Web UI for pipeline visualisation and management, by vojavocni
💚 #3765 Drop python 3.5 support?, by skshetry
💚 #3762 Added S3 SSEKMSKeyId config option, by jackwellsxyz
💚 #3759 plot: update some output strings, by jorgeorpinel
💚 #3758 SSHException: Channel closed, by gurunath-p
💚 #3757 params/metrics: diff: implement tabulated markdown output (--show-md), by skshetry
💚 #3755 Plot: link to docs, by pared
💚 #3753 dvc push is extremely slow if you don't specify a particular thing to push, by greaber
💚 #3751 pre-commit: fails for multi-file patches, by casperdcl
💚 #3748 tests: abstract remote setup to fixture, by pared
💚 #3745 [s3/windows] HeadObject operation: Not Found, by prashaw
💚 #3744 zombie processes from dvc pull failures, by ychou85
💚 #3743 Quick listing of stages, by mribeirodantas
💚 #3739 Remote files not removed when checking out earlier commits., by KaggleShmaggle
💚 #3738 dvc init fails with ".dvc/.gitignore" file not found when .dvc directory is ignored by git, by simongoertzen
💚 #3736 Introduce check-ignore for dvc ignore command., by pared

CLOSED ISSUES

❤️ #3770 fix #3588, by karajan1001
❤️ #3768 push/pull: properly collect run cache, by efiop
❤️ #3764 Restyle stage: hide unwanted warnings, by restyled-io[bot]
❤️ #3763 stage: hide unwanted warnings, by efiop
❤️ #3761 Restyle update: --recursive flag, by restyled-io[bot]
❤️ #3760 update: --recursive flag, by nik123
❤️ #3756 [WIP] Copy file mode local push, by pared
❤️ #3754 Error when following the guide to setup azure storage as remote, by FabianHertwig
❤️ #3752 DVC should check credentials for AWS sooner, by greaber
❤️ #3750 Restyle lint: isort pre-commit & misc improvements, by restyled-io[bot]
❤️ #3749 lint: isort pre-commit & misc improvements, by casperdcl
❤️ #3747 repro: use dvc.yaml by default, by skshetry
❤️ #3746 push/pull: naively transfer run cache, by efiop
❤️ #3742 plot: check field exists, by pared
❤️ #3741 Snap version, by casperdcl
❤️ #3740 run: use multistage dvcfiles by default, by efiop
❤️ #3737 run: remove deprecated -y|--yes, by efiop
❤️ #3735 pull/fetch/push: add stats, by skshetry
❤️ #3734 plot: fix help messages, by pared
❤️ #3733 plot: completion for zsh, by pared
❤️ #3732 cache: rename .dvc/cache/run to .dvc/cache/runs, by efiop
❤️ #3731 Make comparisons in is_cached independent of order, by skshetry
❤️ #3730 ui: simplify/shorten error messages, by skshetry
❤️ #3729 run: add to .gitignore on --no-exec, by efiop
❤️ #3728 [WIP] dvc.yaml: rename stages -> run, by efiop
❤️ #3727 diff: make output consistent with other diff commands, by efiop
❤️ #3726 status: --recursive flag, by nik123
❤️ #3725 dvc: move temporary files to .dvc/tmp, by efiop
❤️ #3724 stage: rename .dvc/cache/stages -> .dvc/cache/run, by efiop

LIKED ISSUE

👍 #3757 params/metrics: diff: implement tabulated markdown output (--show-md), by skshetry
It received 👍 x3, 😄 x0, 🎉 x1 and ❤️ x2.

NOISY ISSUE

🔈 #3749 lint: isort pre-commit & misc improvements, by casperdcl
It received 9 comments.

---

PULL REQUESTS

Last week, 30 pull requests were created, updated or merged.

OPEN PULL REQUEST

Last week, 1 pull request was opened.
💚 #3767 [WIP] pipeline file: disallow punctuation characters in stage name, by skshetry

UPDATED PULL REQUEST

Last week, 5 pull requests were updated.
💛 #3762 Added S3 SSEKMSKeyId config option, by jackwellsxyz
💛 #3759 plot: update some output strings, by jorgeorpinel
💛 #3757 params/metrics: diff: implement tabulated markdown output (--show-md), by skshetry
💛 #3748 tests: abstract remote setup to fixture, by pared
💛 #3716 [WIP] repo: use unified RepoTree for erepos, by pmrowla

MERGED PULL REQUEST

Last week, 24 pull requests were merged.
💜 #3770 fix #3588, by karajan1001
💜 #3768 push/pull: properly collect run cache, by efiop
💜 #3763 stage: hide unwanted warnings, by efiop
💜 #3760 update: --recursive flag, by nik123
💜 #3749 lint: isort pre-commit & misc improvements, by casperdcl
💜 #3747 repro: use dvc.yaml by default, by skshetry
💜 #3746 push/pull: naively transfer run cache, by efiop
💜 #3742 plot: check field exists, by pared
💜 #3741 Snap version, by casperdcl
💜 #3740 run: use multistage dvcfiles by default, by efiop
💜 #3737 run: remove deprecated -y|--yes, by efiop
💜 #3735 pull/fetch/push: add stats, by skshetry
💜 #3734 plot: fix help messages, by pared
💜 #3733 plot: completion for zsh, by pared
💜 #3732 cache: rename .dvc/cache/run to .dvc/cache/runs, by efiop
💜 #3731 Make comparisons in is_cached independent of order, by skshetry
💜 #3730 ui: simplify/shorten error messages, by skshetry
💜 #3729 run: add to .gitignore on --no-exec, by efiop
💜 #3727 diff: make output consistent with other diff commands, by efiop
💜 #3726 status: --recursive flag, by nik123
💜 #3725 dvc: move temporary files to .dvc/tmp, by efiop
💜 #3724 stage: rename .dvc/cache/stages -> .dvc/cache/run, by efiop
💜 #3722 gdrive: download: stream & add progress, by casperdcl
💜 #3715 Dvc remote default in list validation , by karajan1001

---

COMMITS

Last week there were 31 commits.
🛠️ [fix #3588 (#3770) * fix #3588

remove improper edges from output DAG.

• less code modification

• deepsource required

• deepsource required

• refactoring for reducing cognitive complexity.](76b31ac) by karajan1001
  🛠️ [Dvc remote default in list validation (Dvc remote default in list validation  #3715) * fix dvc remote default accepts anything #3470

Add validation to restrict default remote repo in list of remote repos.

• fix dvc remote default accepts anything #3470

1. add two tests
2. add one validation
3. modified remote remove to satithe validation change

• Pass DeepSource

• deepsource required

• Redesigned.

• fix typo

• Revert "deepsource required"

This reverts commit 1ae458c.

• Revert "Pass DeepSource"

This reverts commit b254d10.

Co-authored-by: karajan1001 jah@mail.ustc.edu.cn
Co-authored-by: Ruslan Kuprieiev ruslan@iterative.ai](6093a1f) by karajan1001
🛠️ push/pull: properly collect run cache (#3768) Fixes #1234 by efiop
🛠️ [update: --recursive flag (#3760) * update: --recursive flag

• refactor: applied isort changes to dvc/repo/update.py

• update: removed default values, check for str

• update: fixed code climate cognitive complexity issues](c922da4) by nik123
  🛠️ stage: hide unwanted warnings (#3763) These messages all correpond to expected behaviour, so there is no need
  to polute the output with unnecessary warnings. by efiop
  🛠️ dvc: bump to 1.0.0a1 by efiop
  🛠️ push/pull: naively transfer run cache (#3746) This is the most simple (aka dumb) implementation for this
  functionality that is needed to unblock CICD development. by efiop
  🛠️ plot: fix help messages (#3734) by pared
  🛠️ trying to enable the sponsorship button by shcheklein
  🛠️ fix: bump PyDrive2 to fix Gdrive remote download by shcheklein
  🛠️ pre-commit: use isort seeder by efiop
  🛠️ isort: fix config by efiop
  🛠️ [lint: isort pre-commit & misc improvements (lint: isort pre-commit & misc improvements #3749) * test: clean flake8 config root clutter

• test: pre-commit: add isort

Compatible with black (see PyCQA/isort#694)

• lint: reorder imports

• lint: more complex imports

• lint: fix deepsource

• config: isort: add first party dvc

• tests: restyled: add isort

• temp commit to revert before merge

• Revert "temp commit to revert before merge"

This reverts commit 11db0fd.](2bb7257) by casperdcl
🛠️ repro: use dvc.yaml by default (#3747) Note: This does not change default for pipeline as
it cannot show multiple DAGs at the same time. If we
are able to show multiple DAGs, we can just default to
dvc.yaml. by skshetry
🛠️ plot: check field exists (#3742) by pared
🛠️ [gdrive: download: stream & add progress (#3722) * wip

• gdrive: add progress

Part of #2865
See #2865 (comment)

• gdrive: move towards next pydrive2 release

• depends on file.GetContentFile: stream to disk, add callback PyDrive2#30

• update to latest pydrive>=1.4.11

• avoid unneeded API call

• progress: gdrive: ensure proper bar_format](26e2032) by casperdcl
  🛠️ [pull/fetch/push: add stats (pull/fetch/push: add stats #3735) * pull/fetch/push: add stats

• fix typo: s/logging/logger

• pull: make stats detailed, with summary at the bottom

• tests: mock properly, set return_value

• add a line space](d00bd4d) by skshetry
  🛠️ [Snap version (Snap version #3741) * snap: potential build fix

• snap: fix travis](795f001) by casperdcl
  🛠️ tests: use --single-stage by efiop
  🛠️ run: use multistage dvcfiles by default (#3740) Adding hidden --single-stage to simplify the test migration. by efiop
  🛠️ dvc: bump to 1.0.0a0 by efiop
  🛠️ run: remove deprecated -y|--yes (#3737) by efiop
  🛠️ [ui: simplify/shorten error messages (ui: simplify/shorten error messages #3730) * ui: simplify/shorten error messages

• address efiop's review](c186a8e) by skshetry
  🛠️ plot: completion for zsh (#3733) by pared
  🛠️ cache: rename .dvc/cache/run to .dvc/cache/runs (#3732) To be consistent with how git does it (e.g. objects) as well as with
  future cache structure where we plan on adding dirs/files/objects
  subdirs in the future. by efiop
  🛠️ [Make comparisons in is_cached independent of order (Make comparisons in is_cached independent of order #3731) On is_cached, we used to compare two dicts: one of current stage in memory
  created by run and one that's already written to the file.
  As outs and dicts are lists (which is generate by dumpd), the
  comparison was dependent on the order of outs and dicts.

So, the run before and now, can be in different order and would fail
the comparisons.](2dda540) by skshetry
🛠️ run: add to .gitignore on --no-exec (#3729) by efiop
🛠️ [diff: make output consistent with other diff commands (#3727) * diff: make output consistent with other diff commands

Fixes #3435

• tests: plot: fix flaky test](505b88a) by efiop
  🛠️ [status: --recursive flag (status: --recursive flag #3726) * status: --recursive flag

• tests: fixed test_status_recursive for Windows platform](415a85c) by nik123
  🛠️ dvc: move temporary files to .dvc/tmp (#3725) Fixes #3666 by efiop
  🛠️ stage: rename .dvc/cache/stages -> .dvc/cache/run (#3724) by efiop

---

CONTRIBUTORS

Last week there were 7 contributors.
👤 karajan1001
👤 efiop
👤 nik123
👤 pared
👤 shcheklein
👤 casperdcl
👤 skshetry

---

STARGAZERS

Last week there were 76 stagazers.
⭐ thirteen37
⭐ mokarakaya
⭐ webysther
⭐ vzhikserg
⭐ smart-patrol
⭐ simongoertzen
⭐ dboeckenhoff
⭐ JIoJIaJIu
⭐ pankajkgupta
⭐ tpsatish95
⭐ thiagozanetti
⭐ mribeirodantas
⭐ lfwa
⭐ kureta
⭐ doty-k
⭐ JRMeyer
⭐ dilpreetsingh
⭐ rogervaas
⭐ skariel
⭐ collinarnett
⭐ sschneider
⭐ Legorooj
⭐ Valkea
⭐ piebro
⭐ gmile
⭐ himkt
⭐ salilmishra23
⭐ DavidGOrtega
⭐ biln
⭐ gasparda
⭐ garrrychan
⭐ JakaCikac
⭐ alexrigler
⭐ socheatsok78
⭐ VeggieBaconator
⭐ yo-my-bard
⭐ ajneok
⭐ sylus
⭐ sc250024
⭐ ErikStammes
⭐ abhishek9sharma
⭐ contaconta
⭐ youxkei
⭐ biowfp
⭐ gmyofustc
⭐ progsmile
⭐ maxjeblick
⭐ abhayraw1
⭐ MohamedKari
⭐ jthegedus
⭐ tangxinyao
⭐ theszkut
⭐ chengzhycn
⭐ mlopezantequera
⭐ nuraiman
⭐ BrianRuizy
⭐ MariaFjodorowa
⭐ FabianHertwig
⭐ AntonMu
⭐ peter0083
⭐ seguidor777
⭐ inoue0426
⭐ Mathbee
⭐ churnikov
⭐ NataliaDiaz
⭐ ricardojdferreira
⭐ fabianofilho
⭐ piotr-roslaniec
⭐ neksa
⭐ xzyaoi
⭐ breandan
⭐ MegaMixedReality
⭐ cy20lin
⭐ VikashKothary
⭐ vs74
⭐ jaysonfrancis
You all are the stars! 🌟

---

RELEASES

Last week there were 3 releases.
🚀 0.94.0 0.94.0
🚀 1.0.0a1 1.0.0a1
🚀 1.0.0a0 1.0.0a0

---

That's all for last week, please 👀 Watch and ⭐ Star the repository iterative/dvc to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆