Skip to content

cleanups: todo after implementation of pipeline file #3693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
8 of 10 tasks
skshetry opened this issue Apr 28, 2020 · 3 comments
Closed
8 of 10 tasks

cleanups: todo after implementation of pipeline file #3693

skshetry opened this issue Apr 28, 2020 · 3 comments
Assignees
Labels
enhancement Enhances DVC refactoring Factoring and re-factoring

Comments

@skshetry
Copy link
Collaborator

skshetry commented Apr 28, 2020

Tracking things to cleanup after #3676 is merged, in addition to suggestions in the PR:

  • Remove tags (will make life simpler) tag: getting rid of it #3699
  • Split Repo.collect() for pipelines usage and stage gathering:
    Data related commands don't care about orderings of pipeline stages, whereas for pipelines,
    things are more complex than collect() can handle.
    This will also remove duplicated warnings currently on reproduce.
    Workaround on tag: getting rid of it #3699 for now
  • Why is there schema/serialization/deserialization in 3 places? Yeah, true.
  • Parallel logics of loading/transforming/dumping of Output, Dependency and Stages for single stage and pipeline stages (Requires further investigation)
  • StageLoader is too dynamic.
  • Messed with OutputDuplicationsError: Make sure it's safe to remove current stage from error message.
  • (Feature Request) Some way to remove stage from both pipelines.lock and pipelines.yaml? (Implement removing of the stage #3881)
  • Renaming utils/stage.py and functions inside it. (refactor: rename utils.stage to utils.yaml #3923)
  • Make dump() ordered (Only for Python==3.5).
  • StageParams and OutputParams

Of course, we need lots of tests and UI issues fixes, but this meta issue is only regarding code cleanups.

@jorgeorpinel
Copy link
Contributor

jorgeorpinel commented Jun 3, 2020

I find the long name of options --metrics and --plots misleading, since they only accept one metrics/plot file. What about just --metric and --plot? Cc @dmpetrov

@efiop
Copy link
Contributor

efiop commented Jun 8, 2020

@jorgeorpinel Those are just consistent with the rest of the options: --outs/--outs-no-cache/etc. We might want to introduce an alias or support commas like we do in --params. But for now could leave as is.

@skshetry
Copy link
Collaborator Author

Most of the works is done. Next stage could be fixing the tests for the repro. (De)Serialization can be evaluated later, if we can do anything better. StageParams - the constants are something that we'll have to live with it.

Marking this as done for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhances DVC refactoring Factoring and re-factoring
Projects
None yet
Development

No branches or pull requests

3 participants