Skip to content

Handle git failure to run checkout with a better message #1009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shcheklein opened this issue Aug 13, 2018 · 3 comments · Fixed by #1709
Closed

Handle git failure to run checkout with a better message #1009

shcheklein opened this issue Aug 13, 2018 · 3 comments · Fixed by #1709
Assignees
Labels
enhancement Enhances DVC
Milestone

Comments

@shcheklein
Copy link
Member

When we run a command that is running checkout internally, we could get:

Unexpected error: Cmd('git') failed due to: exit code(1)
  cmdline: git checkout bigrams
  stderr: 'error: Your local changes to the following files would be overwritten by checkout:
    auc.metric
    auc.metric.dvc
    featurization.py
    matrix.pkl.dvc
    model.pkl.dvc
Please commit your changes or stash them before you switch branches.
Aborting'

We should improve the message.

Ideally, we should find a way to traverse git's object graph w/o actual checkouts.

@shcheklein shcheklein added the enhancement Enhances DVC label Aug 13, 2018
@efiop efiop self-assigned this Aug 13, 2018
@efiop efiop added this to the Queue milestone Aug 13, 2018
@efiop
Copy link
Contributor

efiop commented Aug 13, 2018

We should use ls-tree and show branch:file instead of checking out other branches. It will eliminate the problem once and for all. Also need to access dvc cache files directly, instead of using dvc checkout. Already working on it. Thank you!

@AlJohri
Copy link

AlJohri commented Jan 30, 2019

fixing this would also solve #1552

@AlJohri
Copy link

AlJohri commented Jan 30, 2019

I'm using this script locally right now which uses git show (and jq):

#!/usr/bin/env python3

import sys
import argparse
from jq import jq # pylint: disable=no-name-in-module
from git import Repo
from git.exc import GitCommandError

import json
from pygments import highlight
from pygments.lexers.data import JsonLexer
from pygments.formatters.terminal import TerminalFormatter

parser = argparse.ArgumentParser(description="""
python scripts/metrics.py experiments/**/metrics/ml_model_parameter_search.json
python scripts/metrics.py experiments/**/metrics/rule_model_train.json --jq '.total_untagged'
python scripts/metrics.py experiments/**/metrics/rule_model_parameter_search.json --jq '.level_5.score' --all-branches
python scripts/metrics.py experiments/**/metrics/rule_model_parameter_search.json --jq 'to_entries|map(.key, .value.score)' --all-branches
""",
formatter_class=lambda prog: argparse.RawDescriptionHelpFormatter(prog, max_help_position=32))
parser.add_argument('filepaths', nargs='+')
parser.add_argument('--jq')
parser.add_argument('--all-branches', action='store_true')
args = parser.parse_args(args=None if sys.argv[1:] else ['--help'])

repo = Repo('.')

branches = ['master']
if args.all_branches:
    branches = [x.name for x in repo.branches]

for branch in branches:
    for filepath in args.filepaths:
        try:
            content = repo.git.show(f'{branch}:{filepath}')
        except GitCommandError as e:
            if 'exists on disk, but not in' in e.stderr or 'does not exist in' in e.stderr:
                continue
            else:
                raise e
        if args.jq:
            content = json.dumps(jq(args.jq).transform(text=content))
            content = highlight(content, JsonLexer(), TerminalFormatter()).strip()
        print(branch, filepath, end='')
        prefix = '\n' if len(content) > 50 else ' '
        print(f'{prefix}{content}')

here are some example outputs:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhances DVC
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants