Skip to content

get-url should just GET files #4251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
skshetry opened this issue Jul 21, 2020 · 0 comments · Fixed by #4646
Closed

get-url should just GET files #4251

skshetry opened this issue Jul 21, 2020 · 0 comments · Fixed by #4646
Labels
bug Did we break something? triage Needs to be triaged

Comments

@skshetry
Copy link
Collaborator

If an endpoint does not support HEAD, but supports GET, dvc get-url will fail to work.

Eg:

$ dvc get-url https://api.travis-ci.com/v3/job/363241860/log.txt -v
2020-07-21 17:23:50,482 ERROR: failed to get 'https://api.travis-ci.com/v3/job/363241860/log.txt' - dependency 'https://api.travis-ci.com/v3/job/363241860/log.txt' does not exist
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/saugat/repos/iterative/dvc/dvc/command/get_url.py", line 17, in run
    Repo.get_url(self.args.url, out=self.args.out)
  File "/home/saugat/repos/iterative/dvc/dvc/repo/get_url.py", line 19, in get_url
    dep.save()
  File "/home/saugat/repos/iterative/dvc/dvc/output/base.py", line 253, in save
    raise self.DoesNotExistError(self)
dvc.dependency.base.DependencyDoesNotExistError: dependency 'https://api.travis-ci.com/v3/job/363241860/log.txt' does not exist
------------------------------------------------------------

Just commenting dep.save() makes it work, as it does a HEAD request first which fails.
@skshetry skshetry added bug Did we break something? triage Needs to be triaged labels Jul 21, 2020
@weekly-digest weekly-digest bot mentioned this issue Jul 26, 2020
Closed
@skshetry skshetry mentioned this issue Aug 11, 2020
Merged
2 tasks
hl-a-k pushed a commit to hl-a-k/dvc that referenced this issue Sep 30, 2020
bug iterative#4251 fixed
fa5db47 
iterative#4251
hl-a-k pushed a commit to hl-a-k/dvc that referenced this issue Sep 30, 2020
bug iterative#4251 fixed
f524c72 
iterative#4251
@hl-a-k hl-a-k mentioned this issue Sep 30, 2020
Merged
2 tasks
hl-a-k added a commit to hl-a-k/dvc that referenced this issue Oct 3, 2020
@hl-a-k
Revert "bug iterative#4251 fixed iterative#4251"
28e99f8 
This reverts commit f524c72
hl-a-k added a commit to hl-a-k/dvc that referenced this issue Oct 3, 2020
@hl-a-k
Revert "bug iterative#4251 fixed iterative#4251"
13a27a1 
This reverts commit fa5db47
hl-a-k added a commit to hl-a-k/dvc that referenced this issue Oct 3, 2020
@hl-a-k
Fix bug iterative#4251
b7203ed 
We can check if the source exists by using `dep.exists`
efiop pushed a commit that referenced this issue Oct 12, 2020
@hl-a-k
Fixed the bug #4251, and fixed a minor bug which causes of OS default…
59105ec 
… character. (#4646)

* bug fixed
bug description:
1, environment: OS which default character is not 'UTF-8'. For example windows 10 which default character is 'gbk'.
2, operation:
python setup.py install
Traceback (most recent call last):
  File "setup.py", line 149, in <module>
    long_description=open("README.rst", "r").read(),
UnicodeDecodeError: 'gbk' codec can't decode byte 0xa2 in position 41: illegal multibyte sequence

* bug #4251 fixed
#4251

* bug #4251 fixed
#4251

* Revert "bug #4251 fixed #4251"

This reverts commit f524c72

* Revert "bug #4251 fixed #4251"

This reverts commit fa5db47

* Fix bug #4251
We can check if the source exists by using `dep.exists`

Co-authored-by: zwh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Did we break something? triage Needs to be triaged
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix get-url not fetching from url that does not support HEAD request hl-a-k/dvc
1 participant
@skshetry