Skip to content

gdrive: Service account support; Default credentials; Trash only conf… #1096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 46 commits into from
Apr 6, 2020

Conversation

maxhora
Copy link
Contributor

@maxhora maxhora commented Mar 27, 2020

Fixes #1095

  • Service account setup and support;
  • Default OAuth credentials;
  • Trash only config param.

@maxhora maxhora self-assigned this Mar 27, 2020
@shcheklein shcheklein temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 27, 2020 22:40 Inactive
Copy link
Contributor

@jorgeorpinel jorgeorpinel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some doubts and copy edits. Thanks

Copy link
Member

@shcheklein shcheklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the whole doc (user guide one) is a mix of 4 sections which follow each other in some random order w/o explanations and clear connection, some of the should be adjusted (to mention that secret id is optional). Probably optional configure Google Project section should go last, introduction should be extremely clear that you usually need to specify the path (the same in the remote add/modify)

@jorgeorpinel
Copy link
Contributor

Thanks @maxhora I think we'll take this over now 👍

@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 29, 2020 22:52 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 31, 2020 00:41 Inactive
@jorgeorpinel
Copy link
Contributor

jorgeorpinel commented Mar 31, 2020

the whole doc (user guide one) is a mix of 4 sections which follow each other in some random order

I think I've mostly fixed this now. Some changes pending, about to push up. Will re-request your review soon, Ivan.

Probably optional configure Google Project section should go last, introduction should be extremely clear that you usually need to specify the path (the same in the remote add/modify)

Since we're highly recommending it, I didn't move it last. But I did make it extra clear how easy it is to start using GDrive remotes now both in the guide and in the remote add cmd ref.

@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 31, 2020 07:31 Inactive
fastest route to start using GDrive remotes
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 31, 2020 07:33 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 31, 2020 08:12 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 31, 2020 08:27 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx March 31, 2020 08:46 Inactive
@jorgeorpinel jorgeorpinel had a problem deploying to dvc-landing-1095-gxpubiooiy6vx April 5, 2020 08:05 Failure
@jorgeorpinel jorgeorpinel had a problem deploying to dvc-landing-1095-gxpubiooiy6vx April 5, 2020 08:15 Failure
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-gxpubiooiy6vx April 5, 2020 08:47 Inactive
@shcheklein
Copy link
Member

@jorgeorpinel

The build failed on Heroku though

fixed!

Also, it takes a very long time during Generating image thumbnails. Over 10 minutes.

that's what S3 cache is for! @fabiosantoscode and @pavelgrinchenko are still looking into these issues with cache. It should be way faster - similar to blog 2-3 minutes.

can only be referenced by ID).

```dvc
$ dvc remote add gdfolder gdrive://0AIac4JZqHhKmUk9PDA
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add -d here and in other places?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I intentionally only added it in the Quick start section where it's follower by dvc push. I don't think that it adds value in these other examples.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not obvious to be honest, if we use -d in all other places, let's use it everywhere?

Also, still don't see any value in these special names (one place is left).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Names updated now. I accidentally missed those, thanks.

I would rather remove -d from everywhere? The topic of default remotes seems totally separate fom gdrive to me. But then if I remove it from the Quick start example, I then need to add it to dvc push -r myremote... Will do that for now... ⏳

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's def not do dvc push -r, please ... better to include -d everywhere ... or even keep as is for now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to dvc remote add --default just in Quick start for now. That way it's more obvious and no need to explain -d or add it everywhere in this doc?

I noticed the dvc push cmd ref examples need an update and I already started another regular updates PR so I can address it there 🙂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(See #1110)

@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-lewoioukjo69w April 6, 2020 12:09 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-lewoioukjo69w April 6, 2020 13:50 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-lewoioukjo69w April 6, 2020 14:11 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-lewoioukjo69w April 6, 2020 14:45 Inactive
@jorgeorpinel jorgeorpinel requested a review from shcheklein April 6, 2020 14:45
Copy link
Member

@shcheklein shcheklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a minor bug with remote add left I believe ... check may be I missed some other conversations.

really good stuff!

@jorgeorpinel
Copy link
Contributor

Merging, but lmk your final decision about -d and I'll open a quick separate PR. (See #1096 (comment)). Thanks!

@jorgeorpinel jorgeorpinel temporarily deployed to dvc-landing-1095-lewoioukjo69w April 6, 2020 22:39 Inactive
@jorgeorpinel jorgeorpinel merged commit 20d1874 into master Apr 6, 2020
jorgeorpinel added a commit that referenced this pull request Apr 7, 2020
@jorgeorpinel jorgeorpinel deleted the 1095 branch May 5, 2020 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

user-guide: update gdrive setup guide
3 participants