Skip to content

Add Google Cloud documentation #1918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 1, 2018

Conversation

Georgehe4
Copy link
Contributor

No description provided.

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@heuermh
Copy link
Member

heuermh commented Feb 16, 2018

Jenkins, test this please.

transformAlignments \
gs://my_bucket/my_reads.adam \
gs://my_bucket/my_new_reads.adam \
--jars google-cloud-nio-0.22.0-alpha-shaded.jar
Copy link
Member

@heuermh heuermh Feb 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this jar is in Maven Central, would --packages be more appropriate? Note also we have an issue with adam-shell that prevents using --jars (#1349).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 towards using --packages

@coveralls
Copy link

coveralls commented Feb 16, 2018

Coverage Status

Coverage remained the same at 82.675% when pulling 9f3db80 on Georgehe4:google_cloud into 67890b8 on bigdatagenomics:master.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2669/
Test PASSed.

Copy link
Member

@fnothaft fnothaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once you address @heuermh's comment RE: --packages. Thanks for the added documentation, @Georgehe4!

@fnothaft fnothaft added this to the 0.24.0 milestone Feb 18, 2018
@Georgehe4
Copy link
Contributor Author

Adding the dependency via --packages would be ideal except that the NIO file readers rely on Guava 20.0+.

hadoop 2.6+ breaks with guava 17.0+ (

adam/pom.xml

Line 600 in 49cbdb7

<version>16.0.1</version> <!-- note: version 17.0 breaks hadoop 2.6+ at runtime -->
)

So I am using the shaded jar is the only way to prevent naming conflicts.

I don't believe we can shade using the --packages flag, but let me know if anyone knows of a workaround.

@Georgehe4
Copy link
Contributor Author

@Georgehe4
Copy link
Contributor Author

Running with --jars solves the above issue:
https://gist.github.com/Georgehe4/a6ff7e37e3a8d78a3ca7dd52846bed17

@fnothaft
Copy link
Member

fnothaft commented Mar 1, 2018

Thanks for the detailed reply and gists @Georgehe4! It looks like --jars is working fine for you, that had been broken for a while earlier; apparently we fixed it without noticing it! This is good to go from my side, ping @heuermh to approve and merge.

@heuermh
Copy link
Member

heuermh commented Mar 1, 2018

It looks like --jars is working fine for you, that had been broken for a while earlier; apparently we fixed it without noticing it

--jars works ok for adam-submit but not for adam-shell.

@heuermh heuermh merged commit babf839 into bigdatagenomics:master Mar 1, 2018
@heuermh
Copy link
Member

heuermh commented Mar 1, 2018

Thank you, @Georgehe4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants