Skip to content

Python-based builds for AWS Lambda Layers #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

zymergen-jringgenberg
Copy link

These new scripts use the current two-container strategy to build RDKit and assemble an AWS Lambda Layer as a deployable zip file. As the Lambda environment is CentOS-like, the method for building Boost from source is reused here as well.

Note: these builds currently work with the 2018_09 release of RDKit. Subsequent releases of RDKit do not support Python 2.7 in any case.

The Python 2.7 layer can be used in conjunction with the AWS-provided SciPy layer, while the Python 3.6 layer packages NumPy because of dependency conflicts once deployed to Lambda. There might be ways around these with a bit more work.

Copy link
Contributor

@tdudgeon tdudgeon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Joe, thanks for this, and sorry it took a while to get round to review it.
My main concern here is that as this seems to be building with RDKit Release_2018_09_3 then these changes should go on the Release_2019_03* branches, probably start with the Release_2019_03 branch? I think this is better than committing on master.

Otherwise not much to say. In two of the files you are applying the patch for the cartridge build, but I don't think that's necessary as you're not building cartridge?

And I wonder longer term if building the lambda artifacts and images can just be incorporated into the standard build process, but think we need to understand the process better first.

@zymergen-jringgenberg
Copy link
Author

Thanks for the comments! Good catch with the cartridge patching step - I've stripped that out of both files.

And that makes sense about doing this on the release branches - whatever you prefer is fine. I hadn't had time yet to try fixing the Python 3.6 build to work with the March release, but I might have a chance later this week. Obviously the Python 2.7 build won't work with current and future releases.

I'll put these changes together against the Release_2019_03 and push another PR shortly. Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants