-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Support Uploading to S3 using an OutputStream #1139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
An |
I guess I'm not doing anything that outputs to OutputStreams anymore so I don't have a specific need. But I still think it makes sense that APIs that let you write files should return OutputStreams. |
Going to go ahead close this issue. @sixcorners we're currently working on the next iteration of the SDK which address some of these drawbacks so please be on the lookout for it! |
This would be a highly appreciated feature, since it would make it a lot easier to plug S3 into places where you already write into a FileOutputStream, without having to redesign anything. |
Is this implemented yet?. @dagnir @kiiadi @sixcorners We are generating documents using doc4jx which gives us an outputstream, we do not want to store in a document and upload it to the S3. We would directly like to store the inputstream from doc4jx directly to the S3. |
I really appreciate if you could provide some examples to write output stream directly S3 bucket |
@kvsravindrareddy You can only provide an InputStream to the AmazonS3 client for it to read from. You'll need to know the length in bytes of the content prior to uploading if you want it to actually be streamed. AmazonS3 will otherwise just buffer the entire contents in-memory and then send it off. If you do know how many bytes you're sending, then you could pipe streams e.g.:
The question is how does one convert an OutputStream to an InputStream. Alternatives are to write to a temp File or buffer it all in-memory. If you don't know the size, then you won't see any benefit to pipes w/ AmazonS3 compared to using ByteArrayOutputStream + ByteArrayInputStream. Writing to a File will keep memory pressure low, but you'll incur the cost of writing to disk. Hope this helps! |
There is a third option, which avoids using double-memory - https://stackoverflow.com/a/36961688/792238. |
Aren't we outputting to S3? Shouldn't we be getting an output stream to resources in S3 to write stuff?
spring-cloud-aws has an implementation of this here:
SimpleStorageResource
The text was updated successfully, but these errors were encountered: