[usage] Ensure Content-Type headers are set for GCP object storage uploads
#11730
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
team: webapp
aledbf
approved these changes
Jul 28, 2022
|
/hold as it's based on #11689 |
|
/werft run with-preview 👍 started the job as gitpod-build-af-forbidden-bucket-upload.17 |
roboquat
requested review from
csweichel,
geropl and
corneliusludmann
as code owners
added 2 commits
August 2, 2022 07:48
Setting '*/*' doesn't work - it needs to be explict about the exact type of content that that the signed URL will receive.
The signed upload URL only accepts requests with this Content-Type.
andrew-farries
force-pushed
the
af/forbidden-bucket-upload
branch
from
76a9781 to
6788446
Compare
|
/unhold |
|
/hold |
|
/werft run 👍 started the job as gitpod-build-af-forbidden-bucket-upload.19 |
|
started the job as gitpod-build-af-forbidden-bucket-upload.20 because the annotations in the pull request description changed |
|
/unhold |
roboquat
added
deployed: webapp
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
As part of the move towards usage based pricing (#9036), we'd like for the usage aggregator (
components/usage) to be able to upload its usage reports to cloud storage. This will provide an audit trail of usage reports, allowing us to cross reference usage entries in the database with the usage reports that provided the data. In future, we may also allow access to these reports to users directly.In order to be able to upload these reports to GCP Cloud Storage, the signed URL can be created for a specific
Content-Type; onlyPUTrequests to that URL that set the sameContent-Typeheader will be accepted.This PR sets the
Content-Typeheader when generating the signed URL and sets it on theusagecomponentPUTto that URL.Minio object storage has no such requirements on the
Content-Typeheader which is why we only hit this problem in staging/production.Related Issue(s)
Fixes #11688
How to test
This is difficult to test as preview uses minio for object storage, not GCP.
Tested by taking the images built for this branch and hot-patching the images for
content-serviceandusagein staging.Reports are uploaded to the bucket:
Release Notes
Documentation
Werft options: