Description
Use case
Constructing response objects for use in Kinesis Firehose transformation functions.
This is a continuation of #1059 which describes the event object as well as the response object. The implementation for that issue can be found at #1540, but that does not include the response object.
Solution/User Experience
A data class that can be populated during the execution of a function that will be properly formed as a response to a KinesisFirehoseEvent
invocation.
Rough idea
KinesisFirehoseResponse:
records: list[KinssisFirehoseResponseRecord]
KinesisFirehoseResponseRecord:
record_id: str
result: Literal["Ok", "ProcessingFailed"]
data: bytes
metadata: KinesisFirehoseResponseRecordMetadata
KinesisFirehoseResponseRecordMetadata:
partition_keys: dict
Note: ☝🏼 I'm not sure if this is not an exhaustive list of options that can be returned
Alternative solutions
Previously, I've used basic dictionaries for this, but it would be nice to have a more structured data class to use.
The Go example in the Dynamic Partitioning in Kinesis Data Firehose has the concept of a KinesisFirehoseResponse
in their events package.
I believe it would be possible to re-use the KinesisFirehoseEvent
data class from the utilities.data_classes
module, but this seems like it is more geared for the event invocation object as opposed to the response object.
Acknowledgment
- This feature request meets Powertools for AWS Lambda (Python) TenetsShould this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET
Metadata
Metadata
Assignees
Type
Projects
Status
Activity
boring-cyborg commentedon Jun 12, 2023
Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link
troyswanson commentedon Jun 13, 2023
Been playing with a custom implementation for my project. Thought I would share it here:
(This uses Python 3.9 so the type alias syntax is a little different than current.)
myproj/dataclasses/kinesis_firehose.py
Example implementation:
tests/conftest.py
tests/test_kinesis_firehose.py
rubenfonseca commentedon Jun 15, 2023
Hi @troyswanson thank you for opening this! Since the response object can be quite complex, I agree that we could benefit with adding those classes to our dataclasses.
For reference, here's the Go types (https://github.com/aws/aws-lambda-go/blob/main/events/firehose.go#L28-L49)
I can see that you already have some code too. I would love if you could submit a PR for this! What do you think?
19 remaining items