Skip to content

feat(event_source): add support for tumbling windows in Kinesis and DynamoDB events #6658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

kiitosu
Copy link
Contributor

@kiitosu kiitosu commented May 13, 2025

Issue number: #6627

Summary

Changes

Added properties to support tumbling windows in KinesisStreamEvent:

  • window
  • state
  • shard_id
  • event_source_arn
  • is_final_invoke_for_window
  • is_window_terminated_early

User experience

Previously, KinesisStreamEvent did not support tumbling window properties, so users could not use it with tumbling windows.
With this change, users can now use KinesisStreamEvent in Lambda functions that process tumbling window events.

Checklist

If your change doesn't seem to apply, please leave them unchecked.

This is not a breaking change.

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@kiitosu kiitosu requested a review from a team as a code owner May 13, 2025 12:26
@kiitosu kiitosu requested a review from anafalcao May 13, 2025 12:26
@boring-cyborg boring-cyborg bot added the tests label May 13, 2025
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 13, 2025
Copy link

boring-cyborg bot commented May 13, 2025

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@kiitosu kiitosu changed the title Feature/add data for tumbling window feat: support tumbling windows in KinesisStreamEvent May 13, 2025
@kiitosu
Copy link
Contributor Author

kiitosu commented May 13, 2025

@leandrodamascena
I’ve submitted a pull request — how does it look?
Do you think I should include the changes for the DynamoDB stream event as well?

@leandrodamascena leandrodamascena changed the title feat: support tumbling windows in KinesisStreamEvent feat(event_source): add support for tumbling windows in Kinesis and DynamoDB events May 13, 2025
@leandrodamascena leandrodamascena linked an issue May 13, 2025 that may be closed by this pull request
2 tasks
Copy link
Contributor

@leandrodamascena leandrodamascena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kiitosu! What an awesome PR! Thanks a lot for working on this.

Most of my feedback was about making the fields optional. These fields only exist when the client is using TumblingWindows, which is not true in most cases.

Please add support for DynamoDB as well 😄

@github-actions github-actions bot added the feature New feature or functionality label May 13, 2025
Copy link
Contributor

No related issues found. Please ensure there is an open issue related to this change to avoid significant delays or closure.

@github-actions github-actions bot added do-not-merge need-issue PRs that are missing related issues labels May 13, 2025
@leandrodamascena leandrodamascena removed do-not-merge need-issue PRs that are missing related issues labels May 13, 2025
Copy link

codecov bot commented May 13, 2025

Codecov Report

Attention: Patch coverage is 92.98246% with 4 lines in your changes missing coverage. Please review.

Project coverage is 96.11%. Comparing base (fd9a84d) to head (f2406a1).
Report is 6 commits behind head on develop.

Files with missing lines Patch % Lines
...s/utilities/data_classes/dynamo_db_stream_event.py 92.85% 1 Missing and 1 partial ⚠️
...ols/utilities/data_classes/kinesis_stream_event.py 93.10% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #6658      +/-   ##
===========================================
- Coverage    96.12%   96.11%   -0.02%     
===========================================
  Files          253      253              
  Lines        12104    12160      +56     
  Branches       902      904       +2     
===========================================
+ Hits         11635    11687      +52     
- Misses         369      371       +2     
- Partials       100      102       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

kiitosu and others added 10 commits May 13, 2025 22:01
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…ent.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
@pull-request-size pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 13, 2025
@kiitosu
Copy link
Contributor Author

kiitosu commented May 13, 2025

@leandrodamascena
Thank you for your advice!
I applied your suggestions and added support for DynamoDB Stream. Please take a look!

Copy link
Contributor

@leandrodamascena leandrodamascena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @kiitosu! We are almost there! Just a small fix and I'll run the CI.

Thanks for addressing all the feedback.

kiitosu and others added 3 commits May 13, 2025 23:03
…event.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…event.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
…event.py

Co-authored-by: Leandro Damascena <[email protected]>
Signed-off-by: kiitosu <[email protected]>
@kiitosu
Copy link
Contributor Author

kiitosu commented May 13, 2025

@leandrodamascena
Sorry for the silly mistake, and thank you for your kind reply!
I accepted your suggestions and confirmed that make test passes.

@kiitosu kiitosu requested a review from leandrodamascena May 13, 2025 22:23
@leandrodamascena
Copy link
Contributor

Hey @kiitosu! I'll merge this PR tomorrow! Thanks for accepting and addressing all the feedback! 🏅

@leandrodamascena leandrodamascena merged commit 76f16be into aws-powertools:develop May 14, 2025
10 checks passed
Copy link

boring-cyborg bot commented May 14, 2025

Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or functionality size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: Enable TumblingWindow support for KinesisStreamEvent
3 participants