Skip to content

SQS partial batch failure #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pankajagrawal16 opened this issue Oct 2, 2020 · 0 comments · Fixed by #120
Closed

SQS partial batch failure #119

pankajagrawal16 opened this issue Oct 2, 2020 · 0 comments · Fixed by #120
Assignees
Labels
feature-request New feature or request

Comments

@pankajagrawal16
Copy link
Contributor

Is your feature request related to a problem? Please describe.

A lambda processing a batch of messages from SQS is a very common approach and it works smooth for most use cases. Now, for the sake of example, suppose we're processing a batch and one of the messages failed, lambda is going to redrive this batch to the queue again, including the successful ones ! Re-running successful messages is not acceptable for all use cases.

Describe the solution you'd like

A very common execution pattern is running a lambda connected in sqs, in most cases with a batch size not equal to one. In such cases, an error to one of the processed messages will cause the whole batch to return to the queue. For some use cases, it's impossible to rely on such behavior - non idempotent actions. A solution for this problem would improve the experience of using a lambda with SQS.

Describe alternatives you've considered

Additional context
aws-powertools/powertools-lambda-python#92

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant