Skip to content

SQS partial batch failure #119

Closed
@pankajagrawal16

Description

@pankajagrawal16

Is your feature request related to a problem? Please describe.

A lambda processing a batch of messages from SQS is a very common approach and it works smooth for most use cases. Now, for the sake of example, suppose we're processing a batch and one of the messages failed, lambda is going to redrive this batch to the queue again, including the successful ones ! Re-running successful messages is not acceptable for all use cases.

Describe the solution you'd like

A very common execution pattern is running a lambda connected in sqs, in most cases with a batch size not equal to one. In such cases, an error to one of the processed messages will cause the whole batch to return to the queue. For some use cases, it's impossible to rely on such behavior - non idempotent actions. A solution for this problem would improve the experience of using a lambda with SQS.

Describe alternatives you've considered

Additional context
aws-powertools/powertools-lambda-python#92

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @pankajagrawal16

    Issue actions

      SQS partial batch failure · Issue #119 · aws-powertools/powertools-lambda-java