diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md index 37b34bf9750..a0f42b4f304 100644 --- a/docs/utilities/idempotency.md +++ b/docs/utilities/idempotency.md @@ -77,31 +77,16 @@ If you're not [changing the default configuration for the DynamoDB persistence l ???+ tip "Tip: You can share a single state table for all functions" You can reuse the same DynamoDB table to store idempotency state. We add `module_name` and [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank"} in addition to the idempotency key as a hash key. -```yaml hl_lines="5-13 21-23" title="AWS Serverless Application Model (SAM) example" -Resources: - IdempotencyTable: - Type: AWS::DynamoDB::Table - Properties: - AttributeDefinitions: - - AttributeName: id - AttributeType: S - KeySchema: - - AttributeName: id - KeyType: HASH - TimeToLiveSpecification: - AttributeName: expiration - Enabled: true - BillingMode: PAY_PER_REQUEST - - HelloWorldFunction: - Type: AWS::Serverless::Function - Properties: - Runtime: python3.9 - ... - Policies: - - DynamoDBCrudPolicy: - TableName: !Ref IdempotencyTable -``` +=== "sam.yaml" + + ```yaml hl_lines="6-14 24-31" title="AWS Serverless Application Model (SAM) example" + --8<-- "examples/idempotency/sam.yaml" + ``` +=== "cdk.py" + + ```python hl_lines="10 13 16 19-21" title="AWS Cloud Development Kit (CDK) Construct example" + --8<-- "examples/idempotency/cdk.py" + ``` ???+ warning "Warning: Large responses with DynamoDB persistence layer" When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}. @@ -148,7 +133,7 @@ You can quickly start by initializing the `DynamoDBPersistenceLayer` class and u === "Example event" - ```json + ```json { "username": "xyz", "product_id": "123456789" @@ -334,10 +319,12 @@ In this example, we have a Lambda handler that creates a payment for a user subs Imagine the function executes successfully, but the client never receives the response due to a connection issue. It is safe to retry in this instance, as the idempotent decorator will return a previously saved response. -**What we want here** is to instruct Idempotency to use `user` and `product_id` fields from our incoming payload as our idempotency key. If we were to treat the entire request as our idempotency key, a simple HTTP header change would cause our customer to be charged twice. +**What we want here** is to instruct Idempotency to use `user` and `product_id` fields from our incoming payload as our idempotency key. +If we were to treat the entire request as our idempotency key, a simple HTTP header change would cause our customer to be charged twice. ???+ tip "Deserializing JSON strings in payloads for increased accuracy." - The payload extracted by the `event_key_jmespath` is treated as a string by default. This means there could be differences in whitespace even when the JSON payload itself is identical. + The payload extracted by the `event_key_jmespath` is treated as a string by default. + This means there could be differences in whitespace even when the JSON payload itself is identical. To alter this behaviour, we can use the [JMESPath built-in function](jmespath_functions.md#powertools_json-function){target="_blank"} `powertools_json()` to treat the payload as a JSON object (dict) rather than a string. @@ -410,7 +397,8 @@ Imagine the function executes successfully, but the client never receives the re ???+ note This is automatically done when you decorate your Lambda handler with [@idempotent decorator](#idempotent-decorator). -To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/){target="_blank"}, Powertools for AWS Lambda (Python) calculates and includes the remaining invocation available time as part of the idempotency record. +To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/){target="_blank"}, +Powertools for AWS Lambda (Python) calculates and includes the remaining invocation available time as part of the idempotency record. ???+ example If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will execute the invocation again as if it was in the `EXPIRED` state (e.g, `expire_seconds` field elapsed). @@ -418,7 +406,8 @@ To prevent against extended failed retries when a [Lambda function times out](ht This means that if an invocation expired during execution, it will be quickly executed again on the next retry. ???+ important - If you are only using the [@idempotent_function decorator](#idempotent_function-decorator) to guard isolated parts of your code, you must use `register_lambda_context` available in the [idempotency config object](#customizing-the-default-behavior) to benefit from this protection. + If you are only using the [@idempotent_function decorator](#idempotent_function-decorator) to guard isolated parts of your code, + you must use `register_lambda_context` available in the [idempotency config object](#customizing-the-default-behavior) to benefit from this protection. Here is an example on how you register the Lambda context in your handler: @@ -698,14 +687,14 @@ When using DynamoDB as a persistence layer, you can alter the attribute names by Idempotent decorator can be further configured with **`IdempotencyConfig`** as seen in the previous example. These are the available options for further configuration -| Parameter | Default | Description | -| ------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -| **event_key_jmespath** | `""` | JMESPath expression to extract the idempotency key from the event record using [built-in functions](/utilities/jmespath_functions){target="_blank"} | -| **payload_validation_jmespath** | `""` | JMESPath expression to validate whether certain parameters have changed in the event while the event payload | -| **raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request | -| **expires_after_seconds** | 3600 | The number of seconds to wait before a record is expired | -| **use_local_cache** | `False` | Whether to locally cache idempotency results | -| **local_cache_max_items** | 256 | Max number of items to store in local cache | +| Parameter | Default | Description | +| ------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **event_key_jmespath** | `""` | JMESPath expression to extract the idempotency key from the event record using [built-in functions](/utilities/jmespath_functions){target="_blank"} | +| **payload_validation_jmespath** | `""` | JMESPath expression to validate whether certain parameters have changed in the event while the event payload | +| **raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request | +| **expires_after_seconds** | 3600 | The number of seconds to wait before a record is expired | +| **use_local_cache** | `False` | Whether to locally cache idempotency results | +| **local_cache_max_items** | 256 | Max number of items to store in local cache | | **hash_function** | `md5` | Function to use for calculating hashes, as provided by [hashlib](https://docs.python.org/3/library/hashlib.html){target="_blank"} in the standard library. | ### Handling concurrent executions with the same payload diff --git a/examples/idempotency/cdk.py b/examples/idempotency/cdk.py new file mode 100644 index 00000000000..4b22656defe --- /dev/null +++ b/examples/idempotency/cdk.py @@ -0,0 +1,21 @@ +from aws_cdk import RemovalPolicy +from aws_cdk import aws_dynamodb as dynamodb +from aws_cdk import aws_iam as iam +from constructs import Construct + + +class IdempotencyConstruct(Construct): + def __init__(self, scope: Construct, name: str, lambda_role: iam.Role) -> None: + super().__init__(scope, name) + self.idempotency_table = dynamodb.Table( + self, + "IdempotencyTable", + partition_key=dynamodb.Attribute(name="id", type=dynamodb.AttributeType.STRING), + billing_mode=dynamodb.BillingMode.PAY_PER_REQUEST, + removal_policy=RemovalPolicy.DESTROY, + time_to_live_attribute="expiration", + point_in_time_recovery=True, + ) + self.idempotency_table.grant( + lambda_role, "dynamodb:PutItem", "dynamodb:GetItem", "dynamodb:UpdateItem", "dynamodb:DeleteItem" + ) diff --git a/examples/idempotency/sam.yaml b/examples/idempotency/sam.yaml new file mode 100644 index 00000000000..ee9b7540de9 --- /dev/null +++ b/examples/idempotency/sam.yaml @@ -0,0 +1,31 @@ +Transform: AWS::Serverless-2016-10-31 +Resources: + IdempotencyTable: + Type: AWS::DynamoDB::Table + Properties: + AttributeDefinitions: + - AttributeName: id + AttributeType: S + KeySchema: + - AttributeName: id + KeyType: HASH + TimeToLiveSpecification: + AttributeName: expiration + Enabled: true + BillingMode: PAY_PER_REQUEST + + HelloWorldFunction: + Type: AWS::Serverless::Function + Properties: + Runtime: python3.9 + Handler: app.py + Policies: + - Statement: + - Sid: AllowDynamodbReadWrite + Effect: Allow + Action: + - dynamodb:PutItem + - dynamodb:GetItem + - dynamodb:UpdateItem + - dynamodb:DeleteItem + Resource: !GetAtt IdempotencyTable.Arn