From 8b4a7b3178f664288f85750c6fff46939516727a Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Thu, 23 May 2024 17:38:04 +1200
Subject: [PATCH 01/41] docs(idempotency): cleanup redis usage and link with
 setup/infra

---
 docs/utilities/idempotency.md                 | 219 +++++++++---------
 .../using_redis_client_with_aws_secrets.py    |   4 +-
 .../using_redis_client_with_local_certs.py    |   4 +-
 .../templates/cfn_redis_serverless.yaml       |  22 +-
 .../idempotency/templates/sam_redis_vpc.yaml  |  14 --
 5 files changed, 133 insertions(+), 130 deletions(-)
 delete mode 100644 examples/idempotency/templates/sam_redis_vpc.yaml

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 12baabd047e..faede489924 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -73,12 +73,18 @@ We currently support Amazon DynamoDB and Redis as a storage layer. The following
 If you're not [changing the default configuration for the DynamoDB persistence layer](#dynamodbpersistencelayer), this is the expected default configuration:
 
 | Configuration      | Value        | Notes                                                                               |
-| ------------------ | ------------ |-------------------------------------------------------------------------------------|
+| ------------------ | ------------ | ----------------------------------------------------------------------------------- |
 | Partition key      | `id`         |                                                                                     |
 | TTL attribute name | `expiration` | This can only be configured after your table is created if you're using AWS Console |
 
-???+ tip "Tip: You can share a single state table for all functions"
-    You can reuse the same DynamoDB table to store idempotency state. We add `module_name` and [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} in addition to the idempotency key as a hash key.
+| Configuration      | Value        | Notes                                                                                                    |
+| ------------------ | ------------ | -------------------------------------------------------------------------------------------------------- |
+| Partition key      | `id`         | Primary key looks like: <br> `{lambda_fn_name}.{module_name}.{fn_qualified_name}#{idempotency_key_hash}` |
+| TTL attribute name | `expiration` | This can only be configured after your table is created if you're using AWS Console                      |
+
+Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
+
+##### DynamoDB IaC examples
 
 === "AWS Serverless Application Model (SAM) example"
 
@@ -109,7 +115,24 @@ If you're not [changing the default configuration for the DynamoDB persistence l
 
     On subsequent invocations with the same payload, you can expect just 1 `PutItem` request to DynamoDB.
 
-    **Note:** While we try to minimize requests to DynamoDB to 1 per invocation, if your boto3 version is lower than `1.26.194`, you may experience 2 requests in every invocation. Ensure to check your boto3 version and review the [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"} to estimate the cost.
+We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
+
+In both services and self-hosting Redis, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
+
+!!! tip "First time setting it all up? Checkout the official tutorials for [Amazon ElastiCache for Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/LambdaRedis.html) or [Amazon MemoryDB for Redis](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/)"
+
+##### Redis IaC examples
+
+=== "AWS CloudFormation example"
+
+    ```yaml hl_lines="5 21"
+    --8<-- "examples/idempotency/templates/cfn_redis_serverless.yaml"
+    ```
+
+    1. Replace the Security Group ID and Subnet ID to match your VPC settings.
+    2. Replace the Security Group ID and Subnet ID to match your VPC settings.
+
+Once setup, you can find quick start and advanced examples for Redis in [the persistent layers section](RedisCachePersistenceLayer).
 
 <!-- markdownlint-enable MD013 -->
 ### Idempotent decorator
@@ -353,6 +376,8 @@ This persistence layer is built-in, allowing you to use an existing DynamoDB tab
     --8<-- "examples/idempotency/src/customize_persistence_layer.py"
     ```
 
+##### DynamoDB defaults
+
 When using DynamoDB as the persistence layer, you can customize the attribute names by passing the following parameters during the initialization of the persistence layer:
 
 | Parameter                   | Required           | Default                              | Description                                                                                              |
@@ -369,7 +394,75 @@ When using DynamoDB as the persistence layer, you can customize the attribute na
 
 #### RedisPersistenceLayer
 
-This persistence layer is built-in, allowing you to use an existing Redis service. For optimal performance and compatibility, it is strongly recommended to use a Redis service version 7 or higher.
+!!! info "We recommend Redis version 7 or higher for optimal performance."
+
+For a quick start, initialize `RedisCachePersistenceLayer` and pass your cluster host endpoint along with the port to connect to.
+
+For security, we enforce SSL connections by default; to disable it, set `ssl=False`.
+
+=== "Redis quick start"
+    ```python hl_lines="7-9 12 26"
+    --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_config.py"
+    ```
+
+=== "Using an existing Redis client"
+    ```python hl_lines="4 9-11 14 22 36"
+    --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_client.py"
+    ```
+
+=== "Sample event"
+
+    ```json
+    --8<-- "examples/idempotency/src/getting_started_with_idempotency_payload.json"
+    ```
+
+##### Redis SSL connections
+
+We recommend using AWS Secrets Manager to store and rotate certificates safely, and the [Parameters feature](./parameters.md){target="_blank"} to fetch and cache optimally.
+
+For advanced configurations, we also recommend using an existing Redis client for optimal compatibility like SSL certificates and timeout.
+
+=== "Advanced configuration using AWS Secrets"
+    ```python hl_lines="9-11 13 15 25"
+    --8<-- "examples/idempotency/src/using_redis_client_with_aws_secrets.py"
+    ```
+
+    1. JSON stored:
+    ```json
+    {
+    "REDIS_ENDPOINT": "127.0.0.1",
+    "REDIS_PORT": "6379",
+    "REDIS_PASSWORD": "redis-secret"
+    }
+    ```
+
+=== "Advanced configuration with local certificates"
+    ```python hl_lines="14 25-27"
+    --8<-- "examples/idempotency/src/using_redis_client_with_local_certs.py"
+    ```
+
+    1. JSON stored:
+    ```json
+    {
+    "REDIS_ENDPOINT": "127.0.0.1",
+    "REDIS_PORT": "6379",
+    "REDIS_PASSWORD": "redis-secret"
+    }
+    ```
+    2. redis_user.crt file stored in the "certs" directory of your Lambda function
+    3. redis_user_private.key file stored in the "certs" directory of your Lambda function
+    4. redis_ca.pem file stored in the "certs" directory of your Lambda function
+
+##### Redis defaults
+
+You can customize attribute names when instantiating `RedisCachePersistenceLayer` with the following parameters:
+
+| Parameter                   | Required | Default                  | Description                                                                                   |
+| --------------------------- | -------- | ------------------------ | --------------------------------------------------------------------------------------------- |
+| **in_progress_expiry_attr** |          | `in_progress_expiration` | Unix timestamp of when record expires while in progress (in case of the invocation times out) |
+| **status_attr**             |          | `status`                 | Stores status of the Lambda execution during and after invocation                             |
+| **data_attr**               |          | `data`                   | Stores results of successfully executed Lambda handlers                                       |
+| **validation_key_attr**     |          | `validation`             | Hashed representation of the parts of the event used for validation                           |
 
 === "Customizing RedisPersistenceLayer to suit your data structure"
 
@@ -377,15 +470,6 @@ This persistence layer is built-in, allowing you to use an existing Redis servic
     --8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
     ```
 
-When using Redis as the persistence layer, you can customize the attribute names by providing the following parameters upon initialization of the persistence layer:
-
-| Parameter                   | Required           | Default                              | Description                                                                                              |
-| --------------------------- | ------------------ | ------------------------------------ | -------------------------------------------------------------------------------------------------------- |
-| **in_progress_expiry_attr** |                    | `in_progress_expiration`             | Unix timestamp of when record expires while in progress (in case of the invocation times out)            |
-| **status_attr**             |                    | `status`                             | Stores status of the Lambda execution during and after invocation                                        |
-| **data_attr**               |                    | `data`                               | Stores results of successfully executed Lambda handlers                                                  |
-| **validation_key_attr**     |                    | `validation`                         | Hashed representation of the parts of the event used for validation |
-
 ### Idempotency request flow
 
 The following sequence diagrams explain how the Idempotency feature behaves under different scenarios.
@@ -638,110 +722,21 @@ graph TD;
 <i>Race condition with Redis</i>
 </center>
 
-## Redis as persistent storage layer provider
-
-### Redis resources
-
-Before setting up Redis as the persistent storage layer provider, you must have an existing Redis service. We recommend you to use Redis compatible services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"} as your persistent storage layer provider.
-
-???+ tip "No existing Redis service?"
-    If you don't have an existing Redis service, we recommend using [DynamoDB](#dynamodbpersistencelayer) as the persistent storage layer provider.
-
-=== "AWS CloudFormation example"
-
-    ```yaml hl_lines="5"
-    --8<-- "examples/idempotency/templates/cfn_redis_serverless.yaml"
-    ```
-
-    1. Replace the Security Group ID and Subnet ID to match your VPC settings.
-
-### VPC Access
-
-Your Lambda Function must have network access to the Redis endpoint before using it as the idempotency persistent storage layer. In most cases, you will need to [configure VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} for your Lambda Function.
-
-???+ tip "Amazon ElastiCache/MemoryDB for Redis as persistent storage layer provider"
-    If you plan to use Amazon ElastiCache for Redis as the idempotency persistent storage layer, you may find [this AWS tutorial](https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html){target="_blank"} helpful.
-    For those using Amazon MemoryDB for Redis, refer to [this AWS tutorial](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/){target="_blank"} specifically for the VPC setup guidance.
-
-After completing the VPC setup, you can use the templates provided below to set up Lambda functions with access to VPC internal subnets.
-
-=== "AWS Serverless Application Model (SAM) example"
-
-    ```yaml hl_lines="9"
-    --8<-- "examples/idempotency/templates/sam_redis_vpc.yaml"
-    ```
-
-    1. Replace the Security Group ID and Subnet ID to match your VPC settings.
-
-### Configuring Redis persistence layer
-
-You can quickly get started by initializing the `RedisCachePersistenceLayer` class and applying the `idempotent` decorator to your Lambda handler. For a detailed example of using the `RedisCachePersistenceLayer`, refer to the [Persistence layers section](#redispersistencelayer).
-
-???+ info
-    We enforce security best practices by using SSL connections in the `RedisCachePersistenceLayer`; to disable it, set `ssl=False`
-
-=== "Use Persistence Layer with Redis config variables"
-    ```python hl_lines="7-9 12 26"
-    --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_config.py"
-    ```
-
-=== "Use established Redis Client"
-    ```python hl_lines="4 9-11 14 22 36"
-    --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_client.py"
-    ```
-
-=== "Sample event"
-
-    ```json
-    --8<-- "examples/idempotency/src/getting_started_with_idempotency_payload.json"
-    ```
-
-### Custom advanced settings
-
-For advanced configurations, such as setting up SSL certificates or customizing parameters like a custom timeout, you can utilize the Redis client to tailor these specific settings to your needs.
-
-=== "Advanced configuration using AWS Secrets"
-    ```python hl_lines="7-9 11 13 23"
-    --8<-- "examples/idempotency/src/using_redis_client_with_aws_secrets.py"
-    ```
-
-    1. JSON stored:
-    {
-    "REDIS_ENDPOINT": "127.0.0.1",
-    "REDIS_PORT": "6379",
-    "REDIS_PASSWORD": "redis-secret"
-    }
-
-=== "Advanced configuration with local certificates"
-    ```python hl_lines="12 23-25"
-    --8<-- "examples/idempotency/src/using_redis_client_with_local_certs.py"
-    ```
-
-    1. JSON stored:
-    {
-    "REDIS_ENDPOINT": "127.0.0.1",
-    "REDIS_PORT": "6379",
-    "REDIS_PASSWORD": "redis-secret"
-    }
-    2. redis_user.crt file stored in the "certs" directory of your Lambda function
-    3. redis_user_private.key file stored in the "certs" directory of your Lambda function
-    4. redis_ca.pem file stored in the "certs" directory of your Lambda function
-
 ## Advanced
 
 ### Customizing the default behavior
 
 Idempotent decorator can be further configured with **`IdempotencyConfig`** as seen in the previous example. These are the available options for further configuration
 
-| Parameter                       | Default | Description                                                                                                                                                                                                                                  |
-|---------------------------------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **event_key_jmespath**          | `""`    | JMESPath expression to extract the idempotency key from the event record using [built-in functions](./jmespath_functions.md#built-in-jmespath-functions){target="_blank"}                                                                    |
-| **payload_validation_jmespath** | `""`    | JMESPath expression to validate whether certain parameters have changed in the event while the event payload                                                                                                                                 |
-| **raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request                                                                                                                                                                               |
-| **expires_after_seconds**       | 3600    | The number of seconds to wait before a record is expired                                                                                                                                                                                     |
-| **use_local_cache**             | `False` | Whether to locally cache idempotency results                                                                                                                                                                                                 |
-| **local_cache_max_items**       | 256     | Max number of items to store in local cache                                                                                                                                                                                                  |
-| **hash_function**               | `md5`   | Function to use for calculating hashes, as provided by [hashlib](https://docs.python.org/3/library/hashlib.html){target="_blank" rel="nofollow"} in the standard library.                                                                    |
+| Parameter                       | Default | Description                                                                                                                                                                                                                                |
+| ------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| **event_key_jmespath**          | `""`    | JMESPath expression to extract the idempotency key from the event record using [built-in functions](./jmespath_functions.md#built-in-jmespath-functions){target="_blank"}                                                                  |
+| **payload_validation_jmespath** | `""`    | JMESPath expression to validate whether certain parameters have changed in the event while the event payload                                                                                                                               |
+| **raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request                                                                                                                                                                             |
+| **expires_after_seconds**       | 3600    | The number of seconds to wait before a record is expired                                                                                                                                                                                   |
+| **use_local_cache**             | `False` | Whether to locally cache idempotency results                                                                                                                                                                                               |
+| **local_cache_max_items**       | 256     | Max number of items to store in local cache                                                                                                                                                                                                |
+| **hash_function**               | `md5`   | Function to use for calculating hashes, as provided by [hashlib](https://docs.python.org/3/library/hashlib.html){target="_blank" rel="nofollow"} in the standard library.                                                                  |
 | **response_hook**               | `None`  | Function to use for processing the stored Idempotent response. This function hook is called when an existing idempotent response is found. See [Manipulating The Idempotent Response](idempotency.md#manipulating-the-idempotent-response) |
 
 ### Handling concurrent executions with the same payload
diff --git a/examples/idempotency/src/using_redis_client_with_aws_secrets.py b/examples/idempotency/src/using_redis_client_with_aws_secrets.py
index f30751c8808..ddf2659e815 100644
--- a/examples/idempotency/src/using_redis_client_with_aws_secrets.py
+++ b/examples/idempotency/src/using_redis_client_with_aws_secrets.py
@@ -1,3 +1,5 @@
+from __future__ import annotations
+
 from typing import Any
 
 from redis import Redis
@@ -8,7 +10,7 @@
     RedisCachePersistenceLayer,
 )
 
-redis_values: Any = parameters.get_secret("redis_info", transform="json")  # (1)!
+redis_values: dict[str, Any] = parameters.get_secret("redis_info", transform="json")  # (1)!
 
 redis_client = Redis(
     host=redis_values.get("REDIS_HOST"),
diff --git a/examples/idempotency/src/using_redis_client_with_local_certs.py b/examples/idempotency/src/using_redis_client_with_local_certs.py
index cbad1cc92f4..946fd6a064d 100644
--- a/examples/idempotency/src/using_redis_client_with_local_certs.py
+++ b/examples/idempotency/src/using_redis_client_with_local_certs.py
@@ -1,3 +1,5 @@
+from __future__ import annotations
+
 from typing import Any
 
 from redis import Redis
@@ -9,7 +11,7 @@
     RedisCachePersistenceLayer,
 )
 
-redis_values: Any = parameters.get_secret("redis_info", transform="json")  # (1)!
+redis_values: dict[str, Any] = parameters.get_secret("redis_info", transform="json")  # (1)!
 
 
 redis_client = Redis(
diff --git a/examples/idempotency/templates/cfn_redis_serverless.yaml b/examples/idempotency/templates/cfn_redis_serverless.yaml
index 9087efce6f9..8ce9d67f3cb 100644
--- a/examples/idempotency/templates/cfn_redis_serverless.yaml
+++ b/examples/idempotency/templates/cfn_redis_serverless.yaml
@@ -1,4 +1,5 @@
-AWSTemplateFormatVersion: '2010-09-09'
+AWSTemplateFormatVersion: "2010-09-09"
+Transform: AWS::Serverless-2016-10-31
 
 Resources:
   RedisServerlessIdempotency:
@@ -7,7 +8,24 @@ Resources:
       Engine: redis
       ServerlessCacheName: redis-cache
       SecurityGroupIds: # (1)!
-          - security-{your_sg_id}
+        - security-{your_sg_id}
       SubnetIds:
+        - subnet-{your_subnet_id_1}
+        - subnet-{your_subnet_id_2}
+
+  HelloWorldFunction:
+    Type: AWS::Serverless::Function
+    Properties:
+      Runtime: python3.12
+      Handler: app.py
+      VpcConfig: # (1)!
+        SecurityGroupIds:
+          - security-{your_sg_id}
+        SubnetIds:
           - subnet-{your_subnet_id_1}
           - subnet-{your_subnet_id_2}
+      Environment:
+        Variables:
+          POWERTOOLS_SERVICE_NAME: sample
+          REDIS_HOST: !GetAtt RedisServerlessIdempotency.Endpoint.Address
+          REDIS_PORT: !GetAtt RedisServerlessIdempotency.Endpoint.Port
diff --git a/examples/idempotency/templates/sam_redis_vpc.yaml b/examples/idempotency/templates/sam_redis_vpc.yaml
deleted file mode 100644
index 921b1e75b84..00000000000
--- a/examples/idempotency/templates/sam_redis_vpc.yaml
+++ /dev/null
@@ -1,14 +0,0 @@
-AWSTemplateFormatVersion: '2010-09-09'
-Transform: AWS::Serverless-2016-10-31
-Resources:
-  HelloWorldFunction:
-    Type: AWS::Serverless::Function
-    Properties:
-      Runtime: python3.11
-      Handler: app.py
-      VpcConfig: # (1)!
-        SecurityGroupIds:
-          - security-{your_sg_id}
-        SubnetIds:
-          - subnet-{your_subnet_id_1}
-          - subnet-{your_subnet_id_2}

From 92307cb8aef6809eddac1c5ac4be3c3f1eccf795 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 27 May 2024 12:52:37 +1200
Subject: [PATCH 02/41] docs(idempotency): cleanup idempotent decorator; inline
 admonitions

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md                 | 26 +++++++++----------
 .../src/getting_started_with_idempotency.py   |  7 ++---
 examples/idempotency/templates/sam.yaml       | 19 ++++++++------
 3 files changed, 27 insertions(+), 25 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index faede489924..f6046f4ee3f 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -119,12 +119,14 @@ We recommend you start with a Redis compatible management services such as [Amaz
 
 In both services and self-hosting Redis, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
 
-!!! tip "First time setting it all up? Checkout the official tutorials for [Amazon ElastiCache for Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/LambdaRedis.html) or [Amazon MemoryDB for Redis](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/)"
-
 ##### Redis IaC examples
 
 === "AWS CloudFormation example"
 
+    !!! tip inline end "Prefer AWS Console/CLI?"
+
+        Follow the official tutorials for [Amazon ElastiCache for Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/LambdaRedis.html) or [Amazon MemoryDB for Redis](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/)
+
     ```yaml hl_lines="5 21"
     --8<-- "examples/idempotency/templates/cfn_redis_serverless.yaml"
     ```
@@ -132,34 +134,30 @@ In both services and self-hosting Redis, you'll need to configure [VPC access](h
     1. Replace the Security Group ID and Subnet ID to match your VPC settings.
     2. Replace the Security Group ID and Subnet ID to match your VPC settings.
 
-Once setup, you can find quick start and advanced examples for Redis in [the persistent layers section](RedisCachePersistenceLayer).
+Once setup, you can find a quick start and advanced examples for Redis in [the persistent layers section](RedisCachePersistenceLayer).
 
 <!-- markdownlint-enable MD013 -->
-### Idempotent decorator
 
-You can quickly start by initializing the `DynamoDBPersistenceLayer` class and using it with the `idempotent` decorator on your lambda handler.
+### Idempotent decorator
 
-???+ note
-    In this example, the entire Lambda handler is treated as a single idempotent operation. If your Lambda handler can cause multiple side effects, or you're only interested in making a specific logic idempotent, use [`idempotent_function`](#idempotent_function-decorator) instead.
+For simple use cases, you can use the `idempotent` decorator on your Lambda handler function.
 
-!!! tip "See [Choosing a payload subset for idempotency](#choosing-a-payload-subset-for-idempotency) for more elaborate use cases."
+It will treat the entire event as an idempotency key. That is, the same event will return the previously stored result within a [configurable time window](#expiring-idempotency-records) _(1 hour, by default)_.
 
 === "Idempotent decorator"
 
-    ```python hl_lines="4-7 10 24"
+    !!! tip "You can also choose [one or more fields](#choosing-a-payload-subset-for-idempotency) as an idempotency key."
+
+    ```python title="getting_started_with_idempotency.py" hl_lines="5-8 12 25"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency.py"
     ```
 
 === "Sample event"
 
-    ```json
+    ```json title="getting_started_with_idempotency_payload.json"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_payload.json"
     ```
 
-After processing this request successfully, a second request containing the exact same payload above will now return the same response, ensuring our customer isn't charged twice.
-
-!!! question "New to idempotency concept? Please review our [Terminology](#terminology) section if you haven't yet."
-
 ### Idempotent_function decorator
 
 Similar to [idempotent decorator](#idempotent-decorator), you can use `idempotent_function` decorator for any synchronous Python function.
diff --git a/examples/idempotency/src/getting_started_with_idempotency.py b/examples/idempotency/src/getting_started_with_idempotency.py
index 0754f42c6b3..e94f16d7e0c 100644
--- a/examples/idempotency/src/getting_started_with_idempotency.py
+++ b/examples/idempotency/src/getting_started_with_idempotency.py
@@ -1,3 +1,4 @@
+import os
 from dataclasses import dataclass, field
 from uuid import uuid4
 
@@ -7,7 +8,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 
 @dataclass
@@ -17,8 +19,7 @@ class Payment:
     payment_id: str = field(default_factory=lambda: f"{uuid4()}")
 
 
-class PaymentError(Exception):
-    ...
+class PaymentError(Exception): ...
 
 
 @idempotent(persistence_store=persistence_layer)
diff --git a/examples/idempotency/templates/sam.yaml b/examples/idempotency/templates/sam.yaml
index c4eaf766c23..4faab5c4225 100644
--- a/examples/idempotency/templates/sam.yaml
+++ b/examples/idempotency/templates/sam.yaml
@@ -21,11 +21,14 @@ Resources:
       Handler: app.py
       Policies:
         - Statement:
-          - Sid: AllowDynamodbReadWrite
-            Effect: Allow
-            Action:
-              - dynamodb:PutItem
-              - dynamodb:GetItem
-              - dynamodb:UpdateItem
-              - dynamodb:DeleteItem
-            Resource: !GetAtt IdempotencyTable.Arn
+            - Sid: AllowDynamodbReadWrite
+              Effect: Allow
+              Action:
+                - dynamodb:PutItem
+                - dynamodb:GetItem
+                - dynamodb:UpdateItem
+                - dynamodb:DeleteItem
+              Resource: !GetAtt IdempotencyTable.Arn
+      Environment:
+        Variables:
+          IDEMPOTENCY_TABLE: !Ref IdempotencyTable

From 06dc1ee9ba8207a0cc96166f86a8294b36828622 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 27 May 2024 13:18:20 +1200
Subject: [PATCH 03/41] docs(idempotency): cleanup idempotent_decorator section

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md                   | 17 +++++++++--------
 ...orking_with_idempotent_function_dataclass.py |  6 ++++--
 2 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index f6046f4ee3f..3561436ccc1 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -160,24 +160,25 @@ It will treat the entire event as an idempotency key. That is, the same event wi
 
 ### Idempotent_function decorator
 
-Similar to [idempotent decorator](#idempotent-decorator), you can use `idempotent_function` decorator for any synchronous Python function.
+For full flexibility, you can use the `idempotent_function` decorator for any synchronous Python function.
 
-When using `idempotent_function`, you must tell us which keyword parameter in your function signature has the data we should use via **`data_keyword_argument`**.
+When using this decorator, you **must** call your decorated function using keyword arguments.
 
-!!! tip "We support JSON serializable data, [Python Dataclasses](https://docs.python.org/3.12/library/dataclasses.html){target="_blank" rel="nofollow"}, [Parser/Pydantic Models](parser.md){target="_blank"}, and our [Event Source Data Classes](./data_classes.md){target="_blank"}."
-
-???+ warning "Limitation"
-    Make sure to call your decorated function using keyword arguments.
+You can use `data_keyword_argument` to tell us the argument to extract an idempotency key.  We support JSON serializable data, [Dataclasses](https://docs.python.org/3.12/library/dataclasses.html){target="_blank" rel="nofollow"}, Pydantic Models, and [Event Source Data Classes](./data_classes.md){target="_blank"}
 
 === "Using Dataclasses"
 
-    ```python hl_lines="3-7 11 26 37"
+    ```python title="working_with_idempotent_function_dataclass.py" hl_lines="3-7 11 26 39"
     --8<-- "examples/idempotency/src/working_with_idempotent_function_dataclass.py"
     ```
 
+    1. Notice how **`data_keyword_argument`** matches the name of the parameter.
+    <br><br> This allows us to extract one or all fields as idempotency key.
+    2. Different from `idempotent` decorator, we must explicitly register the Lambda context to [protect against timeouts](#lambda-timeouts).
+
 === "Using Pydantic"
 
-    ```python hl_lines="1-5 10 23 34"
+    ```python title="working_with_idempotent_function_pydantic.py" hl_lines="1-5 10 23 34"
     --8<-- "examples/idempotency/src/working_with_idempotent_function_pydantic.py"
     ```
 
diff --git a/examples/idempotency/src/working_with_idempotent_function_dataclass.py b/examples/idempotency/src/working_with_idempotent_function_dataclass.py
index e56c0b42029..bd3ac0158b2 100644
--- a/examples/idempotency/src/working_with_idempotent_function_dataclass.py
+++ b/examples/idempotency/src/working_with_idempotent_function_dataclass.py
@@ -24,12 +24,14 @@ class Order:
 
 
 @idempotent_function(data_keyword_argument="order", config=config, persistence_store=dynamodb)
-def process_order(order: Order):
+def process_order(order: Order):  # (1)!
     return f"processed order {order.order_id}"
 
 
 def lambda_handler(event: dict, context: LambdaContext):
-    config.register_lambda_context(context)  # see Lambda timeouts section
+    # see Lambda timeouts section
+    config.register_lambda_context(context)  # (2)!
+
     order_item = OrderItem(sku="fake", description="sample")
     order = Order(item=order_item, order_id=1)
 

From eb4253409a9e50a54d014a1f9854e9ba3dc5a36f Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 27 May 2024 13:42:02 +1200
Subject: [PATCH 04/41] docs(idempotency): cleanup serialization, fields
 subset, move batch to new common use cases section

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md                 | 59 ++++++++++---------
 ...egrate_idempotency_with_batch_processor.py | 29 ++++-----
 2 files changed, 42 insertions(+), 46 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 3561436ccc1..18ffccc291e 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -188,11 +188,11 @@ By default, `idempotent_function` serializes, stores, and returns your annotated
 
 The output serializer supports any JSON serializable data, **Python Dataclasses** and **Pydantic Models**.
 
-!!! info "When using the `output_serializer` parameter, the data will continue to be stored in DynamoDB as a JSON object."
+!!! info "When using the `output_serializer` parameter, the data will continue to be stored in DynamoDB as a JSON string."
 
 === "Pydantic"
 
-    You can use `PydanticSerializer` to automatically serialize what's retrieved from the persistent storage based on the return type annotated.
+    Use `PydanticSerializer` to automatically serialize what's retrieved from the persistent storage based on the return type annotated.
 
     === "Inferring via the return type"
 
@@ -212,7 +212,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
 
 === "Dataclasses"
 
-     You can use `DataclassSerializer` to automatically serialize what's retrieved from the persistent storage based on the return type annotated.
+     Use `DataclassSerializer` to automatically serialize what's retrieved from the persistent storage based on the return type annotated.
 
     === "Inferring via the return type"
 
@@ -232,7 +232,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
 
 === "Any type"
 
-    You can use `CustomDictSerializer` to have full control over the serialization process for any type. It expects two functions:
+    Use `CustomDictSerializer` to have full control over the serialization process for any type. It expects two functions:
 
     * **to_dict**. Function to convert any type to a JSON serializable dictionary before it saves into the persistent storage.
     * **from_dict**. Function to convert from a dictionary retrieved from persistent storage and serialize in its original form.
@@ -245,42 +245,20 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
     2. This function does the following <br><br>**1**. Receives the dictionary saved into the persistent storage <br>**1** Serializes to `OrderOutput` before `@idempotent` returns back to the caller.
     3. This serializer receives both functions so it knows who to call when to serialize to and from dictionary.
 
-#### Batch integration
-
-You can can easily integrate with [Batch utility](batch.md){target="_blank"} via context manager. This ensures that you process each record in an idempotent manner, and guard against a [Lambda timeout](#lambda-timeouts) idempotent situation.
-
-???+ "Choosing an unique batch record attribute"
-    In this example, we choose `messageId` as our idempotency key since we know it'll be unique.
-
-    Depending on your use case, it might be more accurate [to choose another field](#choosing-a-payload-subset-for-idempotency) your producer intentionally set to define uniqueness.
-
-=== "Integration with Batch Processor"
-
-    ```python hl_lines="2 12 16 20 31 35 37"
-    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor.py"
-    ```
-
-=== "Sample event"
-
-    ```json hl_lines="4"
-    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor_payload.json"
-    ```
-
 ### Choosing a payload subset for idempotency
 
 ???+ tip "Tip: Dealing with always changing payloads"
     When dealing with a more elaborate payload, where parts of the payload always change, you should use **`event_key_jmespath`** parameter.
 
-Use [`IdempotencyConfig`](#customizing-the-default-behavior) to instruct the idempotent decorator to only use a portion of your payload to verify whether a request is idempotent, and therefore it should not be retried.
+Use [`IdempotencyConfig`](#customizing-the-default-behavior)'s **`event_key_jmespath`** parameter to select one or more payload parts as your idempotency key.
 
 > **Payment scenario**
 
 In this example, we have a Lambda handler that creates a payment for a user subscribing to a product. We want to ensure that we don't accidentally charge our customer by subscribing them more than once.
 
-Imagine the function executes successfully, but the client never receives the response due to a connection issue. It is safe to retry in this instance, as the idempotent decorator will return a previously saved response.
+Imagine the function runs successfully, but the client never receives the response due to a connection issue. It is safe to immediately retry in this instance, as the idempotent decorator will return a previously saved response.
 
-**What we want here** is to instruct Idempotency to use `user_id` and `product_id` fields from our incoming payload as our idempotency key.
-If we were to treat the entire request as our idempotency key, a simple HTTP header change would cause our customer to be charged twice.
+**We want** to use `user_id` and `product_id` fields as our idempotency key. If we were to treat the entire request as our idempotency key, a simple HTTP header change would cause our function to run again.
 
 ???+ tip "Deserializing JSON strings in payloads for increased accuracy."
     The payload extracted by the `event_key_jmespath` is treated as a string by default.
@@ -469,6 +447,29 @@ You can customize attribute names when instantiating `RedisCachePersistenceLayer
     --8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
     ```
 
+### Common use cases
+
+#### Batch integration
+
+You can can easily integrate with [Batch](batch.md){target="_blank"} with the [idempotent_function decorator](#idempotent_function-decorator) to handle idempotency per message/record in a given batch.
+
+???+ "Choosing an unique batch record attribute"
+    In this example, we choose `messageId` as our idempotency key since we know it'll be unique.
+
+    Depending on your use case, it might be more accurate [to choose another field](#choosing-a-payload-subset-for-idempotency) your producer intentionally set to define uniqueness.
+
+=== "Integration with Batch Processor"
+
+    ```python title="integrate_idempotency_with_batch_processor.py" hl_lines="3 16 19 25 27"
+    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor.py"
+    ```
+
+=== "Sample event"
+
+    ```json title="integrate_idempotency_with_batch_processor_payload.json" hl_lines="4"
+    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor_payload.json"
+    ```
+
 ### Idempotency request flow
 
 The following sequence diagrams explain how the Idempotency feature behaves under different scenarios.
diff --git a/examples/idempotency/src/integrate_idempotency_with_batch_processor.py b/examples/idempotency/src/integrate_idempotency_with_batch_processor.py
index 957cefb3202..abde83c44e4 100644
--- a/examples/idempotency/src/integrate_idempotency_with_batch_processor.py
+++ b/examples/idempotency/src/integrate_idempotency_with_batch_processor.py
@@ -1,5 +1,6 @@
-from aws_lambda_powertools import Logger
-from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType
+import os
+
+from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response
 from aws_lambda_powertools.utilities.data_classes.sqs_event import SQSRecord
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
@@ -8,13 +9,11 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-logger = Logger()
 processor = BatchProcessor(event_type=EventType.SQS)
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
-config = IdempotencyConfig(
-    event_key_jmespath="messageId",  # see Choosing a payload subset section
-)
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
+config = IdempotencyConfig(event_key_jmespath="messageId")
 
 
 @idempotent_function(data_keyword_argument="record", config=config, persistence_store=dynamodb)
@@ -25,13 +24,9 @@ def record_handler(record: SQSRecord):
 def lambda_handler(event: SQSRecord, context: LambdaContext):
     config.register_lambda_context(context)  # see Lambda timeouts section
 
-    # with Lambda context registered for Idempotency
-    # we can now kick in the Bach processing logic
-    batch = event["Records"]
-    with processor(records=batch, handler=record_handler):
-        # in case you want to access each record processed by your record_handler
-        # otherwise ignore the result variable assignment
-        processed_messages = processor.process()
-        logger.info(processed_messages)
-
-    return processor.response()
+    return process_partial_response(
+        event=event,
+        context=context,
+        processor=processor,
+        record_handler=record_handler,
+    )

From 2344e01e5dbfed714fc44dd86024c6241ba25d63 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 28 May 2024 12:55:28 +1200
Subject: [PATCH 05/41] docs: cleanup handling exceptions

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md                 | 59 +++++++++----------
 .../src/working_with_exceptions.py            | 37 ++++++------
 includes/abbreviations.md                     |  1 +
 3 files changed, 47 insertions(+), 50 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 18ffccc291e..7b1954d94ae 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -305,41 +305,16 @@ Here is an example on how you register the Lambda context in your handler:
 
 ### Handling exceptions
 
-If you are using the `idempotent` decorator on your Lambda handler, any unhandled exceptions that are raised during the code execution will cause **the record in the persistence layer to be deleted**.
-This means that new invocations will execute your code again despite having the same payload. If you don't want the record to be deleted, you need to catch exceptions within the idempotent function and return a successful response.
+There are two failure modes that can cause new invocations to execute your code again despite having the same payload:
 
-<center>
-```mermaid
-sequenceDiagram
-    participant Client
-    participant Lambda
-    participant Persistence Layer
-    Client->>Lambda: Invoke (event)
-    Lambda->>Persistence Layer: Get or set (id=event.search(payload))
-    activate Persistence Layer
-    Note right of Persistence Layer: Locked during this time. Prevents multiple<br/>Lambda invocations with the same<br/>payload running concurrently.
-    Lambda--xLambda: Call handler (event).<br/>Raises exception
-    Lambda->>Persistence Layer: Delete record (id=event.search(payload))
-    deactivate Persistence Layer
-    Lambda-->>Client: Return error response
-```
-<i>Idempotent sequence exception</i>
-</center>
+* **Unhandled exception**. We catch them to delete the idempotency record to prevent inconsistencies, then propagate them.
+* **Persistent layer errors**. We raise **`IdempotencyPersistenceLayerError`** for any persistence layer errors _e.g., remove idempotency record_.
 
-If you are using `idempotent_function`, any unhandled exceptions that are raised _inside_ the decorated function will cause the record in the persistence layer to be deleted, and allow the function to be executed again if retried.
+If an exception is handled or raised **outside** your decorated function, then idempotency will be maintained.
 
-If an Exception is raised _outside_ the scope of the decorated function and after your function has been called, the persistent record will not be affected. In this case, idempotency will be maintained for your decorated function. Example:
-
-=== "Handling exceptions"
-
-    ```python hl_lines="18-22 28 31"
-    --8<-- "examples/idempotency/src/working_with_exceptions.py"
-    ```
-
-???+ warning
-    **We will raise `IdempotencyPersistenceLayerError`** if any of the calls to the persistence layer fail unexpectedly.
-
-    As this happens outside the scope of your decorated function, you are not able to catch it if you're using the `idempotent` decorator on your Lambda handler.
+```python title="working_with_exceptions.py" hl_lines="21 32 38"
+--8<-- "examples/idempotency/src/working_with_exceptions.py"
+```
 
 ### Persistence layers
 
@@ -635,6 +610,26 @@ sequenceDiagram
 <i>Concurrent identical in-flight requests</i>
 </center>
 
+#### Unhandled exception
+
+<center>
+```mermaid
+sequenceDiagram
+    participant Client
+    participant Lambda
+    participant Persistence Layer
+    Client->>Lambda: Invoke (event)
+    Lambda->>Persistence Layer: Get or set (id=event.search(payload))
+    activate Persistence Layer
+    Note right of Persistence Layer: Locked during this time. Prevents multiple<br/>Lambda invocations with the same<br/>payload running concurrently.
+    Lambda--xLambda: Call handler (event).<br/>Raises exception
+    Lambda->>Persistence Layer: Delete record (id=event.search(payload))
+    deactivate Persistence Layer
+    Lambda-->>Client: Return error response
+```
+<i>Idempotent sequence exception</i>
+</center>
+
 #### Lambda request timeout
 
 <center>
diff --git a/examples/idempotency/src/working_with_exceptions.py b/examples/idempotency/src/working_with_exceptions.py
index ff282d5a601..19069fbbc4f 100644
--- a/examples/idempotency/src/working_with_exceptions.py
+++ b/examples/idempotency/src/working_with_exceptions.py
@@ -1,3 +1,5 @@
+import os
+
 import requests
 
 from aws_lambda_powertools.utilities.idempotency import (
@@ -5,33 +7,32 @@
     IdempotencyConfig,
     idempotent_function,
 )
+from aws_lambda_powertools.utilities.idempotency.exceptions import IdempotencyPersistenceLayerError
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 config = IdempotencyConfig()
 
 
-def lambda_handler(event: dict, context: LambdaContext):
-    # If an exception is raised here, no idempotent record will ever get created as the
-    # idempotent function does not get called
-    try:
-        endpoint = "https://jsonplaceholder.typicode.com/comments/"  # change this endpoint to force an exception
-        requests.get(endpoint)
-    except Exception as exc:
-        return str(exc)
-
-    call_external_service(data={"user": "user1", "id": 5})
-
-    # This exception will not cause the idempotent record to be deleted, since it
-    # happens after the decorated function has been successfully called
-    raise Exception
-
-
 @idempotent_function(data_keyword_argument="data", config=config, persistence_store=persistence_layer)
 def call_external_service(data: dict):
+    # Any exception raised will lead to idempotency record to be deleted
     result: requests.Response = requests.post(
         "https://jsonplaceholder.typicode.com/comments/",
-        json={"user": data["user"], "transaction_id": data["id"]},
+        json=data,
     )
     return result.json()
+
+
+def lambda_handler(event: dict, context: LambdaContext):
+    try:
+        call_external_service(data=event)
+    except IdempotencyPersistenceLayerError as e:
+        # No idempotency, but you can decide to error differently.
+        raise RuntimeError(f"Oops, can't talk to persistence layer. Permissions? error: {e}")
+
+    # This exception will not impact the idempotency of 'call_external_service'
+    # because it happens in isolation, or outside their scope.
+    raise SyntaxError("Oops, this shouldn't be here.")
diff --git a/includes/abbreviations.md b/includes/abbreviations.md
index ed52b93fe64..5e0db4dcb27 100644
--- a/includes/abbreviations.md
+++ b/includes/abbreviations.md
@@ -1 +1,2 @@
 *[observability provider]: An AWS Lambda Observability Partner
+*[unhandled exception]: An exception that is not caught by any explicit try/except block

From 663239638972290f2f452beaff93e8b240cecef2 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 24 Jun 2024 21:07:32 +0200
Subject: [PATCH 06/41] docs: move caching to getting started

---
 docs/utilities/idempotency.md                 | 43 ++++++++++---------
 .../src/working_with_local_cache.py           |  5 ++-
 2 files changed, 25 insertions(+), 23 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 7b1954d94ae..eff00f0291a 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -245,6 +245,28 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
     2. This function does the following <br><br>**1**. Receives the dictionary saved into the persistent storage <br>**1** Serializes to `OrderOutput` before `@idempotent` returns back to the caller.
     3. This serializer receives both functions so it knows who to call when to serialize to and from dictionary.
 
+### Using in-memory cache
+
+!!! note "In-memory cache is local to each Lambda execution environment."
+
+You can enable caching with the `use_local_cache` parameter in `IdempotencyConfig`. When enabled, you can adjust cache capacity _(256)_ with `local_cache_max_items`.
+
+By default, caching is disabled since we don't know how big your response could be in relation to your configured memory size.
+
+=== "Enabling cache"
+
+    ```python hl_lines="12"
+    --8<-- "examples/idempotency/src/working_with_local_cache.py"
+    ```
+
+    1. You can adjust cache capacity with [`local_cache_max_items`](#customizing-the-default-behavior) parameter.
+
+=== "Sample event"
+
+    ```json
+    --8<-- "examples/idempotency/src/working_with_local_cache_payload.json"
+    ```
+
 ### Choosing a payload subset for idempotency
 
 ???+ tip "Tip: Dealing with always changing payloads"
@@ -743,27 +765,6 @@ This utility will raise an **`IdempotencyAlreadyInProgressError`** exception if
 
 This is a locking mechanism for correctness. Since we don't know the result from the first invocation yet, we can't safely allow another concurrent execution.
 
-### Using in-memory cache
-
-**By default, in-memory local caching is disabled**, since we don't know how much memory you consume per invocation compared to the maximum configured in your Lambda function.
-
-???+ note "Note: This in-memory cache is local to each Lambda execution environment"
-    This means it will be effective in cases where your function's concurrency is low in comparison to the number of "retry" invocations with the same payload, because cache might be empty.
-
-You can enable in-memory caching with the **`use_local_cache`** parameter:
-
-=== "Caching idempotent transactions in-memory to prevent multiple calls to storage"
-
-    ```python hl_lines="11"
-    --8<-- "examples/idempotency/src/working_with_local_cache.py"
-    ```
-
-=== "Sample event"
-
-    ```json
-    --8<-- "examples/idempotency/src/working_with_local_cache_payload.json"
-    ```
-
 When enabled, the default is to cache a maximum of 256 records in each Lambda execution environment - You can change it with the **`local_cache_max_items`** parameter.
 
 ### Expiring idempotency records
diff --git a/examples/idempotency/src/working_with_local_cache.py b/examples/idempotency/src/working_with_local_cache.py
index 82f39dff2ef..847c66a4d35 100644
--- a/examples/idempotency/src/working_with_local_cache.py
+++ b/examples/idempotency/src/working_with_local_cache.py
@@ -7,8 +7,9 @@
 
 persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
 config = IdempotencyConfig(
-    event_key_jmespath="body",
-    use_local_cache=True,
+    event_key_jmespath="powertools_json(body)",
+    # by default, it holds 256 items in a Least-Recently-Used (LRU) manner
+    use_local_cache=True,  # (1)!
 )
 
 

From 51a7af7d488867b8eca6a2e2226a9f9c09c910bb Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 25 Jun 2024 16:17:50 +0200
Subject: [PATCH 07/41] docs: use env var for DDB table, no hardcode

---
 docs/utilities/idempotency.md                 | 70 ++++++++++---------
 .../src/customize_persistence_layer.py        |  5 +-
 .../src/customize_persistence_layer_redis.py  |  5 +-
 ...g_started_with_idempotency_redis_client.py |  7 +-
 ...g_started_with_idempotency_redis_config.py |  7 +-
 .../integrate_idempotency_with_validator.py   |  5 +-
 .../src/working_with_composite_key.py         |  5 +-
 .../src/working_with_custom_config.py         |  5 +-
 .../src/working_with_custom_session.py        |  5 +-
 ...ith_dataclass_deduced_output_serializer.py |  4 +-
 ..._dataclass_explicitly_output_serializer.py |  4 +-
 .../working_with_idempotency_key_required.py  |  5 +-
 ...otent_function_custom_output_serializer.py |  4 +-
 ...king_with_idempotent_function_dataclass.py |  4 +-
 ...rking_with_idempotent_function_pydantic.py |  5 +-
 .../src/working_with_lambda_timeout.py        |  5 +-
 .../src/working_with_local_cache.py           |  5 +-
 .../src/working_with_payload_subset.py        |  7 +-
 ...with_pydantic_deduced_output_serializer.py |  5 +-
 ...h_pydantic_explicitly_output_serializer.py |  5 +-
 .../src/working_with_record_expiration.py     |  5 +-
 .../src/working_with_response_hook.py         |  4 +-
 .../src/working_with_validation_payload.py    | 12 ++--
 23 files changed, 124 insertions(+), 64 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index eff00f0291a..62e734d1c9c 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -168,7 +168,7 @@ You can use `data_keyword_argument` to tell us the argument to extract an idempo
 
 === "Using Dataclasses"
 
-    ```python title="working_with_idempotent_function_dataclass.py" hl_lines="3-7 11 26 39"
+    ```python title="working_with_idempotent_function_dataclass.py" hl_lines="4-8 12 28 41"
     --8<-- "examples/idempotency/src/working_with_idempotent_function_dataclass.py"
     ```
 
@@ -178,7 +178,7 @@ You can use `data_keyword_argument` to tell us the argument to extract an idempo
 
 === "Using Pydantic"
 
-    ```python title="working_with_idempotent_function_pydantic.py" hl_lines="1-5 10 23 34"
+    ```python title="working_with_idempotent_function_pydantic.py" hl_lines="3-7 12 26 37"
     --8<-- "examples/idempotency/src/working_with_idempotent_function_pydantic.py"
     ```
 
@@ -196,7 +196,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
 
     === "Inferring via the return type"
 
-        ```python hl_lines="6 24 25 32 36 45"
+        ```python hl_lines="8 27 35 38 48"
         --8<-- "examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py"
         ```
 
@@ -206,7 +206,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
 
         Alternatively, you can provide an explicit model as an input to `PydanticSerializer`.
 
-        ```python hl_lines="6 24 25 32 35 44"
+        ```python hl_lines="8 27 35 35 47"
         --8<-- "examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py"
         ```
 
@@ -216,7 +216,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
 
     === "Inferring via the return type"
 
-        ```python hl_lines="8 27-29 36 40 49"
+        ```python hl_lines="9 30 38 41 51"
         --8<-- "examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py"
         ```
 
@@ -226,7 +226,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
 
         Alternatively, you can provide an explicit model as an input to `DataclassSerializer`.
 
-        ```python hl_lines="8 27-29 36 39 48"
+        ```python hl_lines="8 30 38 40 50"
         --8<-- "examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py"
         ```
 
@@ -237,7 +237,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
     * **to_dict**. Function to convert any type to a JSON serializable dictionary before it saves into the persistent storage.
     * **from_dict**. Function to convert from a dictionary retrieved from persistent storage and serialize in its original form.
 
-    ```python hl_lines="8 32 36 40 50 53"
+    ```python hl_lines="9 34 38 42 52 54 64"
     --8<-- "examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py"
     ```
 
@@ -255,7 +255,7 @@ By default, caching is disabled since we don't know how big your response could
 
 === "Enabling cache"
 
-    ```python hl_lines="12"
+    ```python hl_lines="15"
     --8<-- "examples/idempotency/src/working_with_local_cache.py"
     ```
 
@@ -290,7 +290,7 @@ Imagine the function runs successfully, but the client never receives the respon
 
 === "Payment function"
 
-    ```python hl_lines="5-9 16 30"
+    ```python hl_lines="6-10 18 31"
     --8<-- "examples/idempotency/src/working_with_payload_subset.py"
     ```
 
@@ -308,8 +308,14 @@ Imagine the function runs successfully, but the client never receives the respon
 To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/){target="_blank"},
 Powertools for AWS Lambda (Python) calculates and includes the remaining invocation available time as part of the idempotency record.
 
-???+ example
-    If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will execute the invocation again as if it was in the `EXPIRED` state (e.g, `expire_seconds` field elapsed).
+To prevent extended failures, use **`register_lambda_context`** function from your idempotency config to calculate and include the remaining invocation time in your idempotency record.
+
+```python title="working_with_lambda_timeout.py" hl_lines="14 23"
+--8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
+```
+
+???+ example "Mechanics"
+    If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will run the invocation again as if it was in the `EXPIRED` state.
 
     This means that if an invocation expired during execution, it will be quickly executed again on the next retry.
 
@@ -344,7 +350,9 @@ If an exception is handled or raised **outside** your decorated function, then i
 
 This persistence layer is built-in, allowing you to use an existing DynamoDB table or create a new one dedicated to idempotency state (recommended).
 
-=== "Customizing DynamoDBPersistenceLayer to suit your table structure"
+```python title="customize_persistence_layer.py" hl_lines="10-18"
+--8<-- "examples/idempotency/src/customize_persistence_layer.py"
+```
 
     ```python hl_lines="7-15"
     --8<-- "examples/idempotency/src/customize_persistence_layer.py"
@@ -375,12 +383,12 @@ For a quick start, initialize `RedisCachePersistenceLayer` and pass your cluster
 For security, we enforce SSL connections by default; to disable it, set `ssl=False`.
 
 === "Redis quick start"
-    ```python hl_lines="7-9 12 26"
+    ```python title="getting_started_with_idempotency_redis_config.py" hl_lines="8-10 14 27"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_config.py"
     ```
 
 === "Using an existing Redis client"
-    ```python hl_lines="4 9-11 14 22 36"
+    ```python title="getting_started_with_idempotency_redis_client.py" hl_lines="5 10-11 16 24 38"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_client.py"
     ```
 
@@ -438,11 +446,9 @@ You can customize attribute names when instantiating `RedisCachePersistenceLayer
 | **data_attr**               |          | `data`                   | Stores results of successfully executed Lambda handlers                                       |
 | **validation_key_attr**     |          | `validation`             | Hashed representation of the parts of the event used for validation                           |
 
-=== "Customizing RedisPersistenceLayer to suit your data structure"
-
-    ```python hl_lines="9-16"
-    --8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
-    ```
+```python title="customize_persistence_layer_redis.py" hl_lines="15-18"
+--8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
+```
 
 ### Common use cases
 
@@ -777,7 +783,7 @@ You can change this window with the **`expires_after_seconds`** parameter:
 
 === "Adjusting idempotency record expiration"
 
-    ```python hl_lines="11"
+    ```python hl_lines="14"
     --8<-- "examples/idempotency/src/working_with_record_expiration.py"
     ```
 
@@ -811,7 +817,7 @@ With **`payload_validation_jmespath`**, you can provide an additional JMESPath e
 
 === "Payload validation"
 
-    ```python hl_lines="12 20 28"
+    ```python hl_lines="16 25 32"
     --8<-- "examples/idempotency/src/working_with_validation_payload.py"
     ```
 
@@ -847,7 +853,7 @@ This means that we will raise **`IdempotencyKeyError`** if the evaluation of **`
 
 === "Idempotency key required"
 
-    ```python hl_lines="11"
+    ```python hl_lines="14"
     --8<-- "examples/idempotency/src/working_with_idempotency_key_required.py"
     ```
 
@@ -869,13 +875,13 @@ The **`boto_config`** and **`boto3_session`** parameters enable you to pass in a
 
 === "Custom session"
 
-    ```python hl_lines="1 11 13"
+    ```python hl_lines="3 13 16"
     --8<-- "examples/idempotency/src/working_with_custom_session.py"
     ```
 
 === "Custom config"
 
-    ```python hl_lines="1 11 13"
+    ```python hl_lines="3 13 16"
     --8<-- "examples/idempotency/src/working_with_custom_config.py"
     ```
 
@@ -895,7 +901,7 @@ You can optionally set a static value for the partition key using the `static_pk
 
 === "Reusing a DynamoDB table that uses a composite primary key"
 
-    ```python hl_lines="7"
+    ```python hl_lines="10"
     --8<-- "examples/idempotency/src/working_with_composite_key.py"
     ```
 
@@ -924,11 +930,9 @@ You can create your own persistent store from scratch by inheriting the `BasePer
 * **`_update_record()`** – Updates an item in the persistence store.
 * **`_delete_record()`** – Removes an item from the persistence store.
 
-=== "Bring your own persistent store"
-
-    ```python hl_lines="8 18 65 74 96 124"
-    --8<-- "examples/idempotency/src/bring_your_own_persistent_store.py"
-    ```
+```python title="bring_your_own_persistent_store.py" hl_lines="8 18 65 74 96 124"
+--8<-- "examples/idempotency/src/bring_your_own_persistent_store.py"
+```
 
 ???+ danger
     Pay attention to the documentation for each - you may need to perform additional checks inside these methods to ensure the idempotency guarantees remain intact.
@@ -941,7 +945,7 @@ You can set up a `response_hook` in the `IdempotentConfig` class to manipulate t
 
 === "Using an Idempotent Response Hook"
 
-    ```python hl_lines="19 21 27 34"
+    ```python hl_lines="20 22 28 36"
     --8<-- "examples/idempotency/src/working_with_response_hook.py"
     ```
 
@@ -971,7 +975,7 @@ When using response hooks to manipulate returned data from idempotent operations
 
 See [Batch integration](#batch-integration) above.
 
-### Validation utility
+### JSON Schema Validation
 
 The idempotency utility can be used with the `validator` decorator. Ensure that idempotency is the innermost decorator.
 
@@ -983,7 +987,7 @@ The idempotency utility can be used with the `validator` decorator. Ensure that
 
 === "Using Idempotency with JSONSchema Validation utility"
 
-    ```python hl_lines="13"
+    ```python hl_lines="16"
     --8<-- "examples/idempotency/src/integrate_idempotency_with_validator.py"
     ```
 
diff --git a/examples/idempotency/src/customize_persistence_layer.py b/examples/idempotency/src/customize_persistence_layer.py
index 26409191ca9..231ea95e2c0 100644
--- a/examples/idempotency/src/customize_persistence_layer.py
+++ b/examples/idempotency/src/customize_persistence_layer.py
@@ -1,11 +1,14 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     idempotent,
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
+table = os.getenv("IDEMPOTENCY_TABLE")
 persistence_layer = DynamoDBPersistenceLayer(
-    table_name="IdempotencyTable",
+    table_name=table,
     key_attr="idempotency_key",
     expiry_attr="expires_at",
     in_progress_expiry_attr="in_progress_expires_at",
diff --git a/examples/idempotency/src/customize_persistence_layer_redis.py b/examples/idempotency/src/customize_persistence_layer_redis.py
index 7db3d1b53ea..566efc59350 100644
--- a/examples/idempotency/src/customize_persistence_layer_redis.py
+++ b/examples/idempotency/src/customize_persistence_layer_redis.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     idempotent,
 )
@@ -6,8 +8,9 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
+redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
 persistence_layer = RedisCachePersistenceLayer(
-    host="localhost",
+    host=redis_endpoint,
     port=6379,
     in_progress_expiry_attr="in_progress_expiration",
     status_attr="status",
diff --git a/examples/idempotency/src/getting_started_with_idempotency_redis_client.py b/examples/idempotency/src/getting_started_with_idempotency_redis_client.py
index f06d059fad4..38c34951892 100644
--- a/examples/idempotency/src/getting_started_with_idempotency_redis_client.py
+++ b/examples/idempotency/src/getting_started_with_idempotency_redis_client.py
@@ -1,3 +1,4 @@
+import os
 from dataclasses import dataclass, field
 from uuid import uuid4
 
@@ -11,8 +12,9 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
+redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
 client = Redis(
-    host="localhost",
+    host=redis_endpoint,
     port=6379,
     socket_connect_timeout=5,
     socket_timeout=5,
@@ -29,8 +31,7 @@ class Payment:
     payment_id: str = field(default_factory=lambda: f"{uuid4()}")
 
 
-class PaymentError(Exception):
-    ...
+class PaymentError(Exception): ...
 
 
 @idempotent(persistence_store=persistence_layer)
diff --git a/examples/idempotency/src/getting_started_with_idempotency_redis_config.py b/examples/idempotency/src/getting_started_with_idempotency_redis_config.py
index de9c6526059..b4451be438f 100644
--- a/examples/idempotency/src/getting_started_with_idempotency_redis_config.py
+++ b/examples/idempotency/src/getting_started_with_idempotency_redis_config.py
@@ -1,3 +1,4 @@
+import os
 from dataclasses import dataclass, field
 from uuid import uuid4
 
@@ -9,7 +10,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = RedisCachePersistenceLayer(host="localhost", port=6379)
+redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
+persistence_layer = RedisCachePersistenceLayer(host=redis_endpoint, port=6379)
 
 
 @dataclass
@@ -19,8 +21,7 @@ class Payment:
     payment_id: str = field(default_factory=lambda: f"{uuid4()}")
 
 
-class PaymentError(Exception):
-    ...
+class PaymentError(Exception): ...
 
 
 @idempotent(persistence_store=persistence_layer)
diff --git a/examples/idempotency/src/integrate_idempotency_with_validator.py b/examples/idempotency/src/integrate_idempotency_with_validator.py
index af833951446..e69897205a4 100644
--- a/examples/idempotency/src/integrate_idempotency_with_validator.py
+++ b/examples/idempotency/src/integrate_idempotency_with_validator.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
@@ -6,8 +8,9 @@
 from aws_lambda_powertools.utilities.typing import LambdaContext
 from aws_lambda_powertools.utilities.validation import envelopes, validator
 
+table = os.getenv("IDEMPOTENCY_TABLE")
 config = IdempotencyConfig(event_key_jmespath='["message", "username"]')
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 
 @validator(envelope=envelopes.API_GATEWAY_HTTP)
diff --git a/examples/idempotency/src/working_with_composite_key.py b/examples/idempotency/src/working_with_composite_key.py
index f1b70cba99a..33345706078 100644
--- a/examples/idempotency/src/working_with_composite_key.py
+++ b/examples/idempotency/src/working_with_composite_key.py
@@ -1,10 +1,13 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     idempotent,
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable", sort_key_attr="sort_key")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table, sort_key_attr="sort_key")
 
 
 @idempotent(persistence_store=persistence_layer)
diff --git a/examples/idempotency/src/working_with_custom_config.py b/examples/idempotency/src/working_with_custom_config.py
index 30539f88f3c..ddbfc08cd7e 100644
--- a/examples/idempotency/src/working_with_custom_config.py
+++ b/examples/idempotency/src/working_with_custom_config.py
@@ -1,3 +1,5 @@
+import os
+
 from botocore.config import Config
 
 from aws_lambda_powertools.utilities.idempotency import (
@@ -10,7 +12,8 @@
 # See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore-config
 boto_config = Config()
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable", boto_config=boto_config)
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table, boto_config=boto_config)
 
 config = IdempotencyConfig(event_key_jmespath="body")
 
diff --git a/examples/idempotency/src/working_with_custom_session.py b/examples/idempotency/src/working_with_custom_session.py
index aae89f8a3fe..ef324dab721 100644
--- a/examples/idempotency/src/working_with_custom_session.py
+++ b/examples/idempotency/src/working_with_custom_session.py
@@ -1,3 +1,5 @@
+import os
+
 import boto3
 
 from aws_lambda_powertools.utilities.idempotency import (
@@ -10,7 +12,8 @@
 # See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#module-boto3.session
 boto3_session = boto3.session.Session()
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable", boto3_session=boto3_session)
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table, boto3_session=boto3_session)
 
 config = IdempotencyConfig(event_key_jmespath="body")
 
diff --git a/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py b/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py
index c59c8b078f7..bb4592768e7 100644
--- a/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py
+++ b/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py
@@ -1,3 +1,4 @@
+import os
 from dataclasses import dataclass
 
 from aws_lambda_powertools.utilities.idempotency import (
@@ -8,7 +9,8 @@
 from aws_lambda_powertools.utilities.idempotency.serialization.dataclass import DataclassSerializer
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
 
diff --git a/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py b/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py
index fc2412fb1a2..2f6adea057c 100644
--- a/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py
+++ b/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py
@@ -1,3 +1,4 @@
+import os
 from dataclasses import dataclass
 
 from aws_lambda_powertools.utilities.idempotency import (
@@ -8,7 +9,8 @@
 from aws_lambda_powertools.utilities.idempotency.serialization.dataclass import DataclassSerializer
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
 
diff --git a/examples/idempotency/src/working_with_idempotency_key_required.py b/examples/idempotency/src/working_with_idempotency_key_required.py
index 347740ab4a3..dea3965f9b7 100644
--- a/examples/idempotency/src/working_with_idempotency_key_required.py
+++ b/examples/idempotency/src/working_with_idempotency_key_required.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
@@ -5,7 +7,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath='["user.uid", "order_id"]',
     raise_on_no_idempotency_key=True,
diff --git a/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py b/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py
index a62961fa5f3..74d22feb83f 100644
--- a/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py
+++ b/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py
@@ -1,3 +1,4 @@
+import os
 from typing import Dict, Type
 
 from aws_lambda_powertools.utilities.idempotency import (
@@ -8,7 +9,8 @@
 from aws_lambda_powertools.utilities.idempotency.serialization.custom_dict import CustomDictSerializer
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
 
diff --git a/examples/idempotency/src/working_with_idempotent_function_dataclass.py b/examples/idempotency/src/working_with_idempotent_function_dataclass.py
index bd3ac0158b2..b4cc4ab0b53 100644
--- a/examples/idempotency/src/working_with_idempotent_function_dataclass.py
+++ b/examples/idempotency/src/working_with_idempotent_function_dataclass.py
@@ -1,3 +1,4 @@
+import os
 from dataclasses import dataclass
 
 from aws_lambda_powertools.utilities.idempotency import (
@@ -7,7 +8,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
 
diff --git a/examples/idempotency/src/working_with_idempotent_function_pydantic.py b/examples/idempotency/src/working_with_idempotent_function_pydantic.py
index 5dfd42ae0a8..2092fa1b45b 100644
--- a/examples/idempotency/src/working_with_idempotent_function_pydantic.py
+++ b/examples/idempotency/src/working_with_idempotent_function_pydantic.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
@@ -6,7 +8,8 @@
 from aws_lambda_powertools.utilities.parser import BaseModel
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
 
diff --git a/examples/idempotency/src/working_with_lambda_timeout.py b/examples/idempotency/src/working_with_lambda_timeout.py
index 82b8130b6b7..9a4fffb526b 100644
--- a/examples/idempotency/src/working_with_lambda_timeout.py
+++ b/examples/idempotency/src/working_with_lambda_timeout.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.data_classes.sqs_event import SQSRecord
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
@@ -6,7 +8,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 config = IdempotencyConfig()
 
diff --git a/examples/idempotency/src/working_with_local_cache.py b/examples/idempotency/src/working_with_local_cache.py
index 847c66a4d35..ce6b55dc981 100644
--- a/examples/idempotency/src/working_with_local_cache.py
+++ b/examples/idempotency/src/working_with_local_cache.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
@@ -5,7 +7,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath="powertools_json(body)",
     # by default, it holds 256 items in a Least-Recently-Used (LRU) manner
diff --git a/examples/idempotency/src/working_with_payload_subset.py b/examples/idempotency/src/working_with_payload_subset.py
index 9fcc828fe1d..73030630ce6 100644
--- a/examples/idempotency/src/working_with_payload_subset.py
+++ b/examples/idempotency/src/working_with_payload_subset.py
@@ -1,4 +1,5 @@
 import json
+import os
 from dataclasses import dataclass, field
 from uuid import uuid4
 
@@ -9,7 +10,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 # Deserialize JSON string under the "body" key
 # then extract "user" and "product_id" data
@@ -23,8 +25,7 @@ class Payment:
     payment_id: str = field(default_factory=lambda: f"{uuid4()}")
 
 
-class PaymentError(Exception):
-    ...
+class PaymentError(Exception): ...
 
 
 @idempotent(config=config, persistence_store=persistence_layer)
diff --git a/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py b/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py
index f24fda81e86..5ea5b94c2b2 100644
--- a/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py
+++ b/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
@@ -7,7 +9,8 @@
 from aws_lambda_powertools.utilities.parser import BaseModel
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
 
diff --git a/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py b/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py
index 7bd63dfcd9f..af0c0d8bde3 100644
--- a/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py
+++ b/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
@@ -7,7 +9,8 @@
 from aws_lambda_powertools.utilities.parser import BaseModel
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
 
diff --git a/examples/idempotency/src/working_with_record_expiration.py b/examples/idempotency/src/working_with_record_expiration.py
index 738b4749ebc..ec8a7cae842 100644
--- a/examples/idempotency/src/working_with_record_expiration.py
+++ b/examples/idempotency/src/working_with_record_expiration.py
@@ -1,3 +1,5 @@
+import os
+
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
@@ -5,7 +7,8 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath="body",
     expires_after_seconds=5 * 60,  # 5 minutes
diff --git a/examples/idempotency/src/working_with_response_hook.py b/examples/idempotency/src/working_with_response_hook.py
index 2c2208d25a5..e851131781c 100644
--- a/examples/idempotency/src/working_with_response_hook.py
+++ b/examples/idempotency/src/working_with_response_hook.py
@@ -1,4 +1,5 @@
 import datetime
+import os
 import uuid
 from typing import Dict
 
@@ -30,7 +31,8 @@ def my_response_hook(response: Dict, idempotent_data: DataRecord) -> Dict:
     return response
 
 
-dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
+table = os.getenv("IDEMPOTENCY_TABLE")
+dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(response_hook=my_response_hook)
 
 
diff --git a/examples/idempotency/src/working_with_validation_payload.py b/examples/idempotency/src/working_with_validation_payload.py
index d81e7d183bd..ecb047e0116 100644
--- a/examples/idempotency/src/working_with_validation_payload.py
+++ b/examples/idempotency/src/working_with_validation_payload.py
@@ -1,3 +1,4 @@
+import os
 from dataclasses import dataclass, field
 from uuid import uuid4
 
@@ -8,8 +9,12 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
-config = IdempotencyConfig(event_key_jmespath='["user_id", "product_id"]', payload_validation_jmespath="amount")
+table = os.getenv("IDEMPOTENCY_TABLE")
+persistence_layer = DynamoDBPersistenceLayer(table_name=table)
+config = IdempotencyConfig(
+    event_key_jmespath='["user_id", "product_id"]',
+    payload_validation_jmespath="amount",
+)
 
 
 @dataclass
@@ -21,8 +26,7 @@ class Payment:
     payment_id: str = field(default_factory=lambda: f"{uuid4()}")
 
 
-class PaymentError(Exception):
-    ...
+class PaymentError(Exception): ...
 
 
 @idempotent(config=config, persistence_store=persistence_layer)

From 5807a1f0a2604d96f2f77af369e0b0de719ed5f9 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 25 Jun 2024 17:29:45 +0200
Subject: [PATCH 08/41] docs: moved expiration window to getting started;
 updated example to set to 24h

---
 docs/utilities/idempotency.md                 | 62 +++++++++----------
 .../src/working_with_record_expiration.py     |  2 +-
 2 files changed, 30 insertions(+), 34 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 62e734d1c9c..04e51ad8095 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -300,6 +300,35 @@ Imagine the function runs successfully, but the client never receives the respon
     --8<-- "examples/idempotency/src/working_with_payload_subset_payload.json"
     ```
 
+### Adjusting expiration window
+
+!!! note "We expire idempotency records after **an hour** (3600 seconds). After that, a transaction with the same payload [will not be considered idempotent](#expired-idempotency-records)."
+
+You can change this expiration window with the **`expires_after_seconds`** parameter. There is no limit on how long this expiration window can be set to.
+
+=== "Adjusting expiration window"
+
+    ```python hl_lines="14"
+    --8<-- "examples/idempotency/src/working_with_record_expiration.py"
+    ```
+
+=== "Sample event"
+
+    ```json
+    --8<-- "examples/idempotency/src/working_with_record_expiration_payload.json"
+    ```
+
+???+ important "Idempotency record expiration vs DynamoDB time-to-live (TTL)"
+    [DynamoDB TTL is a feature](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html){target="_blank"} to remove items after a certain period of time, it may occur within 48 hours of expiration.
+
+    We don't rely on DynamoDB or any persistence storage layer to determine whether a record is expired to avoid eventual inconsistency states.
+
+    Instead, Idempotency records saved in the storage layer contain timestamps that can be verified upon retrieval and double checked within Idempotency feature.
+
+    **Why?**
+
+    A record might still be valid (`COMPLETE`) when we retrieved, but in some rare cases it might expire a second later. A record could also be [cached in memory](#using-in-memory-cache). You might also want to have idempotent transactions that should expire in seconds.
+
 ### Lambda timeouts
 
 ???+ note
@@ -773,39 +802,6 @@ This is a locking mechanism for correctness. Since we don't know the result from
 
 When enabled, the default is to cache a maximum of 256 records in each Lambda execution environment - You can change it with the **`local_cache_max_items`** parameter.
 
-### Expiring idempotency records
-
-!!! note "By default, we expire idempotency records after **an hour** (3600 seconds)."
-
-In most cases, it is not desirable to store the idempotency records forever. Rather, you want to guarantee that the same payload won't be executed within a period of time.
-
-You can change this window with the **`expires_after_seconds`** parameter:
-
-=== "Adjusting idempotency record expiration"
-
-    ```python hl_lines="14"
-    --8<-- "examples/idempotency/src/working_with_record_expiration.py"
-    ```
-
-=== "Sample event"
-
-    ```json
-    --8<-- "examples/idempotency/src/working_with_record_expiration_payload.json"
-    ```
-
-This will mark any records older than 5 minutes as expired, and [your function will be executed as normal if it is invoked with a matching payload](#expired-idempotency-records).
-
-???+ important "Idempotency record expiration vs DynamoDB time-to-live (TTL)"
-    [DynamoDB TTL is a feature](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html){target="_blank"} to remove items after a certain period of time, it may occur within 48 hours of expiration.
-
-    We don't rely on DynamoDB or any persistence storage layer to determine whether a record is expired to avoid eventual inconsistency states.
-
-    Instead, Idempotency records saved in the storage layer contain timestamps that can be verified upon retrieval and double checked within Idempotency feature.
-
-    **Why?**
-
-    A record might still be valid (`COMPLETE`) when we retrieved, but in some rare cases it might expire a second later. A record could also be [cached in memory](#using-in-memory-cache). You might also want to have idempotent transactions that should expire in seconds.
-
 ### Payload validation
 
 ???+ question "Question: What if your function is invoked with the same payload except some outer parameters have changed?"
diff --git a/examples/idempotency/src/working_with_record_expiration.py b/examples/idempotency/src/working_with_record_expiration.py
index ec8a7cae842..5b222ec7cdc 100644
--- a/examples/idempotency/src/working_with_record_expiration.py
+++ b/examples/idempotency/src/working_with_record_expiration.py
@@ -11,7 +11,7 @@
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath="body",
-    expires_after_seconds=5 * 60,  # 5 minutes
+    expires_after_seconds=24 * 60 * 60,  # 24 hours
 )
 
 

From 9f90c397b735c20cf9cf1d4925c9280716b9b2d3 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 25 Jun 2024 18:30:10 +0200
Subject: [PATCH 09/41] docs: include IdempotencyValidationError in example

---
 docs/utilities/idempotency.md                          |  4 +---
 .../idempotency/src/working_with_validation_payload.py | 10 ++++++++++
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 04e51ad8095..ffe428f5062 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -800,8 +800,6 @@ This utility will raise an **`IdempotencyAlreadyInProgressError`** exception if
 
 This is a locking mechanism for correctness. Since we don't know the result from the first invocation yet, we can't safely allow another concurrent execution.
 
-When enabled, the default is to cache a maximum of 256 records in each Lambda execution environment - You can change it with the **`local_cache_max_items`** parameter.
-
 ### Payload validation
 
 ???+ question "Question: What if your function is invoked with the same payload except some outer parameters have changed?"
@@ -813,7 +811,7 @@ With **`payload_validation_jmespath`**, you can provide an additional JMESPath e
 
 === "Payload validation"
 
-    ```python hl_lines="16 25 32"
+    ```python hl_lines="20 29 36"
     --8<-- "examples/idempotency/src/working_with_validation_payload.py"
     ```
 
diff --git a/examples/idempotency/src/working_with_validation_payload.py b/examples/idempotency/src/working_with_validation_payload.py
index ecb047e0116..80ab43ad53d 100644
--- a/examples/idempotency/src/working_with_validation_payload.py
+++ b/examples/idempotency/src/working_with_validation_payload.py
@@ -2,13 +2,17 @@
 from dataclasses import dataclass, field
 from uuid import uuid4
 
+from aws_lambda_powertools import Logger
 from aws_lambda_powertools.utilities.idempotency import (
     DynamoDBPersistenceLayer,
     IdempotencyConfig,
     idempotent,
 )
+from aws_lambda_powertools.utilities.idempotency.exceptions import IdempotencyValidationError
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
+logger = Logger()
+
 table = os.getenv("IDEMPOTENCY_TABLE")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
@@ -38,6 +42,12 @@ def lambda_handler(event: dict, context: LambdaContext):
             "message": "success",
             "statusCode": 200,
         }
+    except IdempotencyValidationError:
+        logger.exception("Payload tampering detected", payment=payment, failure_type="validation")
+        return {
+            "message": "Unable to process payment at this time. Try again later.",
+            "statusCode": 500,
+        }
     except Exception as exc:
         raise PaymentError(f"Error creating payment {str(exc)}")
 

From 66549a67673538c29902fd927ce18036995b4f30 Mon Sep 17 00:00:00 2001
From: Leandro Damascena <lcdama@amazon.pt>
Date: Wed, 26 Jun 2024 17:24:49 +0100
Subject: [PATCH 10/41] Fixing errors on Redis examples

---
 .../idempotency/src/using_redis_client_with_aws_secrets.py    | 4 ++--
 .../idempotency/src/using_redis_client_with_local_certs.py    | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/examples/idempotency/src/using_redis_client_with_aws_secrets.py b/examples/idempotency/src/using_redis_client_with_aws_secrets.py
index ddf2659e815..ee9e6d78c45 100644
--- a/examples/idempotency/src/using_redis_client_with_aws_secrets.py
+++ b/examples/idempotency/src/using_redis_client_with_aws_secrets.py
@@ -13,8 +13,8 @@
 redis_values: dict[str, Any] = parameters.get_secret("redis_info", transform="json")  # (1)!
 
 redis_client = Redis(
-    host=redis_values.get("REDIS_HOST"),
-    port=redis_values.get("REDIS_PORT"),
+    host=redis_values.get("REDIS_HOST", "localhost"),
+    port=redis_values.get("REDIS_PORT", 6379),
     password=redis_values.get("REDIS_PASSWORD"),
     decode_responses=True,
     socket_timeout=10.0,
diff --git a/examples/idempotency/src/using_redis_client_with_local_certs.py b/examples/idempotency/src/using_redis_client_with_local_certs.py
index 946fd6a064d..2b6a5892c5b 100644
--- a/examples/idempotency/src/using_redis_client_with_local_certs.py
+++ b/examples/idempotency/src/using_redis_client_with_local_certs.py
@@ -15,8 +15,8 @@
 
 
 redis_client = Redis(
-    host=redis_values.get("REDIS_HOST"),
-    port=redis_values.get("REDIS_PORT"),
+    host=redis_values.get("REDIS_HOST", "localhost"),
+    port=redis_values.get("REDIS_PORT", 6379),
     password=redis_values.get("REDIS_PASSWORD"),
     decode_responses=True,
     socket_timeout=10.0,

From 0a46164a6b4fd8d216bff543f662485a4ce08093 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 24 Jun 2024 18:26:03 +0200
Subject: [PATCH 11/41] docs(config): add social links

---
 mkdocs.yml | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/mkdocs.yml b/mkdocs.yml
index d2bab86cd22..988f7dc5f06 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -147,3 +147,13 @@ extra:
   version:
     provider: mike
     default: latest
+  social:
+    - icon: fontawesome/brands/discord
+      link: https://discord.gg/B8zZKbbyET
+      name: Discord Server for Powertools for AWS
+    - icon: material/web
+      link: https://powertools.aws.dev/
+      name: Official website for Powertools for AWS
+    - icon: simple/python
+      link: https://pypi.org/project/aws-lambda-powertools/
+      name: PyPi package

From dbbd6f5caa654ac298c3b4ceb1b8912ac724e73e Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Thu, 23 May 2024 12:05:22 +1200
Subject: [PATCH 12/41] docs(idempotency): cleanup intro and key features

---
 docs/utilities/idempotency.md | 86 ++---------------------------------
 1 file changed, 5 insertions(+), 81 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index ffe428f5062..9e97dcf41e5 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -5,16 +5,14 @@ description: Utility
 
 <!-- markdownlint-disable MD051 -->
 
-The idempotency utility provides a simple solution to convert your Lambda functions into idempotent operations which are safe to retry.
+The idempotency utility allows you to retry operations with the same input within a time window, producing the same output.
 
 ## Key features
 
-* Prevent Lambda handler from executing more than once on the same event payload during a time window
-* Ensure Lambda handler returns the same result when called with the same payload
-* Select a subset of the event as the idempotency key using JMESPath expressions
-* Set a time window in which records with the same payload should be considered duplicates
-* Expires in-progress executions if the Lambda function times out halfway through
-* Support Amazon DynamoDB and Redis as persistence layers
+* Produces the same result when a function is called repeatedly with the same idempotency key
+* Choose your idempotency key from one or more fields, or entire payload
+* Safeguard concurrent requests, timeouts, missing idempotency keys, and payload tampering
+* Support for Amazon DynamoDB, Redis, and bring your own persistence layer
 
 ## Terminology
 
@@ -421,53 +419,6 @@ For security, we enforce SSL connections by default; to disable it, set `ssl=Fal
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_client.py"
     ```
 
-=== "Sample event"
-
-    ```json
-    --8<-- "examples/idempotency/src/getting_started_with_idempotency_payload.json"
-    ```
-
-##### Redis SSL connections
-
-We recommend using AWS Secrets Manager to store and rotate certificates safely, and the [Parameters feature](./parameters.md){target="_blank"} to fetch and cache optimally.
-
-For advanced configurations, we also recommend using an existing Redis client for optimal compatibility like SSL certificates and timeout.
-
-=== "Advanced configuration using AWS Secrets"
-    ```python hl_lines="9-11 13 15 25"
-    --8<-- "examples/idempotency/src/using_redis_client_with_aws_secrets.py"
-    ```
-
-    1. JSON stored:
-    ```json
-    {
-    "REDIS_ENDPOINT": "127.0.0.1",
-    "REDIS_PORT": "6379",
-    "REDIS_PASSWORD": "redis-secret"
-    }
-    ```
-
-=== "Advanced configuration with local certificates"
-    ```python hl_lines="14 25-27"
-    --8<-- "examples/idempotency/src/using_redis_client_with_local_certs.py"
-    ```
-
-    1. JSON stored:
-    ```json
-    {
-    "REDIS_ENDPOINT": "127.0.0.1",
-    "REDIS_PORT": "6379",
-    "REDIS_PASSWORD": "redis-secret"
-    }
-    ```
-    2. redis_user.crt file stored in the "certs" directory of your Lambda function
-    3. redis_user_private.key file stored in the "certs" directory of your Lambda function
-    4. redis_ca.pem file stored in the "certs" directory of your Lambda function
-
-##### Redis defaults
-
-You can customize attribute names when instantiating `RedisCachePersistenceLayer` with the following parameters:
-
 | Parameter                   | Required | Default                  | Description                                                                                   |
 | --------------------------- | -------- | ------------------------ | --------------------------------------------------------------------------------------------- |
 | **in_progress_expiry_attr** |          | `in_progress_expiration` | Unix timestamp of when record expires while in progress (in case of the invocation times out) |
@@ -475,33 +426,6 @@ You can customize attribute names when instantiating `RedisCachePersistenceLayer
 | **data_attr**               |          | `data`                   | Stores results of successfully executed Lambda handlers                                       |
 | **validation_key_attr**     |          | `validation`             | Hashed representation of the parts of the event used for validation                           |
 
-```python title="customize_persistence_layer_redis.py" hl_lines="15-18"
---8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
-```
-
-### Common use cases
-
-#### Batch integration
-
-You can can easily integrate with [Batch](batch.md){target="_blank"} with the [idempotent_function decorator](#idempotent_function-decorator) to handle idempotency per message/record in a given batch.
-
-???+ "Choosing an unique batch record attribute"
-    In this example, we choose `messageId` as our idempotency key since we know it'll be unique.
-
-    Depending on your use case, it might be more accurate [to choose another field](#choosing-a-payload-subset-for-idempotency) your producer intentionally set to define uniqueness.
-
-=== "Integration with Batch Processor"
-
-    ```python title="integrate_idempotency_with_batch_processor.py" hl_lines="3 16 19 25 27"
-    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor.py"
-    ```
-
-=== "Sample event"
-
-    ```json title="integrate_idempotency_with_batch_processor_payload.json" hl_lines="4"
-    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor_payload.json"
-    ```
-
 ### Idempotency request flow
 
 The following sequence diagrams explain how the Idempotency feature behaves under different scenarios.

From 6d58fae35d134352dd63417edcb69ce2aa6d3d0b Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Thu, 23 May 2024 12:12:20 +1200
Subject: [PATCH 13/41] docs(idempotency): cleanup getting started ddb vs redis

---
 docs/utilities/idempotency.md | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 9e97dcf41e5..dcc7cad2db6 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -50,8 +50,7 @@ classDiagram
 
 ## Getting started
 
-???+ note
-    This section uses DynamoDB as the default idempotent persistence storage layer. If you are interested in using Redis as the persistence storage layer, check out the [Redis as persistence storage layer](#redis-as-persistent-storage-layer-provider) Section.
+We use Amazon DynamoDB as the default persistence layer in the documentation. If you prefer Redis, you can learn more from [this section](#redis-as-persistent-storage-layer-provider).
 
 ### IAM Permissions
 

From 11bc9fb6481cad1cf542a0e06753698162e48467 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Thu, 23 May 2024 13:45:20 +1200
Subject: [PATCH 14/41] docs(idempotency): break iam permissions into table;
 IAM permission to clipboard

---
 docs/utilities/idempotency.md | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index dcc7cad2db6..803db24104e 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -54,10 +54,18 @@ We use Amazon DynamoDB as the default persistence layer in the documentation. If
 
 ### IAM Permissions
 
-Your Lambda function IAM Role must have `dynamodb:GetItem`, `dynamodb:PutItem`, `dynamodb:UpdateItem` and `dynamodb:DeleteItem` IAM permissions before using this feature.
+When using Amazon DynamoDB as the persistence layer, you will need the following IAM permissions:
 
-???+ note
-    If you're using our example [AWS Serverless Application Model (SAM)](#required-resources), [AWS Cloud Development Kit (CDK)](#required-resources), or [Terraform](#required-resources) it already adds the required permissions.
+| IAM Permission                       | Operation                                                                |
+| ------------------------------------ | ------------------------------------------------------------------------ |
+| **`dynamodb:GetItem`**{: .copyMe}    | Retrieve idempotent record                                               |
+| **`dynamodb:PutItem`**{: .copyMe}    | New idempotent records, replace expired idempotent records               |
+| **`dynamodb:UpdateItem`**{: .copyMe} | Complete idempotency transaction, and/or update idempotent records state |
+| **`dynamodb:DeleteItem`**{: .copyMe} | Delete idempotent records for unsuccessful idempotency transactions      |
+
+**First time setting it up?**
+
+We provide Infrastrucure as Code examples with [AWS Serverless Application Model (SAM)](#aws-serverless-application-model-sam-example), [AWS Cloud Development Kit (CDK)](#aws-cloud-development-kit-cdk), and [Terraform](#terraform) with the required permissions.
 
 ### Required resources
 

From e65e8bd9763224bc1f9faed263a1065c43511fc0 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Thu, 23 May 2024 15:12:07 +1200
Subject: [PATCH 15/41] docs(idempotency): cleanup dynamodb required resource;
 break subsections and update nav

---
 docs/utilities/idempotency.md | 49 +++++++++++------------------------
 1 file changed, 15 insertions(+), 34 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 803db24104e..4c71570cda5 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -69,18 +69,16 @@ We provide Infrastrucure as Code examples with [AWS Serverless Application Model
 
 ### Required resources
 
-Before getting started, you need to create a persistent storage layer where the idempotency utility can store its state - your lambda functions will need read and write access to it.
+To start, you'll need:
 
-We currently support Amazon DynamoDB and Redis as a storage layer. The following example demonstrates how to create a table in DynamoDB. If you prefer to use Redis, refer go to the section [RedisPersistenceLayer](#redispersistencelayer) section.
+1. A persistent storage layer (DynamoDB or [Redis](#redis-as-persistent-storage-layer-provider))
+2. An AWS Lambda function with [permissions](#iam-permissions) to use your persistent storage layer
 
-**Default table configuration**
+#### DynamoDB table
 
-If you're not [changing the default configuration for the DynamoDB persistence layer](#dynamodbpersistencelayer), this is the expected default configuration:
+!!! tip "You can share a single state table for all functions"
 
-| Configuration      | Value        | Notes                                                                               |
-| ------------------ | ------------ | ----------------------------------------------------------------------------------- |
-| Partition key      | `id`         |                                                                                     |
-| TTL attribute name | `expiration` | This can only be configured after your table is created if you're using AWS Console |
+Unless you're looking to use an [existing table or customize each attribute](#dynamodbpersistencelayer), you only need the following:
 
 | Configuration      | Value        | Notes                                                                                                    |
 | ------------------ | ------------ | -------------------------------------------------------------------------------------------------------- |
@@ -89,7 +87,7 @@ If you're not [changing the default configuration for the DynamoDB persistence l
 
 Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
 
-##### DynamoDB IaC examples
+##### IaC examples
 
 === "AWS Serverless Application Model (SAM) example"
 
@@ -108,38 +106,21 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
     --8<-- "examples/idempotency/templates/terraform.tf"
     ```
 
-???+ warning "Warning: Large responses with DynamoDB persistence layer"
-    When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}.
+##### Limitations
 
-    Larger items cannot be written to DynamoDB and will cause exceptions. If your response exceeds 400kb, consider using Redis as your persistence layer.
+* **DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-as-persistent-storage-layer-provider) as an alternative.
 
-<!-- markdownlint-disable MD013 -->
-???+ info "Info: DynamoDB"
+* **Expect 2 WCU per non-idempotent call**. During the first invocation, we use `PutItem` for locking and `UpdateItem` for completion. Consider reviewing [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"}) to estimate cost.
 
-    During the first invocation with a payload, the Lambda function executes both a `PutItem` and an `UpdateItem` operations to store the data in DynamoDB. If the result returned by your Lambda is less than 1kb, you can expect 2 WCUs per Lambda invocation.
+* **Old boto3 versions can increase costs**. For cost optimization, we use a conditional `PutItem` to always lock a new idempotency record. If locking fails, it means we already have an idempotency record saving us an additional `GetItem` call. However, this is only supported in boto3 `1.26.194` and higher _([June 30th 2023](https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-dynamodb-cost-failed-conditional-writes/){target="_blank"})_.
 
-    On subsequent invocations with the same payload, you can expect just 1 `PutItem` request to DynamoDB.
+#### Redis cluster
 
-We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
+**TODO**: Experiment bringing upfront Redis even at the cost of readability, as setup and usage are disconnected today causing further harm.
 
-In both services and self-hosting Redis, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
+##### Constraints
 
-##### Redis IaC examples
-
-=== "AWS CloudFormation example"
-
-    !!! tip inline end "Prefer AWS Console/CLI?"
-
-        Follow the official tutorials for [Amazon ElastiCache for Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/LambdaRedis.html) or [Amazon MemoryDB for Redis](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/)
-
-    ```yaml hl_lines="5 21"
-    --8<-- "examples/idempotency/templates/cfn_redis_serverless.yaml"
-    ```
-
-    1. Replace the Security Group ID and Subnet ID to match your VPC settings.
-    2. Replace the Security Group ID and Subnet ID to match your VPC settings.
-
-Once setup, you can find a quick start and advanced examples for Redis in [the persistent layers section](RedisCachePersistenceLayer).
+If you'd like to use Redis, please [read here](#redis-as-persistent-storage-layer-provider) on how to setup and access secrets/SSL certs.
 
 <!-- markdownlint-enable MD013 -->
 

From fa2b067b680962a0913cddded3d0f181ed3ceac4 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Sat, 25 May 2024 08:08:30 +1200
Subject: [PATCH 16/41] docs(idempotency): make terminologies crispier

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 4c71570cda5..e0c1692e51f 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -18,11 +18,13 @@ The idempotency utility allows you to retry operations with the same input withi
 
 The property of idempotency means that an operation does not cause additional side effects if it is called more than once with the same input parameters.
 
-**Idempotent operations will return the same result when they are called multiple times with the same parameters**. This makes idempotent operations safe to retry.
+**Idempotency key** is a combination of **(a)** Lambda function name, **(b)** fully qualified name of your function, and **(c)** a hash of the entire payload or part(s) of the payload you specify.
 
-**Idempotency key** is a hash representation of either the entire event or a specific configured subset of the event, and invocation results are **JSON serialized** and stored in your persistence storage layer.
+**Idempotent request** is an operation with the same input previously processed that is not expired in your persistent storage or in-memory cache.
 
-**Idempotency record** is the data representation of an idempotent request saved in your preferred  storage layer. We use it to coordinate whether a request is idempotent, whether it's still valid or expired based on timestamps, etc.
+**Persistence layer** is a storage we use to read, create, expire, and delete idempotency records.
+
+**Idempotency record** is the data representation of an idempotent request in its various status. We use it to coordinate  whether **(a)** a request is idempotent, **(b)** it's not expired, **(c)** JSON response to return, and more.
 
 <center>
 ```mermaid

From 108f45682d4c66dd5ce1d08132fb7f2f2b2e2155 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 27 May 2024 12:23:06 +1200
Subject: [PATCH 17/41] docs(idempotency): line editing before decorators

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index e0c1692e51f..07e927f0c38 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -35,7 +35,7 @@ classDiagram
         status Status
         expiry_timestamp int
         in_progress_expiry_timestamp int
-        response_data Json~str~
+        response_data str~JSON~
         payload_hash str
     }
     class Status {
@@ -60,7 +60,7 @@ When using Amazon DynamoDB as the persistence layer, you will need the following
 
 | IAM Permission                       | Operation                                                                |
 | ------------------------------------ | ------------------------------------------------------------------------ |
-| **`dynamodb:GetItem`**{: .copyMe}    | Retrieve idempotent record                                               |
+| **`dynamodb:GetItem`**{: .copyMe}    | Retrieve idempotent record _(strong consistency)_                        |
 | **`dynamodb:PutItem`**{: .copyMe}    | New idempotent records, replace expired idempotent records               |
 | **`dynamodb:UpdateItem`**{: .copyMe} | Complete idempotency transaction, and/or update idempotent records state |
 | **`dynamodb:DeleteItem`**{: .copyMe} | Delete idempotent records for unsuccessful idempotency transactions      |
@@ -73,7 +73,7 @@ We provide Infrastrucure as Code examples with [AWS Serverless Application Model
 
 To start, you'll need:
 
-1. A persistent storage layer (DynamoDB or [Redis](#redis-as-persistent-storage-layer-provider))
+1. A persistent storage layer - DynamoDB or [Redis](#redis-as-persistent-storage-layer-provider)
 2. An AWS Lambda function with [permissions](#iam-permissions) to use your persistent storage layer
 
 #### DynamoDB table
@@ -82,10 +82,10 @@ To start, you'll need:
 
 Unless you're looking to use an [existing table or customize each attribute](#dynamodbpersistencelayer), you only need the following:
 
-| Configuration      | Value        | Notes                                                                                                    |
-| ------------------ | ------------ | -------------------------------------------------------------------------------------------------------- |
-| Partition key      | `id`         | Primary key looks like: <br> `{lambda_fn_name}.{module_name}.{fn_qualified_name}#{idempotency_key_hash}` |
-| TTL attribute name | `expiration` | This can only be configured after your table is created if you're using AWS Console                      |
+| Configuration      | Value        | Notes                                                                                    |
+| ------------------ | ------------ | ---------------------------------------------------------------------------------------- |
+| Partition key      | `id`         | Format: <br> `{lambda_fn_name}.{module_name}.{fn_qualified_name}#{idempotency_key_hash}` |
+| TTL attribute name | `expiration` | Using AWS Console? this is configurable after table creation                             |
 
 Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
 
@@ -112,7 +112,7 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
 
 * **DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-as-persistent-storage-layer-provider) as an alternative.
 
-* **Expect 2 WCU per non-idempotent call**. During the first invocation, we use `PutItem` for locking and `UpdateItem` for completion. Consider reviewing [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"}) to estimate cost.
+* **Expect 2 WCU per non-idempotent call**. During the first invocation, we use `PutItem` for locking and `UpdateItem` for completion. Consider reviewing [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"} to estimate cost.
 
 * **Old boto3 versions can increase costs**. For cost optimization, we use a conditional `PutItem` to always lock a new idempotency record. If locking fails, it means we already have an idempotency record saving us an additional `GetItem` call. However, this is only supported in boto3 `1.26.194` and higher _([June 30th 2023](https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-dynamodb-cost-failed-conditional-writes/){target="_blank"})_.
 

From dd90766d4485c40371687f1f75e5fc08ca8824ce Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 27 May 2024 14:02:45 +1200
Subject: [PATCH 18/41] docs(idempotency): cleanup timeout section

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 28 ++++++++--------------------
 1 file changed, 8 insertions(+), 20 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 07e927f0c38..b6fd1278a04 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -319,35 +319,23 @@ You can change this expiration window with the **`expires_after_seconds`** param
 
 ### Lambda timeouts
 
-???+ note
-    This is automatically done when you decorate your Lambda handler with [@idempotent decorator](#idempotent-decorator).
-
-To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/){target="_blank"},
-Powertools for AWS Lambda (Python) calculates and includes the remaining invocation available time as part of the idempotency record.
+By default, we protect against [concurrent executions](#handling-concurrent-executions-with-the-same-payload) with the same payload using a locking mechanism. However, if your Lambda function times out before completing the first invocation it will only accept the same request when the [idempotency record expire](#expiring-idempotency-records).
 
 To prevent extended failures, use **`register_lambda_context`** function from your idempotency config to calculate and include the remaining invocation time in your idempotency record.
 
-```python title="working_with_lambda_timeout.py" hl_lines="14 23"
---8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
-```
+=== "Registering the Lambda context"
+
+    > This is unnecessary for [`@idempotent` decorator](#idempotent-decorator), as it captures the Lambda context from your handler function.
+
+    ```python title="working_with_lambda_timeout.py" hl_lines="11 20"
+    --8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
+    ```
 
 ???+ example "Mechanics"
     If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will run the invocation again as if it was in the `EXPIRED` state.
 
     This means that if an invocation expired during execution, it will be quickly executed again on the next retry.
 
-???+ important
-    If you are only using the [@idempotent_function decorator](#idempotent_function-decorator) to guard isolated parts of your code,
-    you must use `register_lambda_context` available in the [idempotency config object](#customizing-the-default-behavior) to benefit from this protection.
-
-Here is an example on how you register the Lambda context in your handler:
-
-=== "Registering the Lambda context"
-
-    ```python hl_lines="11 20"
-    --8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
-    ```
-
 ### Handling exceptions
 
 There are two failure modes that can cause new invocations to execute your code again despite having the same payload:

From cf395471a14edf92a39d6b9c1afe75f2a7828a16 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 28 May 2024 11:38:09 +1200
Subject: [PATCH 19/41] docs(idempotency): use cards for required resources

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index b6fd1278a04..ffc9a804cd0 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -73,8 +73,24 @@ We provide Infrastrucure as Code examples with [AWS Serverless Application Model
 
 To start, you'll need:
 
-1. A persistent storage layer - DynamoDB or [Redis](#redis-as-persistent-storage-layer-provider)
-2. An AWS Lambda function with [permissions](#iam-permissions) to use your persistent storage layer
+<!-- markdownlint-disable MD030 -->
+
+<div class="grid cards" markdown>
+*   :octicons-database-16:{ .lg .middle } __Persistent storage__
+
+    ---
+
+    [Amazon DynamoDB](#dynamodb-table) or [Redis](#redis-as-persistent-storage-layer-provider)
+
+*   :simple-awslambda:{ .lg .middle } **AWS Lambda function**
+
+    ---
+
+    With permissions to use your persistent storage
+
+</div>
+
+<!-- markdownlint-enable MD030 -->
 
 #### DynamoDB table
 

From 6776d79cdc3f4f11796f2a73e6100d8a4ee31ea2 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 28 May 2024 11:41:47 +1200
Subject: [PATCH 20/41] docs(idempotency): note to skip timeout section when
 using handler decorator

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index ffc9a804cd0..cd995a3a6f8 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -335,14 +335,14 @@ You can change this expiration window with the **`expires_after_seconds`** param
 
 ### Lambda timeouts
 
+!!! note "You can skip this section if you are using the [`@idempotent` decorator](#idempotent-decorator)"
+
 By default, we protect against [concurrent executions](#handling-concurrent-executions-with-the-same-payload) with the same payload using a locking mechanism. However, if your Lambda function times out before completing the first invocation it will only accept the same request when the [idempotency record expire](#expiring-idempotency-records).
 
 To prevent extended failures, use **`register_lambda_context`** function from your idempotency config to calculate and include the remaining invocation time in your idempotency record.
 
 === "Registering the Lambda context"
 
-    > This is unnecessary for [`@idempotent` decorator](#idempotent-decorator), as it captures the Lambda context from your handler function.
-
     ```python title="working_with_lambda_timeout.py" hl_lines="11 20"
     --8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
     ```

From 55a9fc67cc9a47289b68bedd9ba36003fcc37d84 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 28 May 2024 11:43:21 +1200
Subject: [PATCH 21/41] docs: remove tabbed content for single timeout snippet

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index cd995a3a6f8..8eef02de5c3 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -341,11 +341,9 @@ By default, we protect against [concurrent executions](#handling-concurrent-exec
 
 To prevent extended failures, use **`register_lambda_context`** function from your idempotency config to calculate and include the remaining invocation time in your idempotency record.
 
-=== "Registering the Lambda context"
-
-    ```python title="working_with_lambda_timeout.py" hl_lines="11 20"
-    --8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
-    ```
+```python title="working_with_lambda_timeout.py" hl_lines="11 20"
+--8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
+```
 
 ???+ example "Mechanics"
     If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will run the invocation again as if it was in the `EXPIRED` state.

From d900b12fb76e8c17e7dad03e77539253cd0aa2f6 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 28 May 2024 13:09:08 +1200
Subject: [PATCH 22/41] docs: cleanup persistence layers attrs, snippet titles
 etc

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 90 +++++++++++++++++++++++++++++++----
 1 file changed, 80 insertions(+), 10 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 8eef02de5c3..356dc15a8ba 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -369,17 +369,13 @@ If an exception is handled or raised **outside** your decorated function, then i
 
 This persistence layer is built-in, allowing you to use an existing DynamoDB table or create a new one dedicated to idempotency state (recommended).
 
-```python title="customize_persistence_layer.py" hl_lines="10-18"
+```python title="customize_persistence_layer.py" hl_lines="7-15"
 --8<-- "examples/idempotency/src/customize_persistence_layer.py"
 ```
 
-    ```python hl_lines="7-15"
-    --8<-- "examples/idempotency/src/customize_persistence_layer.py"
-    ```
-
-##### DynamoDB defaults
+##### DynamoDB attributes
 
-When using DynamoDB as the persistence layer, you can customize the attribute names by passing the following parameters during the initialization of the persistence layer:
+You can customize the attribute names during initialization:
 
 | Parameter                   | Required           | Default                              | Description                                                                                              |
 | --------------------------- | ------------------ | ------------------------------------ | -------------------------------------------------------------------------------------------------------- |
@@ -397,20 +393,67 @@ When using DynamoDB as the persistence layer, you can customize the attribute na
 
 !!! info "We recommend Redis version 7 or higher for optimal performance."
 
-For a quick start, initialize `RedisCachePersistenceLayer` and pass your cluster host endpoint along with the port to connect to.
+For simple setups, initialize `RedisCachePersistenceLayer` with your cluster endpoint and port to connect.
 
 For security, we enforce SSL connections by default; to disable it, set `ssl=False`.
 
 === "Redis quick start"
-    ```python title="getting_started_with_idempotency_redis_config.py" hl_lines="8-10 14 27"
+    ```python title="getting_started_with_idempotency_redis_config.py" hl_lines="7-9 12 26"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_config.py"
     ```
 
 === "Using an existing Redis client"
-    ```python title="getting_started_with_idempotency_redis_client.py" hl_lines="5 10-11 16 24 38"
+    ```python title="getting_started_with_idempotency_redis_client.py" hl_lines="4 9-11 14 22 36"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_client.py"
     ```
 
+=== "Sample event"
+
+    ```json title="getting_started_with_idempotency_payload.json"
+    --8<-- "examples/idempotency/src/getting_started_with_idempotency_payload.json"
+    ```
+
+##### Redis SSL connections
+
+We recommend using AWS Secrets Manager to store and rotate certificates safely, and the [Parameters feature](./parameters.md){target="_blank"} to fetch and cache optimally.
+
+For advanced configurations, we recommend using an existing Redis client for optimal compatibility like SSL certificates and timeout.
+
+=== "Advanced configuration using AWS Secrets"
+    ```python title="using_redis_client_with_aws_secrets.py" hl_lines="9-11 13 15 25"
+    --8<-- "examples/idempotency/src/using_redis_client_with_aws_secrets.py"
+    ```
+
+    1. JSON stored:
+    ```json
+    {
+    "REDIS_ENDPOINT": "127.0.0.1",
+    "REDIS_PORT": "6379",
+    "REDIS_PASSWORD": "redis-secret"
+    }
+    ```
+
+=== "Advanced configuration with local certificates"
+    ```python title="using_redis_client_with_local_certs.py" hl_lines="14 25-27"
+    --8<-- "examples/idempotency/src/using_redis_client_with_local_certs.py"
+    ```
+
+    1. JSON stored:
+    ```json
+    {
+    "REDIS_ENDPOINT": "127.0.0.1",
+    "REDIS_PORT": "6379",
+    "REDIS_PASSWORD": "redis-secret"
+    }
+    ```
+    2. redis_user.crt file stored in the "certs" directory of your Lambda function
+    3. redis_user_private.key file stored in the "certs" directory of your Lambda function
+    4. redis_ca.pem file stored in the "certs" directory of your Lambda function
+
+##### Redis attributes
+
+You can customize the attribute names during initialization:
+
 | Parameter                   | Required | Default                  | Description                                                                                   |
 | --------------------------- | -------- | ------------------------ | --------------------------------------------------------------------------------------------- |
 | **in_progress_expiry_attr** |          | `in_progress_expiration` | Unix timestamp of when record expires while in progress (in case of the invocation times out) |
@@ -418,6 +461,33 @@ For security, we enforce SSL connections by default; to disable it, set `ssl=Fal
 | **data_attr**               |          | `data`                   | Stores results of successfully executed Lambda handlers                                       |
 | **validation_key_attr**     |          | `validation`             | Hashed representation of the parts of the event used for validation                           |
 
+```python title="customize_persistence_layer_redis.py" hl_lines="9-16"
+--8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
+```
+
+### Common use cases
+
+#### Batch integration
+
+You can can easily integrate with [Batch](batch.md){target="_blank"} with the [idempotent_function decorator](#idempotent_function-decorator) to handle idempotency per message/record in a given batch.
+
+???+ "Choosing an unique batch record attribute"
+    In this example, we choose `messageId` as our idempotency key since we know it'll be unique.
+
+    Depending on your use case, it might be more accurate [to choose another field](#choosing-a-payload-subset-for-idempotency) your producer intentionally set to define uniqueness.
+
+=== "Integration with Batch Processor"
+
+    ```python title="integrate_idempotency_with_batch_processor.py" hl_lines="3 16 19 25 27"
+    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor.py"
+    ```
+
+=== "Sample event"
+
+    ```json title="integrate_idempotency_with_batch_processor_payload.json" hl_lines="4"
+    --8<-- "examples/idempotency/src/integrate_idempotency_with_batch_processor_payload.json"
+    ```
+
 ### Idempotency request flow
 
 The following sequence diagrams explain how the Idempotency feature behaves under different scenarios.

From 6c02031ed0eea7150c46d86bf1b7f647d7e30336 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 28 May 2024 13:09:49 +1200
Subject: [PATCH 23/41] docs: typo in batch integration

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 356dc15a8ba..e53a3755b1a 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -469,7 +469,7 @@ You can customize the attribute names during initialization:
 
 #### Batch integration
 
-You can can easily integrate with [Batch](batch.md){target="_blank"} with the [idempotent_function decorator](#idempotent_function-decorator) to handle idempotency per message/record in a given batch.
+You can can easily integrate with [Batch](batch.md){target="_blank"} using the [idempotent_function decorator](#idempotent_function-decorator) to handle idempotency per message/record in a given batch.
 
 ???+ "Choosing an unique batch record attribute"
     In this example, we choose `messageId` as our idempotency key since we know it'll be unique.

From 14d4f4d3b4d3a4e6e70becd11105e1ad9ad2435c Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 28 May 2024 13:10:21 +1200
Subject: [PATCH 24/41] docs: rename batch integration to actual use case name

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index e53a3755b1a..5831d699a4a 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -467,7 +467,7 @@ You can customize the attribute names during initialization:
 
 ### Common use cases
 
-#### Batch integration
+#### Batch processing
 
 You can can easily integrate with [Batch](batch.md){target="_blank"} using the [idempotent_function decorator](#idempotent_function-decorator) to handle idempotency per message/record in a given batch.
 

From bb23ce9ec9b2bf295c6c8dc9dd570e552b57e802 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Thu, 6 Jun 2024 11:01:34 +0200
Subject: [PATCH 25/41] docs(idempotency): cleanup default behavior section

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 5831d699a4a..3a212987321 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -764,15 +764,15 @@ graph TD;
 
 ### Customizing the default behavior
 
-Idempotent decorator can be further configured with **`IdempotencyConfig`** as seen in the previous example. These are the available options for further configuration
+You can override and further extend idempotency behavior via **`IdempotencyConfig`** with the following options:
 
 | Parameter                       | Default | Description                                                                                                                                                                                                                                |
 | ------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
 | **event_key_jmespath**          | `""`    | JMESPath expression to extract the idempotency key from the event record using [built-in functions](./jmespath_functions.md#built-in-jmespath-functions){target="_blank"}                                                                  |
-| **payload_validation_jmespath** | `""`    | JMESPath expression to validate whether certain parameters have changed in the event while the event payload                                                                                                                               |
+| **payload_validation_jmespath** | `""`    | JMESPath expression to validate whether certain parameters have changed in the event while the event payload _e.g., payload tampering._                                                                                                    |
 | **raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request                                                                                                                                                                             |
-| **expires_after_seconds**       | 3600    | The number of seconds to wait before a record is expired                                                                                                                                                                                   |
-| **use_local_cache**             | `False` | Whether to locally cache idempotency results                                                                                                                                                                                               |
+| **expires_after_seconds**       | 3600    | The number of seconds to wait before a record is expired, allowing a new transaction with the same idempotency key                                                                                                                         |
+| **use_local_cache**             | `False` | Whether to cache idempotency results in-memory to save on persistence storage latency and costs                                                                                                                                            |
 | **local_cache_max_items**       | 256     | Max number of items to store in local cache                                                                                                                                                                                                |
 | **hash_function**               | `md5`   | Function to use for calculating hashes, as provided by [hashlib](https://docs.python.org/3/library/hashlib.html){target="_blank" rel="nofollow"} in the standard library.                                                                  |
 | **response_hook**               | `None`  | Function to use for processing the stored Idempotent response. This function hook is called when an existing idempotent response is found. See [Manipulating The Idempotent Response](idempotency.md#manipulating-the-idempotent-response) |

From f795c2ec7c5d3ce56b9f5153d7e6ff007e4352da Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Mon, 24 Jun 2024 18:31:54 +0200
Subject: [PATCH 26/41] docs: move bold to draw attention to whole event as
 idempotency key

Signed-off-by: heitorlessa <lessa@amazon.co.uk>
---
 docs/utilities/idempotency.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 3a212987321..cf6d5e3ace2 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -278,13 +278,13 @@ By default, caching is disabled since we don't know how big your response could
 
 Use [`IdempotencyConfig`](#customizing-the-default-behavior)'s **`event_key_jmespath`** parameter to select one or more payload parts as your idempotency key.
 
-> **Payment scenario**
+> **Example scenario**
 
 In this example, we have a Lambda handler that creates a payment for a user subscribing to a product. We want to ensure that we don't accidentally charge our customer by subscribing them more than once.
 
 Imagine the function runs successfully, but the client never receives the response due to a connection issue. It is safe to immediately retry in this instance, as the idempotent decorator will return a previously saved response.
 
-**We want** to use `user_id` and `product_id` fields as our idempotency key. If we were to treat the entire request as our idempotency key, a simple HTTP header change would cause our function to run again.
+We want to use `user_id` and `product_id` fields as our idempotency key. **If we were** to treat the entire request as our idempotency key, a simple HTTP header change would cause our function to run again.
 
 ???+ tip "Deserializing JSON strings in payloads for increased accuracy."
     The payload extracted by the `event_key_jmespath` is treated as a string by default.

From aa25c420272edd46a315ce3c571aacbbc5474089 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 25 Jun 2024 09:17:21 +0200
Subject: [PATCH 27/41] docs: lead with parameter name over config name

---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index cf6d5e3ace2..d4d999c2b68 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -276,7 +276,7 @@ By default, caching is disabled since we don't know how big your response could
 ???+ tip "Tip: Dealing with always changing payloads"
     When dealing with a more elaborate payload, where parts of the payload always change, you should use **`event_key_jmespath`** parameter.
 
-Use [`IdempotencyConfig`](#customizing-the-default-behavior)'s **`event_key_jmespath`** parameter to select one or more payload parts as your idempotency key.
+Use **`event_key_jmespath`** parameter in [`IdempotencyConfig`](#customizing-the-default-behavior) to select one or more payload parts as your idempotency key.
 
 > **Example scenario**
 

From 8837b1dd753ac4fa9b373d5d83cbb0d49ff1bcb6 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 25 Jun 2024 16:31:46 +0200
Subject: [PATCH 28/41] docs: moved composite key under DDB section

---
 docs/utilities/idempotency.md | 56 +++++++++++++++++------------------
 1 file changed, 28 insertions(+), 28 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index d4d999c2b68..d289c1dd241 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -373,6 +373,34 @@ This persistence layer is built-in, allowing you to use an existing DynamoDB tab
 --8<-- "examples/idempotency/src/customize_persistence_layer.py"
 ```
 
+##### Using a composite primary key
+
+Use `sort_key_attr` parameter when your table is configured with a composite primary key _(hash+range key)_.
+
+When enabled, we will save the idempotency key in the sort key instead. By default, the primary key will now be set to `idempotency#{LAMBDA_FUNCTION_NAME}`.
+
+You can optionally set a static value for the partition key using the `static_pk_value` parameter.
+
+=== "Reusing a DynamoDB table that uses a composite primary key"
+
+    ```python hl_lines="10"
+    --8<-- "examples/idempotency/src/working_with_composite_key.py"
+    ```
+
+=== "Sample Event"
+
+    ```json
+    --8<-- "examples/idempotency/src/working_with_composite_key_payload.json"
+    ```
+
+??? note "Click to expand and learn how table items would look like"
+
+    | id                           | sort_key                         | expiration | status      | data                                      |
+    | ---------------------------- | -------------------------------- | ---------- | ----------- | ----------------------------------------- |
+    | idempotency#MyLambdaFunction | 1e956ef7da78d0cb890be999aecc0c9e | 1636549553 | COMPLETED   | {"user_id": 12391, "message": "success"}  |
+    | idempotency#MyLambdaFunction | 2b2cdb5f86361e97b4383087c1ffdf27 | 1636549571 | COMPLETED   | {"user_id": 527212, "message": "success"} |
+    | idempotency#MyLambdaFunction | f091d2527ad1c78f05d54cc3f363be80 | 1636549585 | IN_PROGRESS |                                           |
+
 ##### DynamoDB attributes
 
 You can customize the attribute names during initialization:
@@ -871,34 +899,6 @@ The **`boto_config`** and **`boto3_session`** parameters enable you to pass in a
     --8<-- "examples/idempotency/src/working_with_custom_config_payload.json"
     ```
 
-### Using a DynamoDB table with a composite primary key
-
-When using a composite primary key table (hash+range key), use `sort_key_attr` parameter when initializing your persistence layer.
-
-With this setting, we will save the idempotency key in the sort key instead of the primary key. By default, the primary key will now be set to `idempotency#{LAMBDA_FUNCTION_NAME}`.
-
-You can optionally set a static value for the partition key using the `static_pk_value` parameter.
-
-=== "Reusing a DynamoDB table that uses a composite primary key"
-
-    ```python hl_lines="10"
-    --8<-- "examples/idempotency/src/working_with_composite_key.py"
-    ```
-
-=== "Sample Event"
-
-    ```json
-    --8<-- "examples/idempotency/src/working_with_composite_key_payload.json"
-    ```
-
-The example function above would cause data to be stored in DynamoDB like this:
-
-| id                           | sort_key                         | expiration | status      | data                                      |
-| ---------------------------- | -------------------------------- | ---------- | ----------- | ----------------------------------------- |
-| idempotency#MyLambdaFunction | 1e956ef7da78d0cb890be999aecc0c9e | 1636549553 | COMPLETED   | {"user_id": 12391, "message": "success"}  |
-| idempotency#MyLambdaFunction | 2b2cdb5f86361e97b4383087c1ffdf27 | 1636549571 | COMPLETED   | {"user_id": 527212, "message": "success"} |
-| idempotency#MyLambdaFunction | f091d2527ad1c78f05d54cc3f363be80 | 1636549585 | IN_PROGRESS |                                           |
-
 ### Bring your own persistent store
 
 This utility provides an abstract base class (ABC), so that you can implement your choice of persistent storage layer.

From 0145c8d4633282a6c54d15f075058ad45b74b175 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 25 Jun 2024 17:32:01 +0200
Subject: [PATCH 29/41] docs: cut unnecessary anchor name

---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index d289c1dd241..120bd2a431b 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -271,7 +271,7 @@ By default, caching is disabled since we don't know how big your response could
     --8<-- "examples/idempotency/src/working_with_local_cache_payload.json"
     ```
 
-### Choosing a payload subset for idempotency
+### Choosing a payload subset
 
 ???+ tip "Tip: Dealing with always changing payloads"
     When dealing with a more elaborate payload, where parts of the payload always change, you should use **`event_key_jmespath`** parameter.

From ea1792e0df606840fde65369bdad15587740e9cc Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 25 Jun 2024 18:03:28 +0200
Subject: [PATCH 30/41] docs: fix broken links after sections renaming

---
 docs/utilities/idempotency.md | 37 +++++++++++++++++++++++------------
 1 file changed, 24 insertions(+), 13 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 120bd2a431b..4843aea1f31 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -52,7 +52,7 @@ classDiagram
 
 ## Getting started
 
-We use Amazon DynamoDB as the default persistence layer in the documentation. If you prefer Redis, you can learn more from [this section](#redis-as-persistent-storage-layer-provider).
+We use Amazon DynamoDB as the default persistence layer in the documentation. If you prefer Redis, you can learn more from [this section](#redis-cluster).
 
 ### IAM Permissions
 
@@ -80,7 +80,7 @@ To start, you'll need:
 
     ---
 
-    [Amazon DynamoDB](#dynamodb-table) or [Redis](#redis-as-persistent-storage-layer-provider)
+    [Amazon DynamoDB](#dynamodb-table) or [Redis](#redis-cluster)
 
 *   :simple-awslambda:{ .lg .middle } **AWS Lambda function**
 
@@ -126,7 +126,7 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
 
 ##### Limitations
 
-* **DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-as-persistent-storage-layer-provider) as an alternative.
+* **DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-cluster) as an alternative.
 
 * **Expect 2 WCU per non-idempotent call**. During the first invocation, we use `PutItem` for locking and `UpdateItem` for completion. Consider reviewing [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"} to estimate cost.
 
@@ -138,7 +138,22 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
 
 ##### Constraints
 
-If you'd like to use Redis, please [read here](#redis-as-persistent-storage-layer-provider) on how to setup and access secrets/SSL certs.
+##### Redis IaC examples
+
+=== "AWS CloudFormation example"
+
+    !!! tip inline end "Prefer AWS Console/CLI?"
+
+        Follow the official tutorials for [Amazon ElastiCache for Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/LambdaRedis.html) or [Amazon MemoryDB for Redis](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/)
+
+    ```yaml hl_lines="5 21"
+    --8<-- "examples/idempotency/templates/cfn_redis_serverless.yaml"
+    ```
+
+    1. Replace the Security Group ID and Subnet ID to match your VPC settings.
+    2. Replace the Security Group ID and Subnet ID to match your VPC settings.
+
+Once setup, you can find a quick start and advanced examples for Redis in [the persistent layers section](#redispersistencelayer).
 
 <!-- markdownlint-enable MD013 -->
 
@@ -146,11 +161,11 @@ If you'd like to use Redis, please [read here](#redis-as-persistent-storage-laye
 
 For simple use cases, you can use the `idempotent` decorator on your Lambda handler function.
 
-It will treat the entire event as an idempotency key. That is, the same event will return the previously stored result within a [configurable time window](#expiring-idempotency-records) _(1 hour, by default)_.
+It will treat the entire event as an idempotency key. That is, the same event will return the previously stored result within a [configurable time window](#adjusting-expiration-window) _(1 hour, by default)_.
 
 === "Idempotent decorator"
 
-    !!! tip "You can also choose [one or more fields](#choosing-a-payload-subset-for-idempotency) as an idempotency key."
+    !!! tip "You can also choose [one or more fields](#choosing-a-payload-subset) as an idempotency key."
 
     ```python title="getting_started_with_idempotency.py" hl_lines="5-8 12 25"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency.py"
@@ -337,7 +352,7 @@ You can change this expiration window with the **`expires_after_seconds`** param
 
 !!! note "You can skip this section if you are using the [`@idempotent` decorator](#idempotent-decorator)"
 
-By default, we protect against [concurrent executions](#handling-concurrent-executions-with-the-same-payload) with the same payload using a locking mechanism. However, if your Lambda function times out before completing the first invocation it will only accept the same request when the [idempotency record expire](#expiring-idempotency-records).
+By default, we protect against [concurrent executions](#handling-concurrent-executions-with-the-same-payload) with the same payload using a locking mechanism. However, if your Lambda function times out before completing the first invocation it will only accept the same request when the [idempotency record expire](#adjusting-expiration-window).
 
 To prevent extended failures, use **`register_lambda_context`** function from your idempotency config to calculate and include the remaining invocation time in your idempotency record.
 
@@ -502,7 +517,7 @@ You can can easily integrate with [Batch](batch.md){target="_blank"} using the [
 ???+ "Choosing an unique batch record attribute"
     In this example, we choose `messageId` as our idempotency key since we know it'll be unique.
 
-    Depending on your use case, it might be more accurate [to choose another field](#choosing-a-payload-subset-for-idempotency) your producer intentionally set to define uniqueness.
+    Depending on your use case, it might be more accurate [to choose another field](#choosing-a-payload-subset) your producer intentionally set to define uniqueness.
 
 === "Integration with Batch Processor"
 
@@ -951,10 +966,6 @@ When using response hooks to manipulate returned data from idempotent operations
 
 ## Compatibility with other utilities
 
-### Batch
-
-See [Batch integration](#batch-integration) above.
-
 ### JSON Schema Validation
 
 The idempotency utility can be used with the `validator` decorator. Ensure that idempotency is the innermost decorator.
@@ -965,7 +976,7 @@ The idempotency utility can be used with the `validator` decorator. Ensure that
 
 	Make sure to account for this behavior, if you set the `event_key_jmespath`.
 
-=== "Using Idempotency with JSONSchema Validation utility"
+=== "Using Idempotency with validation utility"
 
     ```python hl_lines="16"
     --8<-- "examples/idempotency/src/integrate_idempotency_with_validator.py"

From 97211e65d4d3440414a89db86499ffcd4291c022 Mon Sep 17 00:00:00 2001
From: Leandro Damascena <lcdama@amazon.pt>
Date: Thu, 27 Jun 2024 11:02:19 +0100
Subject: [PATCH 31/41] Making mypy happy

---
 examples/idempotency/src/customize_persistence_layer.py      | 2 +-
 .../idempotency/src/customize_persistence_layer_redis.py     | 2 +-
 examples/idempotency/src/getting_started_with_idempotency.py | 2 +-
 .../src/getting_started_with_idempotency_redis_client.py     | 2 +-
 .../src/getting_started_with_idempotency_redis_config.py     | 2 +-
 .../src/integrate_idempotency_with_batch_processor.py        | 5 +++--
 .../idempotency/src/integrate_idempotency_with_validator.py  | 2 +-
 examples/idempotency/src/working_with_composite_key.py       | 2 +-
 examples/idempotency/src/working_with_custom_config.py       | 2 +-
 examples/idempotency/src/working_with_custom_session.py      | 2 +-
 .../src/working_with_dataclass_deduced_output_serializer.py  | 2 +-
 .../working_with_dataclass_explicitly_output_serializer.py   | 2 +-
 examples/idempotency/src/working_with_exceptions.py          | 2 +-
 .../idempotency/src/working_with_idempotency_key_required.py | 2 +-
 ...king_with_idempotent_function_custom_output_serializer.py | 2 +-
 .../src/working_with_idempotent_function_dataclass.py        | 2 +-
 .../src/working_with_idempotent_function_pydantic.py         | 2 +-
 examples/idempotency/src/working_with_lambda_timeout.py      | 2 +-
 examples/idempotency/src/working_with_local_cache.py         | 2 +-
 examples/idempotency/src/working_with_payload_subset.py      | 2 +-
 .../src/working_with_pydantic_deduced_output_serializer.py   | 2 +-
 .../working_with_pydantic_explicitly_output_serializer.py    | 2 +-
 examples/idempotency/src/working_with_record_expiration.py   | 2 +-
 examples/idempotency/src/working_with_response_hook.py       | 2 +-
 examples/idempotency/src/working_with_validation_payload.py  | 2 +-
 25 files changed, 27 insertions(+), 26 deletions(-)

diff --git a/examples/idempotency/src/customize_persistence_layer.py b/examples/idempotency/src/customize_persistence_layer.py
index 231ea95e2c0..a4e9aa6993e 100644
--- a/examples/idempotency/src/customize_persistence_layer.py
+++ b/examples/idempotency/src/customize_persistence_layer.py
@@ -6,7 +6,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(
     table_name=table,
     key_attr="idempotency_key",
diff --git a/examples/idempotency/src/customize_persistence_layer_redis.py b/examples/idempotency/src/customize_persistence_layer_redis.py
index 566efc59350..40aef433396 100644
--- a/examples/idempotency/src/customize_persistence_layer_redis.py
+++ b/examples/idempotency/src/customize_persistence_layer_redis.py
@@ -8,7 +8,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
+redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT", "localhost")
 persistence_layer = RedisCachePersistenceLayer(
     host=redis_endpoint,
     port=6379,
diff --git a/examples/idempotency/src/getting_started_with_idempotency.py b/examples/idempotency/src/getting_started_with_idempotency.py
index e94f16d7e0c..b17426c06f2 100644
--- a/examples/idempotency/src/getting_started_with_idempotency.py
+++ b/examples/idempotency/src/getting_started_with_idempotency.py
@@ -8,7 +8,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 
diff --git a/examples/idempotency/src/getting_started_with_idempotency_redis_client.py b/examples/idempotency/src/getting_started_with_idempotency_redis_client.py
index 38c34951892..24dfe1be117 100644
--- a/examples/idempotency/src/getting_started_with_idempotency_redis_client.py
+++ b/examples/idempotency/src/getting_started_with_idempotency_redis_client.py
@@ -12,7 +12,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
+redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT", "localhost")
 client = Redis(
     host=redis_endpoint,
     port=6379,
diff --git a/examples/idempotency/src/getting_started_with_idempotency_redis_config.py b/examples/idempotency/src/getting_started_with_idempotency_redis_config.py
index b4451be438f..f3917042b28 100644
--- a/examples/idempotency/src/getting_started_with_idempotency_redis_config.py
+++ b/examples/idempotency/src/getting_started_with_idempotency_redis_config.py
@@ -10,7 +10,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
+redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT", "localhost")
 persistence_layer = RedisCachePersistenceLayer(host=redis_endpoint, port=6379)
 
 
diff --git a/examples/idempotency/src/integrate_idempotency_with_batch_processor.py b/examples/idempotency/src/integrate_idempotency_with_batch_processor.py
index abde83c44e4..120c8f12da9 100644
--- a/examples/idempotency/src/integrate_idempotency_with_batch_processor.py
+++ b/examples/idempotency/src/integrate_idempotency_with_batch_processor.py
@@ -1,4 +1,5 @@
 import os
+from typing import Any, Dict
 
 from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response
 from aws_lambda_powertools.utilities.data_classes.sqs_event import SQSRecord
@@ -11,7 +12,7 @@
 
 processor = BatchProcessor(event_type=EventType.SQS)
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="messageId")
 
@@ -21,7 +22,7 @@ def record_handler(record: SQSRecord):
     return {"message": record.body}
 
 
-def lambda_handler(event: SQSRecord, context: LambdaContext):
+def lambda_handler(event: Dict[str, Any], context: LambdaContext):
     config.register_lambda_context(context)  # see Lambda timeouts section
 
     return process_partial_response(
diff --git a/examples/idempotency/src/integrate_idempotency_with_validator.py b/examples/idempotency/src/integrate_idempotency_with_validator.py
index e69897205a4..675dbd249a9 100644
--- a/examples/idempotency/src/integrate_idempotency_with_validator.py
+++ b/examples/idempotency/src/integrate_idempotency_with_validator.py
@@ -8,7 +8,7 @@
 from aws_lambda_powertools.utilities.typing import LambdaContext
 from aws_lambda_powertools.utilities.validation import envelopes, validator
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 config = IdempotencyConfig(event_key_jmespath='["message", "username"]')
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
diff --git a/examples/idempotency/src/working_with_composite_key.py b/examples/idempotency/src/working_with_composite_key.py
index 33345706078..92bf1e6ec9a 100644
--- a/examples/idempotency/src/working_with_composite_key.py
+++ b/examples/idempotency/src/working_with_composite_key.py
@@ -6,7 +6,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table, sort_key_attr="sort_key")
 
 
diff --git a/examples/idempotency/src/working_with_custom_config.py b/examples/idempotency/src/working_with_custom_config.py
index ddbfc08cd7e..3d0f464a1dd 100644
--- a/examples/idempotency/src/working_with_custom_config.py
+++ b/examples/idempotency/src/working_with_custom_config.py
@@ -12,7 +12,7 @@
 # See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore-config
 boto_config = Config()
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table, boto_config=boto_config)
 
 config = IdempotencyConfig(event_key_jmespath="body")
diff --git a/examples/idempotency/src/working_with_custom_session.py b/examples/idempotency/src/working_with_custom_session.py
index ef324dab721..af414c829de 100644
--- a/examples/idempotency/src/working_with_custom_session.py
+++ b/examples/idempotency/src/working_with_custom_session.py
@@ -12,7 +12,7 @@
 # See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#module-boto3.session
 boto3_session = boto3.session.Session()
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table, boto3_session=boto3_session)
 
 config = IdempotencyConfig(event_key_jmespath="body")
diff --git a/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py b/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py
index bb4592768e7..e6f74cb8f9a 100644
--- a/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py
+++ b/examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py
@@ -9,7 +9,7 @@
 from aws_lambda_powertools.utilities.idempotency.serialization.dataclass import DataclassSerializer
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
diff --git a/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py b/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py
index 2f6adea057c..05ea956d696 100644
--- a/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py
+++ b/examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py
@@ -9,7 +9,7 @@
 from aws_lambda_powertools.utilities.idempotency.serialization.dataclass import DataclassSerializer
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
diff --git a/examples/idempotency/src/working_with_exceptions.py b/examples/idempotency/src/working_with_exceptions.py
index 19069fbbc4f..b416a61b60a 100644
--- a/examples/idempotency/src/working_with_exceptions.py
+++ b/examples/idempotency/src/working_with_exceptions.py
@@ -10,7 +10,7 @@
 from aws_lambda_powertools.utilities.idempotency.exceptions import IdempotencyPersistenceLayerError
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 config = IdempotencyConfig()
diff --git a/examples/idempotency/src/working_with_idempotency_key_required.py b/examples/idempotency/src/working_with_idempotency_key_required.py
index dea3965f9b7..465a7d47e0a 100644
--- a/examples/idempotency/src/working_with_idempotency_key_required.py
+++ b/examples/idempotency/src/working_with_idempotency_key_required.py
@@ -7,7 +7,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath='["user.uid", "order_id"]',
diff --git a/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py b/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py
index 74d22feb83f..5d6c1ea3b99 100644
--- a/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py
+++ b/examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py
@@ -9,7 +9,7 @@
 from aws_lambda_powertools.utilities.idempotency.serialization.custom_dict import CustomDictSerializer
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
diff --git a/examples/idempotency/src/working_with_idempotent_function_dataclass.py b/examples/idempotency/src/working_with_idempotent_function_dataclass.py
index b4cc4ab0b53..3a4e347b22a 100644
--- a/examples/idempotency/src/working_with_idempotent_function_dataclass.py
+++ b/examples/idempotency/src/working_with_idempotent_function_dataclass.py
@@ -8,7 +8,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
diff --git a/examples/idempotency/src/working_with_idempotent_function_pydantic.py b/examples/idempotency/src/working_with_idempotent_function_pydantic.py
index 2092fa1b45b..45b57499a29 100644
--- a/examples/idempotency/src/working_with_idempotent_function_pydantic.py
+++ b/examples/idempotency/src/working_with_idempotent_function_pydantic.py
@@ -8,7 +8,7 @@
 from aws_lambda_powertools.utilities.parser import BaseModel
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
diff --git a/examples/idempotency/src/working_with_lambda_timeout.py b/examples/idempotency/src/working_with_lambda_timeout.py
index 9a4fffb526b..eac423607ad 100644
--- a/examples/idempotency/src/working_with_lambda_timeout.py
+++ b/examples/idempotency/src/working_with_lambda_timeout.py
@@ -8,7 +8,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 config = IdempotencyConfig()
diff --git a/examples/idempotency/src/working_with_local_cache.py b/examples/idempotency/src/working_with_local_cache.py
index ce6b55dc981..571098715f7 100644
--- a/examples/idempotency/src/working_with_local_cache.py
+++ b/examples/idempotency/src/working_with_local_cache.py
@@ -7,7 +7,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath="powertools_json(body)",
diff --git a/examples/idempotency/src/working_with_payload_subset.py b/examples/idempotency/src/working_with_payload_subset.py
index 73030630ce6..c16508cbbb2 100644
--- a/examples/idempotency/src/working_with_payload_subset.py
+++ b/examples/idempotency/src/working_with_payload_subset.py
@@ -10,7 +10,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 
 # Deserialize JSON string under the "body" key
diff --git a/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py b/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py
index 5ea5b94c2b2..b904a5ad670 100644
--- a/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py
+++ b/examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py
@@ -9,7 +9,7 @@
 from aws_lambda_powertools.utilities.parser import BaseModel
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
diff --git a/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py b/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py
index af0c0d8bde3..b888b58a87c 100644
--- a/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py
+++ b/examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py
@@ -9,7 +9,7 @@
 from aws_lambda_powertools.utilities.parser import BaseModel
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(event_key_jmespath="order_id")  # see Choosing a payload subset section
 
diff --git a/examples/idempotency/src/working_with_record_expiration.py b/examples/idempotency/src/working_with_record_expiration.py
index 5b222ec7cdc..e1696ee7bbf 100644
--- a/examples/idempotency/src/working_with_record_expiration.py
+++ b/examples/idempotency/src/working_with_record_expiration.py
@@ -7,7 +7,7 @@
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath="body",
diff --git a/examples/idempotency/src/working_with_response_hook.py b/examples/idempotency/src/working_with_response_hook.py
index e851131781c..e800f3a0356 100644
--- a/examples/idempotency/src/working_with_response_hook.py
+++ b/examples/idempotency/src/working_with_response_hook.py
@@ -31,7 +31,7 @@ def my_response_hook(response: Dict, idempotent_data: DataRecord) -> Dict:
     return response
 
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 dynamodb = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(response_hook=my_response_hook)
 
diff --git a/examples/idempotency/src/working_with_validation_payload.py b/examples/idempotency/src/working_with_validation_payload.py
index 80ab43ad53d..12b8423e7c4 100644
--- a/examples/idempotency/src/working_with_validation_payload.py
+++ b/examples/idempotency/src/working_with_validation_payload.py
@@ -13,7 +13,7 @@
 
 logger = Logger()
 
-table = os.getenv("IDEMPOTENCY_TABLE")
+table = os.getenv("IDEMPOTENCY_TABLE", "")
 persistence_layer = DynamoDBPersistenceLayer(table_name=table)
 config = IdempotencyConfig(
     event_key_jmespath='["user_id", "product_id"]',

From 9dc9e94972dd8d93534ed9c5cb7d70d4eeee93aa Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 23 Jul 2024 09:36:39 +0200
Subject: [PATCH 32/41] docs: fix conflicts out of order

---
 docs/utilities/idempotency.md | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 4843aea1f31..540a8c0c3ca 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -105,7 +105,7 @@ Unless you're looking to use an [existing table or customize each attribute](#dy
 
 Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
 
-##### IaC examples
+##### DynamoDB IaC examples
 
 === "AWS Serverless Application Model (SAM) example"
 
@@ -134,9 +134,9 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
 
 #### Redis cluster
 
-**TODO**: Experiment bringing upfront Redis even at the cost of readability, as setup and usage are disconnected today causing further harm.
+We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
 
-##### Constraints
+In both services and self-hosting Redis, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
 
 ##### Redis IaC examples
 
@@ -356,7 +356,7 @@ By default, we protect against [concurrent executions](#handling-concurrent-exec
 
 To prevent extended failures, use **`register_lambda_context`** function from your idempotency config to calculate and include the remaining invocation time in your idempotency record.
 
-```python title="working_with_lambda_timeout.py" hl_lines="11 20"
+```python title="working_with_lambda_timeout.py" hl_lines="14 23"
 --8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
 ```
 
@@ -384,7 +384,7 @@ If an exception is handled or raised **outside** your decorated function, then i
 
 This persistence layer is built-in, allowing you to use an existing DynamoDB table or create a new one dedicated to idempotency state (recommended).
 
-```python title="customize_persistence_layer.py" hl_lines="7-15"
+```python title="customize_persistence_layer.py" hl_lines="10-18"
 --8<-- "examples/idempotency/src/customize_persistence_layer.py"
 ```
 
@@ -441,12 +441,12 @@ For simple setups, initialize `RedisCachePersistenceLayer` with your cluster end
 For security, we enforce SSL connections by default; to disable it, set `ssl=False`.
 
 === "Redis quick start"
-    ```python title="getting_started_with_idempotency_redis_config.py" hl_lines="7-9 12 26"
+    ```python title="getting_started_with_idempotency_redis_config.py" hl_lines="8-10 14 27"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_config.py"
     ```
 
 === "Using an existing Redis client"
-    ```python title="getting_started_with_idempotency_redis_client.py" hl_lines="4 9-11 14 22 36"
+    ```python title="getting_started_with_idempotency_redis_client.py" hl_lines="5 10-11 16 24 38"
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_client.py"
     ```
 
@@ -504,7 +504,7 @@ You can customize the attribute names during initialization:
 | **data_attr**               |          | `data`                   | Stores results of successfully executed Lambda handlers                                       |
 | **validation_key_attr**     |          | `validation`             | Hashed representation of the parts of the event used for validation                           |
 
-```python title="customize_persistence_layer_redis.py" hl_lines="9-16"
+```python title="customize_persistence_layer_redis.py" hl_lines="15-18"
 --8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
 ```
 

From aeece160875e41d30e4d7c2808a4c4e4969ff3e3 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 23 Jul 2024 09:45:34 +0200
Subject: [PATCH 33/41] docs(leandro's feedback): add caching in key features

---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 540a8c0c3ca..f128ae73cbc 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -12,7 +12,7 @@ The idempotency utility allows you to retry operations with the same input withi
 * Produces the same result when a function is called repeatedly with the same idempotency key
 * Choose your idempotency key from one or more fields, or entire payload
 * Safeguard concurrent requests, timeouts, missing idempotency keys, and payload tampering
-* Support for Amazon DynamoDB, Redis, and bring your own persistence layer
+* Support for Amazon DynamoDB, Redis, bring your own persistence layer, and in-memory caching
 
 ## Terminology
 

From 8d3fee29db84746cb853a5daab959f53ecb8f9fa Mon Sep 17 00:00:00 2001
From: Heitor Lessa <lessa@amazon.nl>
Date: Tue, 23 Jul 2024 10:41:51 +0200
Subject: [PATCH 34/41] Apply suggestions from code review

Co-authored-by: Leandro Damascena <lcdama@amazon.pt>
Signed-off-by: Heitor Lessa <lessa@amazon.nl>
---
 docs/utilities/idempotency.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index f128ae73cbc..1d832fb0aee 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -101,7 +101,7 @@ Unless you're looking to use an [existing table or customize each attribute](#dy
 | Configuration      | Value        | Notes                                                                                    |
 | ------------------ | ------------ | ---------------------------------------------------------------------------------------- |
 | Partition key      | `id`         | Format: <br> `{lambda_fn_name}.{module_name}.{fn_qualified_name}#{idempotency_key_hash}` |
-| TTL attribute name | `expiration` | Using AWS Console? this is configurable after table creation                             |
+| TTL attribute name | `expiration` | Using AWS Console? This is configurable after table creation                             |
 
 Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
 
@@ -207,7 +207,7 @@ By default, `idempotent_function` serializes, stores, and returns your annotated
 
 The output serializer supports any JSON serializable data, **Python Dataclasses** and **Pydantic Models**.
 
-!!! info "When using the `output_serializer` parameter, the data will continue to be stored in DynamoDB as a JSON string."
+!!! info "When using the `output_serializer` parameter, the data will continue to be stored in your persistent storage as a JSON string."
 
 === "Pydantic"
 
@@ -321,7 +321,7 @@ We want to use `user_id` and `product_id` fields as our idempotency key. **If we
 
 ### Adjusting expiration window
 
-!!! note "We expire idempotency records after **an hour** (3600 seconds). After that, a transaction with the same payload [will not be considered idempotent](#expired-idempotency-records)."
+!!! note "By default, we expire idempotency records after **an hour** (3600 seconds). After that, a transaction with the same payload [will not be considered idempotent](#expired-idempotency-records)."
 
 You can change this expiration window with the **`expires_after_seconds`** parameter. There is no limit on how long this expiration window can be set to.
 
@@ -436,7 +436,7 @@ You can customize the attribute names during initialization:
 
 !!! info "We recommend Redis version 7 or higher for optimal performance."
 
-For simple setups, initialize `RedisCachePersistenceLayer` with your cluster endpoint and port to connect.
+For simple setups, initialize `RedisCachePersistenceLayer` with your Redis endpoint and port to connect.
 
 For security, we enforce SSL connections by default; to disable it, set `ssl=False`.
 

From 3c4e56bf3b80b770684bce7c831c2ad69023822e Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 23 Jul 2024 10:12:28 +0200
Subject: [PATCH 35/41] docs(leandro's feedback): time window placement

---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 1d832fb0aee..b66108a1709 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -5,7 +5,7 @@ description: Utility
 
 <!-- markdownlint-disable MD051 -->
 
-The idempotency utility allows you to retry operations with the same input within a time window, producing the same output.
+The idempotency utility allows you to retry operations within a time window with the same input, producing the same output.
 
 ## Key features
 

From fc02e17eafd7d68b8e25a5ae32186b40c6cbad83 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 23 Jul 2024 10:13:02 +0200
Subject: [PATCH 36/41] docs(leandro's feedback): key features success vs
 failure ambiguity

---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index b66108a1709..f03d488e168 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -9,7 +9,7 @@ The idempotency utility allows you to retry operations within a time window with
 
 ## Key features
 
-* Produces the same result when a function is called repeatedly with the same idempotency key
+* Produces the previous successful result when a function is called repeatedly with the same idempotency key
 * Choose your idempotency key from one or more fields, or entire payload
 * Safeguard concurrent requests, timeouts, missing idempotency keys, and payload tampering
 * Support for Amazon DynamoDB, Redis, bring your own persistence layer, and in-memory caching

From ea159c9ab0a76061da37fed5628ebefe81dc634c Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 23 Jul 2024 10:24:10 +0200
Subject: [PATCH 37/41] docs(leandro's feedback): Redis anchor name

---
 docs/utilities/idempotency.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index f03d488e168..81371ef0a70 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -52,7 +52,7 @@ classDiagram
 
 ## Getting started
 
-We use Amazon DynamoDB as the default persistence layer in the documentation. If you prefer Redis, you can learn more from [this section](#redis-cluster).
+We use Amazon DynamoDB as the default persistence layer in the documentation. If you prefer Redis, you can learn more from [this section](#redis-database).
 
 ### IAM Permissions
 
@@ -80,7 +80,7 @@ To start, you'll need:
 
     ---
 
-    [Amazon DynamoDB](#dynamodb-table) or [Redis](#redis-cluster)
+    [Amazon DynamoDB](#dynamodb-table) or [Redis](#redis-database)
 
 *   :simple-awslambda:{ .lg .middle } **AWS Lambda function**
 
@@ -126,13 +126,13 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
 
 ##### Limitations
 
-* **DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-cluster) as an alternative.
+* **DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-database) as an alternative.
 
 * **Expect 2 WCU per non-idempotent call**. During the first invocation, we use `PutItem` for locking and `UpdateItem` for completion. Consider reviewing [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"} to estimate cost.
 
 * **Old boto3 versions can increase costs**. For cost optimization, we use a conditional `PutItem` to always lock a new idempotency record. If locking fails, it means we already have an idempotency record saving us an additional `GetItem` call. However, this is only supported in boto3 `1.26.194` and higher _([June 30th 2023](https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-dynamodb-cost-failed-conditional-writes/){target="_blank"})_.
 
-#### Redis cluster
+#### Redis database
 
 We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
 

From b1ad23e19975d195a14b2c0edf875c3efecc13c1 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Tue, 23 Jul 2024 10:27:35 +0200
Subject: [PATCH 38/41] docs(leandro's feedback): remove ambiguity on Redis VPC
 connectivity

---
 docs/utilities/idempotency.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 81371ef0a70..52d6fdc23c4 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -123,6 +123,7 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
     ```terraform hl_lines="14-26 64-70"
     --8<-- "examples/idempotency/templates/terraform.tf"
     ```
+`
 
 ##### Limitations
 
@@ -136,7 +137,7 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
 
 We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
 
-In both services and self-hosting Redis, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
+In both services, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
 
 ##### Redis IaC examples
 

From f052d2cb54e4963cfed99750c348ab039e7264d4 Mon Sep 17 00:00:00 2001
From: Heitor Lessa <lessa@amazon.nl>
Date: Tue, 23 Jul 2024 10:44:28 +0200
Subject: [PATCH 39/41] Update docs/utilities/idempotency.md

Co-authored-by: Leandro Damascena <lcdama@amazon.pt>
Signed-off-by: Heitor Lessa <lessa@amazon.nl>
---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 52d6fdc23c4..8b7a56bc77f 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -24,7 +24,7 @@ The property of idempotency means that an operation does not cause additional si
 
 **Persistence layer** is a storage we use to read, create, expire, and delete idempotency records.
 
-**Idempotency record** is the data representation of an idempotent request in its various status. We use it to coordinate  whether **(a)** a request is idempotent, **(b)** it's not expired, **(c)** JSON response to return, and more.
+**Idempotency record** is the data representation of an idempotent request saved in the persistent layer and in its various status. We use it to coordinate  whether **(a)** a request is idempotent, **(b)** it's not expired, **(c)** JSON response to return, and more.
 
 <center>
 ```mermaid

From 738f2fc989422c03f69e0d5166e32796c63677d4 Mon Sep 17 00:00:00 2001
From: Heitor Lessa <lessa@amazon.nl>
Date: Tue, 23 Jul 2024 10:44:52 +0200
Subject: [PATCH 40/41] Update docs/utilities/idempotency.md

Co-authored-by: Leandro Damascena <lcdama@amazon.pt>
Signed-off-by: Heitor Lessa <lessa@amazon.nl>
---
 docs/utilities/idempotency.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index 8b7a56bc77f..e5c3f0239c0 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -22,7 +22,7 @@ The property of idempotency means that an operation does not cause additional si
 
 **Idempotent request** is an operation with the same input previously processed that is not expired in your persistent storage or in-memory cache.
 
-**Persistence layer** is a storage we use to read, create, expire, and delete idempotency records.
+**Persistence layer** is a storage we use to create, read, expire, and delete idempotency records.
 
 **Idempotency record** is the data representation of an idempotent request saved in the persistent layer and in its various status. We use it to coordinate  whether **(a)** a request is idempotent, **(b)** it's not expired, **(c)** JSON response to return, and more.
 

From f8318f456a69b617c29a5f1677dc2f3b87bf5fa2 Mon Sep 17 00:00:00 2001
From: heitorlessa <lessa@amazon.co.uk>
Date: Thu, 25 Jul 2024 10:24:01 +0200
Subject: [PATCH 41/41] docs: move primary key for both persistence storages
 plus additional ctx

---
 docs/utilities/idempotency.md | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md
index e5c3f0239c0..06bf15748cb 100644
--- a/docs/utilities/idempotency.md
+++ b/docs/utilities/idempotency.md
@@ -92,18 +92,22 @@ To start, you'll need:
 
 <!-- markdownlint-enable MD030 -->
 
-#### DynamoDB table
+!!! note "Primary key for any persistence storage"
+    We combine the Lambda function name and the [fully qualified name](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} for classes/functions to
+    prevent accidental reuse for similar code sharing input/output.
+
+    Primary key sample: `{lambda_fn_name}.{module_name}.{fn_qualified_name}#{idempotency_key_hash}`
 
-!!! tip "You can share a single state table for all functions"
+#### DynamoDB table
 
 Unless you're looking to use an [existing table or customize each attribute](#dynamodbpersistencelayer), you only need the following:
 
-| Configuration      | Value        | Notes                                                                                    |
-| ------------------ | ------------ | ---------------------------------------------------------------------------------------- |
-| Partition key      | `id`         | Format: <br> `{lambda_fn_name}.{module_name}.{fn_qualified_name}#{idempotency_key_hash}` |
-| TTL attribute name | `expiration` | Using AWS Console? This is configurable after table creation                             |
+| Configuration      | Value        | Notes                                                        |
+| ------------------ | ------------ | ------------------------------------------------------------ |
+| Partition key      | `id`         |                                                              |
+| TTL attribute name | `expiration` | Using AWS Console? This is configurable after table creation |
 
-Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
+You **can** use a single DynamoDB table for all functions annotated with Idempotency.
 
 ##### DynamoDB IaC examples