refactor(idempotency): replace Redis name with Cache and add valkey-glide support (#6685)

* Adding cache name

* Replacing Redis with Cache

* More changes

Author: leandrodamascena

Makefile

@@ -7,18 +7,18 @@ target:
 dev:
 	pip install --upgrade pip pre-commit poetry
 	@$(MAKE) dev-version-plugin
-	poetry install --extras "all redis datamasking"
+	poetry install --extras "all redis datamasking valkey"
 	pre-commit install
 
 dev-quality-code:
 	pip install --upgrade pip pre-commit poetry
 	@$(MAKE) dev-version-plugin
-	poetry install --extras "all redis datamasking"
+	poetry install --extras "all redis datamasking valkey"
 	pre-commit install
 
 dev-gitpod:
 	pip install --upgrade pip poetry
-	poetry install --extras "all redis datamasking"
+	poetry install --extras "all redis datamasking valkey"
 	pre-commit install
 
 format-check:

aws_lambda_powertools/utilities/idempotency/persistence/cache.py

@@ -0,0 +1,11 @@
+from aws_lambda_powertools.utilities.idempotency.persistence.redis import (
+    CacheClientProtocol,
+    CacheConnection,
+    CachePersistenceLayer,
+)
+
+__all__ = [
+    "CacheClientProtocol",
+    "CachePersistenceLayer",
+    "CacheConnection",
+]

aws_lambda_powertools/utilities/idempotency/persistence/redis.py

@@ -12,7 +12,7 @@ The idempotency utility allows you to retry operations within a time window with
 * Produces the previous successful result when a function is called repeatedly with the same idempotency key
 * Choose your idempotency key from one or more fields, or entire payload
 * Safeguard concurrent requests, timeouts, missing idempotency keys, and payload tampering
-* Support for Amazon DynamoDB, Redis, bring your own persistence layer, and in-memory caching
+* Support for Amazon DynamoDB, Valkey, Redis OSS, or any Redis-compatible cache as the persistence layer
 
 ## Terminology
 
@@ -82,7 +82,7 @@ To start, you'll need:
 
     ---
 
-    [Amazon DynamoDB](#dynamodb-table) or [Redis](#redis-database)
+    [Amazon DynamoDB](#dynamodb-table) or [Valkey/Redis OSS/Redis compatible](#cache-database)
 
 *   :simple-awslambda:{ .lg .middle } **AWS Lambda function**
 
@@ -139,13 +139,13 @@ You **can** use a single DynamoDB table for all functions annotated with Idempot
 
 * **Old boto3 versions can increase costs**. For cost optimization, we use a conditional `PutItem` to always lock a new idempotency record. If locking fails, it means we already have an idempotency record saving us an additional `GetItem` call. However, this is only supported in boto3 `1.26.194` and higher _([June 30th 2023](https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-dynamodb-cost-failed-conditional-writes/){target="_blank"})_.
 
-#### Redis database
+#### Cache database
 
-We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
+We recommend starting with a managed cache service, such as [Amazon ElastiCache for Valkey and for Redis OSS](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB](https://aws.amazon.com/memorydb/){target="_blank"}.
 
 In both services, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
 
-##### Redis IaC examples
+##### Cache configuration
 
 === "AWS CloudFormation example"
 
@@ -160,7 +160,7 @@ In both services, you'll need to configure [VPC access](https://docs.aws.amazon.
     1. Replace the Security Group ID and Subnet ID to match your VPC settings.
     2. Replace the Security Group ID and Subnet ID to match your VPC settings.
 
-Once setup, you can find a quick start and advanced examples for Redis in [the persistent layers section](#redispersistencelayer).
+Once setup, you can find a quick start and advanced examples for Cache in [the persistent layers section](#cachepersistencelayer).
 
 <!-- markdownlint-enable MD013 -->
 
@@ -464,17 +464,22 @@ You can customize the attribute names during initialization:
 | **sort_key_attr**           |                    |                                      | Sort key of the table (if table is configured with a sort key).                                          |
 | **static_pk_value**         |                    | `idempotency#{LAMBDA_FUNCTION_NAME}` | Static value to use as the partition key. Only used when **sort_key_attr** is set.                       |
 
-#### RedisPersistenceLayer
+#### CachePersistenceLayer
 
-!!! info "We recommend Redis version 7 or higher for optimal performance."
+The `CachePersistenceLayer` enables you to use Valkey, Redis OSS, or any Redis-compatible cache as the persistence layer for idempotency state.
 
-For simple setups, initialize `RedisCachePersistenceLayer` with your Redis endpoint and port to connect.
+We recommend using [`valkey-glide`](https://pypi.org/project/valkey-glide/){target="_blank"} for Valkey or [`redis`](https://pypi.org/project/redis/){target="_blank"} for Redis. However, any Redis OSS-compatible client should work.
 
-For security, we enforce SSL connections by default; to disable it, set `ssl=False`.
+For simple setups, initialize `CachePersistenceLayer` with your Cache endpoint and port to connect. Note that for security, we enforce SSL connections by default; to disable it, set `ssl=False`.
 
-=== "Redis quick start"
-    ```python title="getting_started_with_idempotency_redis_config.py" hl_lines="8-10 14 27"
-    --8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_config.py"
+=== "Cache quick start"
+    ```python title="getting_started_with_idempotency_cache_config.py" hl_lines="8-10 14 27"
+    --8<-- "examples/idempotency/src/getting_started_with_idempotency_cache_config.py"
+    ```
+
+=== "Using an existing Valkey Glide client"
+    ```python title="getting_started_with_idempotency_valkey_client.py" hl_lines="5 10-12 16-22 24 37"
+    --8<-- "examples/idempotency/src/getting_started_with_idempotency_valkey_client.py"
     ```
 
 === "Using an existing Redis client"
@@ -488,23 +493,23 @@ For security, we enforce SSL connections by default; to disable it, set `ssl=Fal
     --8<-- "examples/idempotency/src/getting_started_with_idempotency_payload.json"
     ```
 
-##### Redis SSL connections
+##### Cache SSL connections
 
 We recommend using AWS Secrets Manager to store and rotate certificates safely, and the [Parameters feature](./parameters.md){target="_blank"} to fetch and cache optimally.
 
-For advanced configurations, we recommend using an existing Redis client for optimal compatibility like SSL certificates and timeout.
+For advanced configurations, we recommend using an existing Valkey client for optimal compatibility like SSL certificates and timeout.
 
 === "Advanced configuration using AWS Secrets"
-    ```python title="using_redis_client_with_aws_secrets.py" hl_lines="9-11 13 15 25"
-    --8<-- "examples/idempotency/src/using_redis_client_with_aws_secrets.py"
+    ```python title="using_cache_client_with_aws_secrets.py" hl_lines="5 9-11 13 15 18 19 23"
+    --8<-- "examples/idempotency/src/using_cache_client_with_aws_secrets.py"
     ```
 
     1. JSON stored:
     ```json
     {
-    "REDIS_ENDPOINT": "127.0.0.1",
-    "REDIS_PORT": "6379",
-    "REDIS_PASSWORD": "redis-secret"
+    "CACHE_HOST": "127.0.0.1",
+    "CACHE_PORT": "6379",
+    "CACHE_PASSWORD": "cache-secret"
     }
     ```
 
@@ -516,16 +521,16 @@ For advanced configurations, we recommend using an existing Redis client for opt
     1. JSON stored:
     ```json
     {
-    "REDIS_ENDPOINT": "127.0.0.1",
-    "REDIS_PORT": "6379",
-    "REDIS_PASSWORD": "redis-secret"
+    "CACHE_HOST": "127.0.0.1",
+    "CACHE_PORT": "6379",
+    "CACHE_PASSWORD": "cache-secret"
     }
     ```
-    2. redis_user.crt file stored in the "certs" directory of your Lambda function
-    3. redis_user_private.key file stored in the "certs" directory of your Lambda function
-    4. redis_ca.pem file stored in the "certs" directory of your Lambda function
+    2. cache_user.crt file stored in the "certs" directory of your Lambda function
+    3. cache_user_private.key file stored in the "certs" directory of your Lambda function
+    4. cache_ca.pem file stored in the "certs" directory of your Lambda function
 
-##### Redis attributes
+##### Cache attributes
 
 You can customize the attribute names during initialization:
 
@@ -811,28 +816,28 @@ sequenceDiagram
 <i>Optional idempotency key</i>
 </center>
 
-#### Race condition with Redis
+#### Race condition with Cache
 
 <center>
 ```mermaid
 graph TD;
-    A(Existing orphan record in redis)-->A1;
+    A(Existing orphan record in cache)-->A1;
     A1[Two Lambda invoke at same time]-->B1[Lambda handler1];
-    B1-->B2[Fetch from Redis];
+    B1-->B2[Fetch from Cache];
     B2-->B3[Handler1 got orphan record];
     B3-->B4[Handler1 acquired lock];
     B4-->B5[Handler1 overwrite orphan record]
     B5-->B6[Handler1 continue to execution];
     A1-->C1[Lambda handler2];
-    C1-->C2[Fetch from Redis];
+    C1-->C2[Fetch from Cache];
     C2-->C3[Handler2 got orphan record];
     C3-->C4[Handler2 failed to acquire lock];
-    C4-->C5[Handler2 wait and fetch from Redis];
+    C4-->C5[Handler2 wait and fetch from Cache];
     C5-->C6[Handler2 return without executing];
     B6-->D(Lambda handler executed only once);
     C6-->D;
 ```
-<i>Race condition with Redis</i>
+<i>Race condition with Cache</i>
 </center>
 
 ## Advanced

docs/utilities/idempotency.md

@@ -5,13 +5,13 @@
 from aws_lambda_powertools.utilities.idempotency import (
     idempotent,
 )
-from aws_lambda_powertools.utilities.idempotency.persistence.redis import (
-    RedisCachePersistenceLayer,
+from aws_lambda_powertools.utilities.idempotency.persistence.cache import (
+    CachePersistenceLayer,
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT", "localhost")
-persistence_layer = RedisCachePersistenceLayer(host=redis_endpoint, port=6379)
+redis_endpoint = os.getenv("CACHE_CLUSTER_ENDPOINT", "localhost")
+persistence_layer = CachePersistenceLayer(host=redis_endpoint, port=6379)
 
 
 @dataclass
@@ -34,7 +34,7 @@ def lambda_handler(event: dict, context: LambdaContext):
             "statusCode": 200,
         }
     except Exception as exc:
-        raise PaymentError(f"Error creating payment {str(exc)}")
+        raise PaymentError(f"Error creating payment {str(exc)}") from exc
 
 
 def create_subscription_payment(event: dict) -> Payment:

examples/idempotency/src/getting_started_with_idempotency_redis_config.py renamed to examples/idempotency/src/getting_started_with_idempotency_cache_config.py

@@ -7,21 +7,21 @@
 from aws_lambda_powertools.utilities.idempotency import (
     idempotent,
 )
-from aws_lambda_powertools.utilities.idempotency.persistence.redis import (
-    RedisCachePersistenceLayer,
+from aws_lambda_powertools.utilities.idempotency.persistence.cache import (
+    CachePersistenceLayer,
 )
 from aws_lambda_powertools.utilities.typing import LambdaContext
 
-redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT", "localhost")
+cache_endpoint = os.getenv("CACHE_CLUSTER_ENDPOINT", "localhost")
 client = Redis(
-    host=redis_endpoint,
+    host=cache_endpoint,
     port=6379,
     socket_connect_timeout=5,
     socket_timeout=5,
     max_connections=1000,
 )
 
-persistence_layer = RedisCachePersistenceLayer(client=client)
+persistence_layer = CachePersistenceLayer(client=client)
 
 
 @dataclass

examples/idempotency/src/getting_started_with_idempotency_redis_client.py

@@ -0,0 +1,53 @@
+import os
+from dataclasses import dataclass, field
+from uuid import uuid4
+
+from glide import GlideClient, GlideClientConfiguration, NodeAddress
+
+from aws_lambda_powertools.utilities.idempotency import (
+    idempotent,
+)
+from aws_lambda_powertools.utilities.idempotency.persistence.cache import (
+    CachePersistenceLayer,
+)
+from aws_lambda_powertools.utilities.typing import LambdaContext
+
+cache_endpoint = os.getenv("CACHE_CLUSTER_ENDPOINT", "localhost")
+client_config = GlideClientConfiguration(
+    addresses=[
+        NodeAddress(
+            host="localhost",
+            port=6379,
+        ),
+    ],
+)
+client = GlideClient.create(config=client_config)
+
+persistence_layer = CachePersistenceLayer(client=client)  # type: ignore[arg-type]
+
+
+@dataclass
+class Payment:
+    user_id: str
+    product_id: str
+    payment_id: str = field(default_factory=lambda: f"{uuid4()}")
+
+
+class PaymentError(Exception): ...
+
+
+@idempotent(persistence_store=persistence_layer)
+def lambda_handler(event: dict, context: LambdaContext):
+    try:
+        payment: Payment = create_subscription_payment(event)
+        return {
+            "payment_id": payment.payment_id,
+            "message": "success",
+            "statusCode": 200,
+        }
+    except Exception as exc:
+        raise PaymentError(f"Error creating payment {str(exc)}")
+
+
+def create_subscription_payment(event: dict) -> Payment:
+    return Payment(**event)

examples/idempotency/src/getting_started_with_idempotency_valkey_client.py

@@ -0,0 +1,39 @@
+from __future__ import annotations
+
+from typing import Any
+
+from glide import BackoffStrategy, GlideClient, GlideClientConfiguration, NodeAddress, ServerCredentials
+
+from aws_lambda_powertools.utilities import parameters
+from aws_lambda_powertools.utilities.idempotency import IdempotencyConfig, idempotent
+from aws_lambda_powertools.utilities.idempotency.persistence.cache import (
+    CachePersistenceLayer,
+)
+
+cache_values: dict[str, Any] = parameters.get_secret("cache_info", transform="json")  # (1)!
+
+client_config = GlideClientConfiguration(
+    addresses=[
+        NodeAddress(
+            host=cache_values.get("CACHE_HOST", "localhost"),
+            port=cache_values.get("CACHE_PORT", 6379),
+        ),
+    ],
+    credentials=ServerCredentials(
+        password=cache_values.get("CACHE_PASSWORD", ""),
+    ),
+    request_timeout=10,
+    use_tls=True,
+    reconnect_strategy=BackoffStrategy(num_of_retries=10, factor=2, exponent_base=1),
+)
+valkey_client = GlideClient.create(config=client_config)
+
+persistence_layer = CachePersistenceLayer(client=valkey_client)  # type: ignore[arg-type]
+config = IdempotencyConfig(
+    expires_after_seconds=2 * 60,  # 2 minutes
+)
+
+
+@idempotent(config=config, persistence_store=persistence_layer)
+def lambda_handler(event, context):
+    return {"message": "Hello"}