Skip to content

Commit e09fe44

Browse files
author
Tom McCarthy
authored
docs(idempotency): add support for DynamoDB composite keys (#808)
1 parent 56ee00a commit e09fe44

File tree

1 file changed

+62
-6
lines changed

1 file changed

+62
-6
lines changed

docs/utilities/idempotency.md

+62-6
Original file line numberDiff line numberDiff line change
@@ -289,16 +289,40 @@ The client was successful in receiving the result after the retry. Since the Lam
289289

290290
### Handling exceptions
291291

292-
**The record in the persistence layer will be deleted** if your Lambda handler returns an exception. This means that new invocations will execute again despite having the same payload.
292+
If you are using the `idempotent` decorator on your Lambda handler, any unhandled exceptions that are raised during the code execution will cause **the record in the persistence layer to be deleted**.
293+
This means that new invocations will execute your code again despite having the same payload. If you don't want the record to be deleted, you need to catch exceptions within the idempotent function and return a successful response.
293294

294-
If you don't want the record to be deleted, you need to catch exceptions within the handler and return a successful response.
295295

296296
![Idempotent sequence exception](../media/idempotent_sequence_exception.png)
297297

298+
If you are using `idempotent_function`, any unhandled exceptions that are raised _inside_ the decorated function will cause the record in the persistence layer to be deleted, and allow the function to be executed again if retried.
299+
If an Exception is raised _outside_ the scope of the decorated function and after your function has been called, the persistent record will not be affected. In this case, idempotency will be maintained for your decorated function. Example:
300+
301+
=== "app.py"
302+
303+
```python hl_lines="2-4 8-10"
304+
def lambda_handler(event, context):
305+
# If an exception is raised here, no idempotent record will ever get created as the
306+
# idempotent function does not get called
307+
do_some_stuff()
308+
309+
result = call_external_service(data={"user": "user1", "id": 5})
310+
311+
# This exception will not cause the idempotent record to be deleted, since it
312+
# happens after the decorated function has been successfully called
313+
raise Exception
314+
315+
316+
@idempotent_function(data_keyword_argument="data", config=config, persistence_store=dynamodb)
317+
def call_external_service(data: dict, **kwargs):
318+
result = requests.post('http://example.com', json={"user": data['user'], "transaction_id": data['id']}
319+
return result.json()
320+
```
321+
298322
!!! warning
299-
**We will raise `IdempotencyPersistenceLayerError`** if any of the calls to the persistence layer fail unexpectedly.
323+
**We will raise `IdempotencyPersistenceLayerError`** if any of the calls to the persistence layer fail unexpectedly.
300324

301-
As this happens outside the scope of your Lambda handler, you are not going to be able to catch it.
325+
As this happens outside the scope of your decorated function, you are not able to catch it if you're using the `idempotent` decorator on your Lambda handler.
302326

303327
### Persistence layers
304328

@@ -321,16 +345,18 @@ This persistence layer is built-in, and you can either use an existing DynamoDB
321345
)
322346
```
323347

324-
These are knobs you can use when using DynamoDB as a persistence layer:
348+
When using DynamoDB as a persistence layer, you can alter the attribute names by passing these parameters when initializing the persistence layer:
325349

326350
Parameter | Required | Default | Description
327351
------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------
328352
**table_name** | :heavy_check_mark: | | Table name to store state
329-
**key_attr** | | `id` | Primary key of the table. Hashed representation of the payload
353+
**key_attr** | | `id` | Partition key of the table. Hashed representation of the payload (unless **sort_key_attr** is specified)
330354
**expiry_attr** | | `expiration` | Unix timestamp of when record expires
331355
**status_attr** | | `status` | Stores status of the lambda execution during and after invocation
332356
**data_attr** | | `data` | Stores results of successfully executed Lambda handlers
333357
**validation_key_attr** | | `validation` | Hashed representation of the parts of the event used for validation
358+
**sort_key_attr** | | | Sort key of the table (if table is configured with a sort key).
359+
**static_pk_value** | | `idempotency#{LAMBDA_FUNCTION_NAME}` | Static value to use as the partition key. Only used when **sort_key_attr** is set.
334360

335361
## Advanced
336362

@@ -590,6 +616,36 @@ The **`boto_config`** and **`boto3_session`** parameters enable you to pass in a
590616
...
591617
```
592618

619+
### Using a DynamoDB table with a composite primary key
620+
621+
If you wish to use this utility with a DynamoDB table that is configured with a composite primary key (uses both partition key and sort key), you
622+
should set the `sort_key_attr` parameter when initializing your persistence layer. When this parameter is set, the partition key value for all idempotency entries
623+
will be the same, with the idempotency key being saved as the sort key instead of the partition key. You can optionally set a static value for the partition
624+
key using the `static_pk_value` parameter. If not specified, it will default to `idempotency#{LAMBDA_FUNCTION_NAME}`.
625+
626+
=== "MyLambdaFunction"
627+
628+
```python hl_lines="5"
629+
from aws_lambda_powertools.utilities.idempotency import DynamoDBPersistenceLayer, idempotent
630+
631+
persistence_layer = DynamoDBPersistenceLayer(
632+
table_name="IdempotencyTable",
633+
sort_key_attr='sort_key')
634+
635+
636+
@idempotent(persistence_store=persistence_layer)
637+
def handler(event, context):
638+
return {"message": "success": "id": event['body']['id]}
639+
```
640+
641+
The example function above would cause data to be stored in DynamoDB like this:
642+
643+
| id | sort_key | expiration | status | data |
644+
|------------------------------|----------------------------------|------------|-------------|-------------------------------------|
645+
| idempotency#MyLambdaFunction | 1e956ef7da78d0cb890be999aecc0c9e | 1636549553 | COMPLETED | {"id": 12391, "message": "success"} |
646+
| idempotency#MyLambdaFunction | 2b2cdb5f86361e97b4383087c1ffdf27 | 1636549571 | COMPLETED | {"id": 527212, "message": "success"}|
647+
| idempotency#MyLambdaFunction | f091d2527ad1c78f05d54cc3f363be80 | 1636549585 | IN_PROGRESS | |
648+
593649
### Bring your own persistent store
594650

595651
This utility provides an abstract base class (ABC), so that you can implement your choice of persistent storage layer.

0 commit comments

Comments
 (0)