v1.21.0

Summary

After some vacation period, we're back with a new minor release with major features:

• Bring your own boto3 sessions for cross-account operations & snapshot testing
• New features on Feature Flags
• Idempotency unit testing made easier
• JSON Schema Validation utility contains new data elements to more easily construct a validation error
• New utility: we're now exposing our internal custom JMESPath Functions so you can easily decode and deserialize JSON objects found in various formats within Lambda Event Sources.

New Contributors

I'd like to personally thank our new contributors to the project :)

• @Tankanow made their first contribution in #697
• @DanyC97 made their first contribution in #716
• @gwlester made their first contribution in #710

Detailed changes

Boto3 sessions

You can now pass in your own boto3 session when using Parameters, Batch, and Idempotency.

This is helpful in two typical scenarios: 1/ You want to run an operation in another account like fetching secrets/parameters somewhere else, 2/ Use snapshot testing tools like Placebo that will replay session data that happened earlier when doing unit testing.

from aws_lambda_powertools.utilities import parameters
import boto3

boto3_session = boto3.session.Session()
ssm_provider = parameters.SSMProvider(boto3_session=boto3_session)

def handler(event, context):
    # Retrieve a single parameter
    value = ssm_provider.get("/my/parameter")
    ...

Feature flags

There's been three main improvements in Feature flags utility as part of this release: New rule conditions, Bring your own Logger for debugging, and Getting a copy of fetched configuration from the store

New rule conditions

You can now use the following new rule conditions to evaluate your feature flags for inequality, comparison, and more explicit contains logic, where a is the key and b is the value passed as a context input for evaluation:

Action	Equivalent expression
KEY_GREATER_THAN_VALUE	lambda a, b: a > b
KEY_GREATER_THAN_OR_EQUAL_VALUE	lambda a, b: a >= b
KEY_LESS_THAN_VALUE	lambda a, b: a < b
KEY_LESS_THAN_OR_EQUAL_VALUE	lambda a, b: a <= b
KEY_IN_VALUE	lambda a, b: a in b
KEY_NOT_IN_VALUE	lambda a, b: a not in b
VALUE_IN_KEY	lambda a, b: b in a
VALUE_NOT_IN_KEY	lambda a, b: b not in a

Example

Feature flag schema

{
    "premium_features": {
        "default": false,
        "rules": {
            "customer tier equals premium": {
                "when_match": true,
                "conditions": [
                    {
                        "action": "VALUE_IN_KEY",
                        "key": "groups",
                        "value": "PAID_PREMIUM",
                    }
                ]
            }
        }
    },
    "ten_percent_off_campaign": {
        "default": false
    }
}

App

from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="features"
)

feature_flags = FeatureFlags(store=app_config)

def lambda_handler(event, context):
    # groups: ["FREE_TIER", "PAID_BASIC", "PAID_PREMIUM"]
    ctx={"tenant_id": "6", "username": "a", "groups": event.get("groups", [])}

    # Evaluate whether customer's tier has access to premium features
    # based on `has_premium_features` rules
    has_premium_features: bool = feature_flags.evaluate(name="premium_features",
                                                        context=ctx, default=False)
    if has_premium_features:
        # enable premium features
        ...

Accessing raw configuration fetched

Previously, if you were using a single application configuration and a feature schema in a single AppConfig key, we would only use the feature flags schema and discard the rest.

You can now access the raw configuration with a new property get_raw_configuration within AppConfig Store:

from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="configuration",
    envelope = "feature_flags"
)

feature_flags = FeatureFlags(store=app_config)

config = app_config.get_raw_configuration

Unit testing idempotency

We have improved how you can unit test your code when using @idempotent and @idempotent_function decorators.

You can now disable all interactions with the idempotence store using POWERTOOLS_IDEMPOTENCY_DISABLED environment variable, and monkeypatch the DynamoDB resource client Idempotency utility uses if you wish to either use DynamoDB Local or mock all I/O operations.

import boto3

import app

def test_idempotent_lambda():
    # Create our own Table resource using the endpoint for our DynamoDB Local instance
    resource = boto3.resource("dynamodb", endpoint_url='http://localhost:8000')
    table = resource.Table(app.persistence_layer.table_name)
    app.persistence_layer.table = table

    result = app.handler({'testkey': 'testvalue'}, {})
    assert result['payment_id'] == 12345

New data elements for JSON Schema validation errors

When validating input/output with the Validator, you can now access new properties in SchemaValidationError to more easily construct your custom errors based on what went wrong.

Property	Type	Description
message	str	Powertools formatted error message
validation_message	str, optional	Containing human-readable information what is wrong, e.g. data.property[index] must be smaller than or equal to 42
name	str, optional	name of a path in the data structure, e.g. data.property[index]
path	List, optional	path as an array in the data structure, e.g. ['data', 'property', 'index']
value	Any, optional	The invalid value, e.g. {"message": "hello world"}
definition	Any, optional	JSON Schema definition
rule	str, optional	rule which the data is breaking (e.g. maximum, required)
rule_definition	Any, optional	The specific rule definition (e.g. 42, ['message', 'username'])

Sample

from aws_lambda_powertools.utilities.validation import validate
from aws_lambda_powertools.utilities.validation.exceptions import SchemaValidationError

import schemas

def handler(event, context):
    try:
        validate(event=event, schema=schemas.INPUT)
    except SchemaValidationError as e:
        message = "data must contain ['message', 'username'] properties"

        assert str(e.value) == e.value.message
        assert e.value.validation_message == message
        assert e.value.name == "data"
        assert e.value.path is not None
        assert e.value.value == data
        assert e.value.definition == schema
        assert e.value.rule == "required"
        assert e.value.rule_definition == schema.get("required")

        raise

    return event

New JMESPath Powertools functions

Last but not least, as part of a documentation revamp in Idempotency by @walmsles, we're now exposing an internal feature used by many Lambda Powertools utilities which is the ability to extract and decode JSON objects.

You can now use JMESPath (JSON Query language) Lambda Powertools functions to easily decode and deserialize JSON often found as compressed (Kinesis, CloudWatch Logs, etc), as strings (SNS, SQS, EventBridge, API Gateway, etc), or as base64 (Kinesis).

We're exposing three custom JMESPath functions you can use such as powertools_json, powertools_base64, powertools_base64_gzip, and a new standalone function that will use JMESPath to search and extract the data you want called extract_data_from_envelope.

Sample

from aws_lambda_powertools.utilities.jmespath_utils import extract_data_from_envelope
from aws_lambda_powertools.utilities.typing import LambdaContext


def handler(event: dict, context: LambdaContext...

v1.20.2

Summary

Bugfix release to address a bug with event handler which caused issues for api gateway events when using strip_prefix together with a root level handler. Thanks @BVMiko for the fix!

Changes

🐛 Bug and hot fixes

• fix: Fix issue with strip_prefixes (#647) by @BVMiko

🔧 Maintenance

• chore(deps-dev): bump mkdocs-material from 7.2.4 to 7.2.6 (#665) by @dependabot
• chore(deps): bump boto3 from 1.18.26 to 1.18.32 (#663) by @dependabot
• chore(deps-dev): bump pytest from 6.2.4 to 6.2.5 (#662) by @dependabot
• chore(license): Add THIRD-PARTY-LICENSES to pyproject.toml (#641) by @BastianZim

This release was made possible by the following contributors:

@BVMiko, @BastianZim, @cakepietoast

v1.20.1

Summary

Hotfix release to address a bug in idempotency hashing logic to ensure data is sorted as part of the process - Huge thanks to @walmsles!

We also now support CodeSpaces to ease contribution, also thanks to @michaelbrewer

Changes

📜 Documentation updates

• chore: markdown linter fixes (#636) by @michaelbrewer

🐛 Bug and hot fixes

• fix(idempotency): sorting keys before hashing (#639) by @heitorlessa

🔧 Maintenance

• chore: setup codespaces (#637) by @michaelbrewer
• chore(license): add third party license (#635) by @BastianZim

This release was made possible by the following contributors:

@BastianZim, @heitorlessa and @michaelbrewer

v1.20.0

Summary

This release highlights 1/ support for Python 3.9, 2/ support for [API Gateway][apigateway-http-authorizer] and [AppSync Lambda Authorizers][appsync-authorizer], 3/ support for API Gateway Custom Domain Mappings, 4/ support to make [any Python synchronous function idempotent][idempotency_function], and a number of documentation improvements & bugfixes.

Lambda Authorizer support

AppSync

This release adds Data Class support for AppSyncAuthorizerEvent, AppSyncAuthorizerResponse, and correlation ID in Logger.

You can use AppSyncAuthorizerEvent to easily access all self-documented properties, and AppSyncAuthorizerResponse to serialize the response in the expected format.

You can read more in the announcement blog post for more details

from typing import Dict

from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.logging.logger import Logger
from aws_lambda_powertools.utilities.data_classes.appsync_authorizer_event import (
    AppSyncAuthorizerEvent,
    AppSyncAuthorizerResponse,
)
from aws_lambda_powertools.utilities.data_classes.event_source import event_source

logger = Logger()


def get_user_by_token(token: str):
    """Look a user by token"""


@logger.inject_lambda_context(correlation_id_path=correlation_paths.APPSYNC_AUTHORIZER)
@event_source(data_class=AppSyncAuthorizerEvent)
def lambda_handler(event: AppSyncAuthorizerEvent, context) -> Dict:
    user = get_user_by_token(event.authorization_token)

    if not user:
        # No user found, return not authorized
        return AppSyncAuthorizerResponse().to_dict()

    return AppSyncAuthorizerResponse(
        authorize=True,
        resolver_context={"id": user.id},
        # Only allow admins to delete events
        deny_fields=None if user.is_admin else ["Mutation.deleteEvent"],
    ).asdict()

API Gateway

This release adds support for both Lambda Authorizer for payload v1 - APIGatewayAuthorizerRequestEvent, APIGatewayAuthorizerResponse - and v2 formats APIGatewayAuthorizerEventV2, APIGatewayAuthorizerResponseV2.

Similar to AppSync, you can use APIGatewayAuthorizerRequestEvent and APIGatewayAuthorizerEventV2 to easily access all self-documented properties available, and its corresponding APIGatewayAuthorizerResponse and APIGatewayAuthorizerResponseV2 to serialize the response in the expected format.

You can read more in the announcement blog post for more details

v2 format

from aws_lambda_powertools.utilities.data_classes import event_source
from aws_lambda_powertools.utilities.data_classes.api_gateway_authorizer_event import (
    APIGatewayAuthorizerEventV2,
    APIGatewayAuthorizerResponseV2,
)
from secrets import compare_digest


def get_user_by_token(token):
    if compare_digest(token, "Foo"):
        return {"name": "Foo"}
    return None


@event_source(data_class=APIGatewayAuthorizerEventV2)
def handler(event: APIGatewayAuthorizerEventV2, context):
    user = get_user_by_token(event.get_header_value("x-token"))

    if user is None:
        # No user was found, so we return not authorized
        return APIGatewayAuthorizerResponseV2().asdict()

    # Found the user and setting the details in the context
    return APIGatewayAuthorizerResponseV2(authorize=True, context=user).asdict()

v1 format

from aws_lambda_powertools.utilities.data_classes import event_source
from aws_lambda_powertools.utilities.data_classes.api_gateway_authorizer_event import (
    APIGatewayAuthorizerRequestEvent,
    APIGatewayAuthorizerResponse,
    HttpVerb,
)
from secrets import compare_digest


def get_user_by_token(token):
    if compare_digest(token, "admin-foo"):
        return {"isAdmin": True, "name": "Admin"}
    elif compare_digest(token, "regular-foo"):
        return {"name": "Joe"}
    else:
        return None


@event_source(data_class=APIGatewayAuthorizerRequestEvent)
def handler(event: APIGatewayAuthorizerRequestEvent, context):
    user = get_user_by_token(event.get_header_value("Authorization"))

    # parse the `methodArn` as an `APIGatewayRouteArn`
    arn = event.parsed_arn
    # Create the response builder from parts of the `methodArn`
    policy = APIGatewayAuthorizerResponse(
        principal_id="user",
        region=arn.region,
        aws_account_id=arn.aws_account_id,
        api_id=arn.api_id,
        stage=arn.stage
    )

    if user is None:
        # No user was found, so we return not authorized
        policy.deny_all_routes()
        return policy.asdict()

    # Found the user and setting the details in the context
    policy.context = user

    # Conditional IAM Policy
    if user.get("isAdmin", False):
        policy.allow_all_routes()
    else:
        policy.allow_route(HttpVerb.GET, "/user-profile")

    return policy.asdict()

Custom Domain API Mappings

When using Custom Domain API Mappings feature, you must use the new strip_prefixes param in the ApiGatewayResolver constructor.

Scenario: You have a custom domain api.mydomain.dev and set an API Mapping payment to forward requests to your Payments API, the path argument will be /payment/<your_actual_path>.

This will lead to a HTTP 404 despite having your Lambda configured correctly. See the example below on how to account for this change.

from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver

tracer = Tracer()
logger = Logger()
app = ApiGatewayResolver(strip_prefixes=["/payment"])

@app.get("/subscriptions/<subscription>")
@tracer.capture_method
def get_subscription(subscription):
    return {"subscription_id": subscription}

@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
@tracer.capture_lambda_handler
def lambda_handler(event, context):
    return app.resolve(event, context)

Make any Python function idempotent

Previously, you could only make the entire Lambda function handler idempotent. You can now make any Python function idempotent with the new [idempotent_function][idempotency_function].

This also enables easy integration with any other utility in Powertools. Take example the Batch utility, where you wouldn't want to make the entire Lambda handler idempotent as the batch will vary, instead you'd want to make sure you can process a given message only once.

As a trade-off to allow any Python function with an arbitrary number of parameters, you must call your function with a keyword argument, and you tell us upfront which one that might be using data_keyword_argument, so we can apply all operations like hashing the idempotency token, payload extraction, parameter validation, etc.

import uuid

from aws_lambda_powertools.utilities.batch import sqs_batch_processor
from aws_lambda_powertools.utilities.idempotency import idempotent_function, DynamoDBPersistenceLayer, IdempotencyConfig


dynamodb = DynamoDBPersistenceLayer(table_name="idem")
config =  IdempotencyConfig(
    event_key_jmespath="messageId",  # see "Choosing a payload subset for idempotency" docs section
    use_local_cache=True,
)

@idempotent_function(data_keyword_argument="data", config=config, persistence_store=dynamodb)
def dummy(arg_one, arg_two, data: dict, **kwargs):
    return {"data": data}


@idempotent_function(data_keyword_argument="record", config=config, persistence_store=dynamodb)
def record_handler(record):
    return {"message": record["body"]}


@sqs_batch_processor(record_handler=record_handler)
def lambda_handler(event, context):
    # `data` parameter must be called as a keyword argument to work
    dummy("hello", "universe", data="test")
    return {"statusCode": 200}

Changes

🌟New features and non-breaking changes

• feat(data-classes): authorizer for http api and rest api (#620) by @michaelbrewer
• feat(data-classes): data_as_bytes prop KinesisStreamRecordPayload (#628) by @hjurong
• feat(general): support for Python 3.9 (#626) by @heitorlessa
• feat(event-handler): prefixes to strip for custom mappings (#579) by @michaelbrewer
• feat(data-classes): AppSync Lambda authorizer event (#610) by @michaelbrewer
• feat(idempotency): support for any synchronous function (#625) by @heitorlessa

📜 Documentation updates

• docs(data-classes): make authorizer concise; use enum (#630) by @heitorlessa
• feat(data-classes): authorizer for http api and rest api (#620) by @michaelbrewer
• feat(data-classes): AppSync Lambda authorizer event (#610) by @michaelbrewer
• chore(docs): correct markdown based on markdown lint (#603) by @michaelbrewer
• docs(feature-flags): correct link and json examples (#605) by @michaelbrewer

🐛 Bug and hot fixes

• fix(api-gateway): strip stage name from request path (#622) by @michaelbrewer

🔧 Maintenance

• feat(idempotency): support for any synchronous function (#625) by @heitorlessa
• chore(deps): bump boto3 from 1.18.25 to 1.18.26 (#627) by @dependabot
• feat(general): support for Python 3.9 (#626) by @heitorlessa
• chore(deps): bump boto3 from 1.18.24 to 1.18.25 (#623) by @dependabot
• chore(api-docs): enable allow_reuse to fix the docs (#612) by @michaelbrewer
• chore(deps): bump boto3 from 1.18.22 to 1.18.24 (#619) by @dependabot
• refactor(event_ha...

v1.19.0

Summary

This release highlights 1/ a brand new Feature Flags utility, 2/ auto-disable Tracer in non-Lambda environments to ease unit testing, 3/ API Gateway event handler now supports a custom JSON serializer, and a number of documentation improvements & bugfixes.

We hope you enjoy this new utility as much as we did working on it!!

New Feature Flags utility in Beta

Special thanks to: @risenberg-cyberark, @michaelbrewer, @pcolazurdo and @am29d

You might have heard of feature flags when:

• Looking to conditionally enable a feature in your application for your customers
• A/B testing a new feature for a subset of your customers
• Working with trunk-based development where a feature might not be available right now
• Working on short-lived features that will only be enabled for select customers

This new utility makes this entire process so much easier by fetching feature flags configuration from AWS AppConfig, and evaluating contextual values against rules you defined to decide whether a feature should be enabled.

Let's dive into the code to better understand what this all means.

Evaluating whether a customer should have access to premium features

from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="features"
)

feature_flags = FeatureFlags(store=app_config)

def lambda_handler(event, context):
    # Get customer's tier from incoming request
    ctx = { "tier": event.get("tier", "standard") }

    has_premium_features: bool = feature_flags.evaluate(name="premium_features",
                                                        context=ctx, default=False)
    if has_premium_features:
        # enable premium features
        ...

Sample feature flag configuration in AWS AppConfig

{
  "premium_features": {
    "default": false,
    "rules": {
      "customer tier equals premium": {
        "when_match": true,
        "conditions": [
          {
            "action": "EQUALS",
            "key": "tier",
            "value": "premium"
          }
        ]
      }
    }
  },
  "ten_percent_off_campaign": {
    "default": false
  }
}

Notice we have premium_features flag that will conditionally be available for premium customers, and a static feature flag named ten_percent_off_campaign that is disabled by default.

Sample invocation event for this function

{
    "username": "lessa",
    "tier": "premium",
    "basked_id": "random_id"
}

There's a LOT going on here. Allow me to break it down:

1. We're defining a feature flag configuration that is stored in AWS AppConfig
2. We initialize an AppConfigStore using AWS AppConfig values created via Infrastructure as code (available on docs)
3. We initialize FeatureFlags and use our previously instantiated AppConfigStore
4. We call evaluate method and pass the name of the premium feature, along with our contextual information our rules should run against, and a sentinel value to be used in case service errors happen
5. Feature flags go through the rules defined in premium_features and evaluate whether tier key has the value premium
6. FeatureFlags returns True which is then stored as has_premium_features variable

But what if you have multiple feature flags and only want all enabled features?

We've got you covered! You can use get_enabled_features to make a single call to AWS AppConfig and return a list of all enabled features at once.

from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app = ApiGatewayResolver()

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="features"
)

feature_flags = FeatureFlags(store=app_config)


@app.get("/products")
def list_products():
    ctx = {
        **app.current_event.headers,
        **app.current_event.json_body
    }

    # all_features is evaluated to ["geo_customer_campaign", "ten_percent_off_campaign"]
    all_features: list[str] = feature_flags.get_enabled_features(context=ctx)

    if "geo_customer_campaign" in all_features:
        # apply discounts based on geo
        ...

    if "ten_percent_off_campaign" in all_features:
        # apply additional 10% for all customers
        ...

def lambda_handler(event, context):
    return app.resolve(event, context)

But hang on, why Beta?

We want to hear from you on the UX and evaluate how we can make it easier for you to bring your own feature flag store such as Redis, HashiCorp Consul, etc.

When would you use feature flags vs env variables vs Parameters utility?

Environment variables. For when you need simple configuration that will rarely if ever change, because changing it requires a Lambda function deployment.

Parameters utility. For when you need access to secrets, or fetch parameters in different formats from AWS System Manager Parameter Store or Amazon DynamoDB.

Feature flags utility. For when you need static or feature flags that will be enable conditionally based on the input and on a set of rules per feature whether that applies for all customers or on a per customer basis.

In both Parameters and Feature Flags utility you can change their config without having to change your application code.

Changes

Changes

🌟New features and non-breaking changes

• feat(tracer): auto-disable tracer when for non-Lambda envs (#598) by @michaelbrewer
• feat(feature-flags): Add not_in action and rename contains to in (#589) by @risenberg-cyberark
• refactor(feature-flags): add debug for all features evaluation" (#590) by @heitorlessa
• refactor(feature-flags): optimize UX and maintenance (#563) by @heitorlessa
• feat(api-gateway): add support for custom serializer (#568) by @michaelbrewer
• feat(params): expose high level max_age, raise_on_transform_error (#567) by @michaelbrewer
• feat(data-classes): decode json_body if based64 encoded (#560) by @michaelbrewer

📜 Documentation updates

• fix(feature-toggles): correct cdk example (#601) by @michaelbrewer
• docs(parameters): auto-transforming values based on suffix (#573) by @dreamorosi
• docs(feature-flags): create concrete documentation (#594) by @am29d
• refactor(feature-flags): optimize UX and maintenance (#563) by @heitorlessa
• docs(readme): add code coverage badge (#577) by @michaelbrewer

🐛 Bug and hot fixes

• fix(feature-flags): bug handling multiple conditions (#599) by @risenberg-cyberark
• fix(parser): apigw wss validation check_message_id; housekeeping (#553) by @michaelbrewer

🔧 Maintenance

• chore(deps): bump boto3 from 1.18.15 to 1.18.17 (#597) by @dependabot
• chore(deps-dev): bump mkdocs-material from 7.2.2 to 7.2.3 (#596) by @dependabot
• chore(deps): bump boto3 from 1.18.1 to 1.18.15 (#591) by @dependabot
• chore(deps): bump codecov/codecov-action from 2.0.1 to 2.0.2 (#558) by @dependabot
• fix(deps): bump poetry to latest (#592) by @michaelbrewer
• chore(deps-dev): bump mkdocs-material from 7.2.1 to 7.2.2 (#582) by @dependabot
• refactor(feature-flags): add debug for all features evaluation" (#590) by @heitorlessa
• chore(deps-dev): bump pdoc3 from 0.9.2 to 0.10.0 (#584) by @dependabot
• docs(feature-flags): correct docs and typing (#588) by @michaelbrewer
• refactor(feature-flags): optimize UX and maintenance (#563) by @heitorlessa
• chore(deps-dev): bump isort from 5.9.2 to 5.9.3 (#574) by @dependabot
• chore(deps-dev): bump mkdocs-material from 7.2.0 to 7.2.1 (#566) by @dependabot
• chore(deps-dev): bump mkdocs-material from 7.1.11 to 7.2.0 (#551) by @dependabot
• chore(deps-dev): bump flake8-black from 0.2.1 to 0.2.3 (#541) by @dependabot

This release was made possible by the following contributors:

@am29d, @dependabot, @dependabot[bot], @dreamorosi, @heitorlessa, @michaelbrewer, @risenberg-cyberark and @pcolazurdo

v1.18.1

Summary

Fix a pesky regression introduced when fixing API Gateway/ALB Event Handler routing regex in 1.18.0. This bug made numeric and safe URI chars to return 404 -- Big thanks to @moretension for raising it

Changes

🐛 Bug and hot fixes

• fix(api-gateway): route regression for non-word and unsafe URI chars (#556) by @heitorlessa

This release was made possible by the following contributors:

@heitorlessa

v1.18.0

Summary

This release mainly focused on bug fixes and a few minor features, so we can spend time documenting a new utility (Feature Toggles) that will be fully available in 1.19.0.

Bug fixes were largely on MyPy errors (~600 to 38) across the entire code base. We also fixed a a) faulty greedy regex in the API Gateway event handler when dealing with long and dynamic routes (more in Changes section), b) Parser authorization and version fields for API Gateway that should've been optional, and c) Event Source Data Classes to conform with AppSync Scalar by including milliseconds in the time resolution.

We also had two new first-time contributors: 👏 @walmsles and @whardier, we appreciate your help with this release!

Changes

New get_correlation_id method in Logger

You can now retrieve the latest correlation ID previously set in the Logger instance at any part of the code base. This is useful when propagating correlation ID for external calls whether these are AWS service integrations or 3rd party endpoints.

from aws_lambda_powertools import Logger

logger = Logger(service="payment")

@logger.inject_lambda_context(correlation_id_path="headers.my_request_id_header")
def handler(event, context):
    logger.debug(f"NEW Correlation ID => {logger.get_correlation_id()}")

Debug mode and HTTP service errors in Event Handlers

You can now easily enable debug mode for API Gateway or ALB event handlers. Additionally, we've made it easier to raise quick HTTP service errors in response to malformed requests, resources not found, etc.

from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
from aws_lambda_powertools.event_handler.exceptions import (
    BadRequestError,
    InternalServerError,
    NotFoundError,
    ServiceError,
    UnauthorizedError,
)

app = ApiGatewayResolver(debug=True)


@app.get(rule="/bad-request-error")
def bad_request_error():
    # HTTP  400
    raise BadRequestError("Missing required parameter")

@app.get(rule="/unauthorized-error")
def unauthorized_error():
    # HTTP 401
    raise UnauthorizedError("Unauthorized")

@app.get(rule="/not-found-error")
def not_found_error():
    # HTTP 404
    raise NotFoundError

@app.get(rule="/internal-server-error")
def internal_server_error():
    # HTTP 500
    raise InternalServerError("Internal server error")

@app.get(rule="/service-error", cors=True)
def service_error():
    raise ServiceError(502, "Something went wrong!")
    # alternatively
    # from http import HTTPStatus
    # raise ServiceError(HTTPStatus.BAD_GATEWAY.value, "Something went wrong)

def handler(event, context):
    return app.resolve(event, context)

Data model sub-classing in AppSync event handler

When building data-driven APIs using GraphQL and AppSync, you might have a set of reusable methods you want closer to the data model. Event Handler for AppSync supports a new parameter data_model to facilitate that.

You can now subclass AppSyncResolverEvent from Event Source Data Classes while handling incoming requests with Event Handler for AppSync.

  from aws_lambda_powertools import Logger, Tracer

  from aws_lambda_powertools.logging import correlation_paths
  from aws_lambda_powertools.event_handler import AppSyncResolver

  tracer = Tracer(service="sample_resolver")
  logger = Logger(service="sample_resolver")
  app = AppSyncResolver()


  class MyCustomModel(AppSyncResolverEvent):
      @property
      def country_viewer(self) -> str:
          return self.request_headers.get("cloudfront-viewer-country")

  @app.resolver(field_name="listLocations")
  @app.resolver(field_name="locations")
  def get_locations(name: str, description: str = ""):
      if app.current_event.country_viewer == "US":
        ...
      return name + description

  @logger.inject_lambda_context(correlation_id_path=correlation_paths.APPSYNC_RESOLVER)
  @tracer.capture_lambda_handler
  def lambda_handler(event, context):
      return app.resolve(event, context, data_model=MyCustomModel)

Bugfix API Gateway routing params

This fixes a regex bug that used a greedy pattern ending with incorrect path resolution, as any path after a pattern would be included in the argument.

Excerpt:

@app.get("/accounts/<account_id>")
def get_account(account_id: str):
    print(f"Account ID ({account_id}) would be 123")

# Greedy regex would inject the incorrect function parameter
@app.get("/accounts/<account_id>/source_networks")
def get_account_networks(account_id: str):
    print(f"Account ID ({account_id}) would be 123/source_networks")

In this example, say we have a GET request as /accounts/123 and another as /accounts/123/source_networks, we'd have the following effect prior to this fix:

Function	Regex	Effective account_id value
get_account	r'^/accounts/(?P<account_id>.+)$'	123
get_account_networks	r'^/accounts/(?P<account_id>.+)/source_networks$'	123/source_networks

With this fix, account_id parameter would be 123 in both occasions due to word boundary not being non-greedy. This also allows an arbitrary number of dynamic route paths and static route paths.

Function	Regex	Effective account_id value
get_account	r'^/accounts/(?P<account_id>\\w+\\b)$'	123
get_account_networks	r'^/accounts/(?P<account_id>\\w+\\b)/source_networks$'	123

🌟New features and non-breaking changes

• feat(appsync): Support AppSyncResolverEvent subclassing (#526) by @whardier
• feat(logger): add get_correlation_id method (#516) by @michaelbrewer
• feat(feat-toggle): new simple feature toggles rule engine (WIP) (#494) by @risenberg-cyberark
• feat(api-gateway): add debug mode (#507) by @michaelbrewer
• feat(api-gateway): add common HTTP service errors (#506) by @michaelbrewer

📜 Documentation updates

• docs(api-gateway): new HTTP service error exceptions (#546) by @heitorlessa
• docs(logger): new get_correlation_id method (#545) by @heitorlessa
• feat(appsync): Support AppSyncResolverEvent subclassing (#526) by @whardier

🐛 Bug and hot fixes

• fix(api-gateway): non-greedy route pattern regex (#533) by @heitorlessa
• fix(tracer): mypy generic to preserve decorated method signature (#529) by @heitorlessa
• fix(data-classes): include milliseconds in scalar types (#504) by @michaelbrewer
• fix(parser): Make ApiGateway version, authorizer fields optional (#532) by @walmsles
• fix(mypy): addresses lack of optional types (#521) by @michaelbrewer

🔧 Maintenance

• chore: bump 1.18.0 (#547) by @heitorlessa
• chore(deps-dev): bump mkdocs-material from 7.1.10 to 7.1.11 (#542) by @dependabot
• chore(deps): bump codecov/codecov-action from 1 to 2.0.1 (#539) by @dependabot
• refactor(feature-toggles): code coverage and housekeeping (#530) by @michaelbrewer
• chore(deps): bump boto3 from 1.18.0 to 1.18.1 (#528) by @dependabot
• chore(deps): bump boto3 from 1.17.110 to 1.18.0 (#527) by @dependabot
• chore(deps-dev): bump mkdocs-material from 7.1.9 to 7.1.10 (#522) by @dependabot
• chore(deps): bump boto3 from 1.17.102 to 1.17.110 (#523) by @dependabot
• chore(deps-dev): bump isort from 5.9.1 to 5.9.2 (#514) by @dependabot
• chore(mypy): add mypy support to makefile (#508) by @michaelbrewer

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer, @risenberg-cyberark, @walmsles and @whardier

v1.17.1

Summary

This patch release fixes JSON Schema Validation utility when a built-in format date-time is used but previously unrecognized.

Additionally, this includes Dark Mode (🕶️) to the Documentation, Serverless Framework and CDK examples for installing Lambda Layers from SAR, and clarifications on logger and auto-capture tracer's feature.

Changes

📜 Documentation updates

• docs(logger): add FAQ for cross-account searches (#501) by @heitorlessa
• docs: add Layers example for Serverless framework & CDK (#500) by @heitorlessa
• docs(tracer): additional scenario when to disable auto-capture (#499) by @heitorlessa
• docs: enable dark mode switch (#471) by @michaelbrewer

🐛 Bug and hot fixes

• fix(validator): handle built-in custom formats correctly (#498) by @heitorlessa

🔧 Maintenance

• chore: bump 1.17.1 (#502) by @heitorlessa
• chore(deps): bump boto3 from 1.17.101 to 1.17.102 (#493) by @dependabot
• chore(deps-dev): bump flake8-eradicate from 1.0.0 to 1.1.0 (#492) by @dependabot
• chore(deps-dev): bump isort from 5.8.0 to 5.9.1 (#487) by @dependabot
• chore(deps): bump boto3 from 1.17.91 to 1.17.101 (#490) by @dependabot
• chore(deps): bump email-validator from 1.1.2 to 1.1.3 (#478) by @dependabot
• chore(deps-dev): bump mkdocs-material from 7.1.7 to 7.1.9 (#491) by @dependabot
• chore(deps): bump boto3 from 1.17.89 to 1.17.91 (#473) by @dependabot

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa and @michaelbrewer

v1.17.0

Summary

This release brings a 1/ new handy decorator event_source for data classes, 2/ API Gateway HTTP API support for Parser, 3/ a new parameter to reset logging state upon every Lambda invocation in Logger, and 4/ high level methods to fetch attribute values and their types when using DynamoDBStreamEvent data class.

Event source decorator

Similarly to Parser's parse decorator, you can now use event_source decorator to instantiate a data class as part of the invocation more easily.

from typing import Dict, Any

from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEventV2, event_source
from aws_lambda_powertools.utilities.typing import LambdaContext


@event_source(data_class=APIGatewayProxyEventV2)
def lambda_handler(event: APIGatewayProxyEventV2, context: LambdaContext) -> Dict[str, Any]:
    assert event.get_header_value("x-foo") == "Foo"

This also gives you the ability to combine other utilities like Idempotency which expects a dictionary to serialize/deserialize without losing the benefits of self-documented schema and high level data methods.

from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEventV2, event_source
from aws_lambda_powertools.utilities.typing import LambdaContext
from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer, idempotent
)

persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")

@event_source(data_class=APIGatewayProxyEventV2)  
@idempotent(persistence_store=persistence_layer)
def lambda_handler(event: APIGatewayProxyEventV2, context):
    assert isinstance(event, APIGatewayProxyEventV2)

API Gateway HTTP API Parser support

Parser now supports API Gateway HTTP API as a native model in addition to REST API.

from typing import Dict, Any

from aws_lambda_powertools.utilities.parser import envelopes, event_parser
from aws_lambda_powertools.utilities.parser.models import APIGatewayProxyEventV2Model


@event_parser(model=APIGatewayProxyEventV2Model)
def lambda_handler(event: APIGatewayProxyEventV2Model, context: LambdaContext) -> Dict[str, Any]:
    ...

Like REST API, Parser also supports an envelope for HTTP API to help you extract your model from the payload, providing deep data validation, type enforcement at runtime, and more.

from typing import Dict, Any

from aws_lambda_powertools.utilities.parser import envelopes, event_parser
from aws_lambda_powertools.utilities.parser.models import APIGatewayProxyEventV2Model
from aws_lambda_powertools.utilities.parser.pydantic import EmailStr, UUID4, Json


class UserModel(BaseModel):
    id: UUID4
    username: str
    preferences: Json
    email: EmailStr


@event_parser(model=APIGatewayProxyEventV2Model, envelope=envelopes.ApiGatewayV2Envelope)
def lambda_handler(event: UserModel, context: LambdaContext) -> Dict[str, Any]:
    assert event.username == "ran_isenberg"

Resetting Logger state on every invocation

As Logger is initialized in the global scope, logging state can persist across invocations. It is a good practice to always append new keys and use default values (None if unavailable), so Logger can update and remove keys on a per invocation basis.

However, there are times where you might add keys conditionally depending on the incoming event. For these type of use cases, you can now use clear_state parameter when using inject_lambda_context decorator.

from aws_lambda_powertools import Logger

logger = Logger(service="payment")

@logger.inject_lambda_context(clear_state=True)
def handler(event, context):
    if event.get("special_key"):
        # Should only be available in the first request log
        # as the second request doesn't contain `special_key`
        logger.append_keys(debugging_key="value")

    logger.info("Collecting payment")

DynamoDB Stream Event AttributeValueType and AttributeValue

You can now introspect what DynamoDB Record type and value is more easily with AttributeValueType and AttributeValue. This is useful when you might want to deserialize or deal with certain records and their attributes differently based on their type, e.g. a map vs string.

Each DynamoDB key available in the stream record now has high level methods such as get_value and get_type.

from aws_lambda_powertools.utilities.data_classes import event_source, DynamoDBStreamEvent
from aws_lambda_powertools.utilities.data_classes.dynamo_db_stream_event import AttributeValueType, AttributeValue
from aws_lambda_powertools.utilities.typing import LambdaContext


@event_source(data_class=DynamoDBStreamEvent)
def lambda_handler(event: DynamoDBStreamEvent, context: LambdaContext):
    for record in event.records:
        key: AttributeValue = record.dynamodb.keys["id"]
        if key == AttributeValueType.Number:
            # {"N": "123.45"} => "123.45"
            assert key.get_value == key.n_value
            print(key.get_value)
        elif key == AttributeValueType.Map:
            assert key.get_value == key.map_value
            print(key.get_value)

You can find their respective implementations and all contributors below.

Changes

🌟New features and non-breaking changes

• feat(logger): add option to clear state per invocation (#467) by @heitorlessa
• feat(data-classes): add AttributeValueType to DynamoDBStreamEvent (#462) by @michaelbrewer
• feat(data-classes): decorator to instantiate data_classes and docs updates (#442) by @michaelbrewer
• feat(parser): add support for API Gateway HTTP API #434 (#441) by @risenberg-cyberark

📜 Documentation updates

• feat(logger): add option to clear state per invocation (#467) by @heitorlessa
• feat(data-classes): decorator to instantiate data_classes and docs updates (#442) by @michaelbrewer
• docs: include new public roadmap (#452) by @heitorlessa
• feat(parser): add support for API Gateway HTTP API #434 (#441) by @risenberg-cyberark

🔧 Maintenance

• chore(deps): bump boto3 from 1.17.88 to 1.17.89 (#466) by @dependabot
• chore(deps): bump boto3 from 1.17.87 to 1.17.88 (#463) by @dependabot
• chore(deps-dev): bump mkdocs-material from 7.1.6 to 7.1.7 (#464) by @dependabot
• chore(deps): bump boto3 from 1.17.86 to 1.17.87 (#459) by @dependabot
• chore(deps): bump boto3 from 1.17.85 to 1.17.86 (#458) by @dependabot
• chore(deps): bump boto3 from 1.17.84 to 1.17.85 (#455) by @dependabot
• chore(deps-dev): bump pytest-cov from 2.12.0 to 2.12.1 (#454) by @dependabot
• chore: update mergify to require approval on dependabot (#456) by @heitorlessa
• chore: bump actions/setup-python from 1 to 2.2.2 (#445) by @dependabot
• chore: bump xenon from 0.7.1 to 0.7.3 (#446) by @dependabot
• chore: bump boto3 from 1.17.78 to 1.17.84 (#449) by @dependabot
• chore: bump mkdocs-material from 7.1.5 to 7.1.6 (#451) by @dependabot
• build(deps-dev): bump pytest-asyncio from 0.14.0 to 0.15.1 (#448) by @dependabot
• chore: enable mergify (#450) by @heitorlessa
• chore: enable dependabot for dep upgrades (#444) by @heitorlessa
• chore: assited changelog pre-generation, auto-label PR (#443) by @heitorlessa

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer and @risenberg-cyberark

v1.16.1

Summary

Emergency release to patch Pydantic against CVE-2021-29510used by Parser utility - This causes resource starvation when parsing infinity/-inf values.

Thanks to @risenberg-cyberark for the quick turnaround.

Changes

• feat(parser): security issue in Pydantic #436 (#437) by @risenberg-cyberark

This release was made possible by the following contributors:

@risenberg-cyberark