From 64f7aa8ad66503cc86b1f377355b4ebdb24cfd7e Mon Sep 17 00:00:00 2001 From: Heitor Lessa Date: Sun, 23 Aug 2020 17:00:55 +0200 Subject: [PATCH 1/8] docs: move concurrent asynchronous under escape hatch Signed-off-by: heitorlessa --- docs/content/core/tracer.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/core/tracer.mdx b/docs/content/core/tracer.mdx index 36a0cded9f0..4ebdc478d97 100644 --- a/docs/content/core/tracer.mdx +++ b/docs/content/core/tracer.mdx @@ -199,7 +199,7 @@ You can use `tracer.provider` attribute to access all methods provided by AWS X- This is useful when you need a feature available in X-Ray that is not available in the Tracer utility, for example [thread-safe](https://github.com/aws/aws-xray-sdk-python/#user-content-trace-threadpoolexecutor), or [context managers](https://github.com/aws/aws-xray-sdk-python/#user-content-start-a-custom-segmentsubsegment). -## Concurrent asynchronous functions +### Concurrent asynchronous functions As of now, X-Ray SDK will raise an exception when async functions are run and traced concurrently. From 3faf915b54d6e646e3939b88995e3a2254be5824 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Sun, 23 Aug 2020 19:01:18 +0200 Subject: [PATCH 2/8] docs: create Patching modules section; cleanup response wording Signed-off-by: heitorlessa --- docs/content/core/tracer.mdx | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/docs/content/core/tracer.mdx b/docs/content/core/tracer.mdx index 4ebdc478d97..a01cf8fb083 100644 --- a/docs/content/core/tracer.mdx +++ b/docs/content/core/tracer.mdx @@ -58,7 +58,7 @@ You can trace your Lambda function handler via `capture_lambda_handler`. When using this decorator, Tracer performs these additional tasks to ease operations: * Creates a `ColdStart` annotation to easily filter traces that have had an initialization overhead -* Adds any response, or full exceptions generated by the handler as metadata +* Captures any response, or full exceptions generated by the handler, and include as tracing metadata ```python:title=lambda_handler.py from aws_lambda_powertools import Tracer @@ -70,6 +70,8 @@ def handler(event, context): payment = collect_payment(charge_id) ... +# Disables Tracer from capturing response and adding as metadata +# Useful when dealing with sensitive data @tracer.capture_lambda_handler(capture_response=False) # highlight-line def handler(event, context): return "sensitive_information" @@ -170,7 +172,23 @@ def handler(evt, ctx): # highlight-line another_result = list(collect_payment_gen()) ``` -## Tracing aiohttp requests +## Patching modules + +Tracer automatically patches all [supported libraries by X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-patching.html) during initialization, by default. Underneath, AWS X-Ray SDK checks whether a supported library has been imported before patching. + +If you're looking to shave a few microseconds, or milliseconds depending on your function memory configuration, you can patch specific modules using `patch_modules` param: + +```python:title=app.py +import boto3 +import requests + +from aws_lambda_powertools import Tracer + +modules_to_be_patched = ["boto3", "requests"] +tracer = Tracer(patch_modules=modules_to_be_patched) # highlight-line +``` + +### Tracing aiohttp requests This snippet assumes you have aiohttp as a dependency From 10cb3a98bcf01abb0e120d871508e698b7fefc23 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Sun, 23 Aug 2020 19:22:11 +0200 Subject: [PATCH 3/8] docs: make sensitive info more explicit with an example Signed-off-by: heitorlessa --- docs/content/core/tracer.mdx | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/content/core/tracer.mdx b/docs/content/core/tracer.mdx index a01cf8fb083..4042d51a861 100644 --- a/docs/content/core/tracer.mdx +++ b/docs/content/core/tracer.mdx @@ -17,12 +17,6 @@ Tracer is an opinionated thin wrapper for [AWS X-Ray Python SDK](https://github. * Support tracing async methods, generators, and context managers * Auto patch supported modules, or a tuple of explicit modules supported by AWS X-Ray - - Returning sensitive information from your Lambda handler or functions, where Tracer is used? -

- You can disable Tracer from capturing their responses as tracing metadata with capture_response=False parameter in both capture_lambda_handler and capture_method decorators. -
- ## Initialization Your AWS Lambda function must have permission to send traces to AWS X-Ray - Here is an example using AWS Serverless Application Model (SAM) @@ -69,7 +63,15 @@ def handler(event, context): charge_id = event.get('charge_id') payment = collect_payment(charge_id) ... +``` + + + Returning sensitive information from your Lambda handler or functions, where Tracer is used? +

+ You can disable Tracer from capturing their responses as tracing metadata with capture_response=False parameter in both capture_lambda_handler and capture_method decorators. +
+```python:title=do_not_capture_response_as_metadata.py # Disables Tracer from capturing response and adding as metadata # Useful when dealing with sensitive data @tracer.capture_lambda_handler(capture_response=False) # highlight-line From 44db087cbd1605d3ca55a01a1f7e979d6a20d720 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Sun, 23 Aug 2020 19:23:29 +0200 Subject: [PATCH 4/8] docs: fix typos, log_event & sampling wording Signed-off-by: heitorlessa --- docs/content/core/logger.mdx | 37 ++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/docs/content/core/logger.mdx b/docs/content/core/logger.mdx index 225fb401cf1..618d7a78435 100644 --- a/docs/content/core/logger.mdx +++ b/docs/content/core/logger.mdx @@ -58,7 +58,7 @@ Key | Type | Example | Description **sampling_rate** | int | 0.1 | Debug logging sampling rate in percentage e.g. 1% in this case **message** | any | "Collecting payment" | Log statement value. Unserializable JSON values will be casted to string -## Capturing context Lambda info +## Capturing Lambda context info You can enrich your structured logs with key Lambda context information via `inject_lambda_context`. @@ -79,22 +79,6 @@ def handler(event, context): ... ``` -You can also explicitly log any incoming event using `log_event` param or via `POWERTOOLS_LOGGER_LOG_EVENT` env var. - - - This is disabled by default to prevent sensitive info being logged. -
- -```python:title=log_handler_event.py -from aws_lambda_powertools import Logger - -logger = Logger() - -@logger.inject_lambda_context(log_event=True) # highlight-start -def handler(event, context): - ... -``` - When used, this will include the following keys: Key | Type | Example @@ -145,6 +129,23 @@ Key | Type | Example } ``` +
+ +You can also explicitly log any incoming event using `log_event` param or via `POWERTOOLS_LOGGER_LOG_EVENT` env var. + + + This is disabled by default to prevent sensitive info being logged. +
+ +```python:title=log_handler_event.py +from aws_lambda_powertools import Logger + +logger = Logger() + +@logger.inject_lambda_context(log_event=True) # highlight-line +def handler(event, context): + ... +``` ## Appending additional keys @@ -222,7 +223,7 @@ If you ever forget to use `child` param, we will return an existing `Logger` wit You can dynamically set a percentage of your logs to **DEBUG** level using `sample_rate` param or via env var `POWERTOOLS_LOGGER_SAMPLE_RATE`. -This happens on an entire request basis, and DEBUG level is set at the constructor. That means, concurrent requests or infrequent invocations are more likely to occur as [new Lambda execution contexts are created](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html), not reused. +Sampling calculation happens at the Logger class initialization. This means, when configured it, sampling it's more likely to happen during concurrent requests, or infrequent invocations as [new Lambda execution contexts are created](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html), not reused. If you want this logic to happen on every invocation regardless whether Lambda reuses the execution environment or not, then create your Logger inside your Lambda handler. From 705371e99474be60b3f2f93479bb511ad98e5362 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Sun, 23 Aug 2020 19:46:34 +0200 Subject: [PATCH 5/8] docs: subtle rewording for better clarity Signed-off-by: heitorlessa --- docs/content/core/metrics.mdx | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/content/core/metrics.mdx b/docs/content/core/metrics.mdx index 696074c6812..9b750c28622 100644 --- a/docs/content/core/metrics.mdx +++ b/docs/content/core/metrics.mdx @@ -5,7 +5,9 @@ description: Core utility import Note from "../../src/components/Note" -Metrics creates custom metrics asynchronously via logging metrics to standard output following Amazon CloudWatch Embedded Metric Format (EMF). +Metrics creates custom metrics asynchronously by logging metrics to standard output following Amazon CloudWatch Embedded Metric Format (EMF). + +These metrics can be visualized through [Amazon CloudWatch Console](https://console.aws.amazon.com/cloudwatch/). **Key features** @@ -32,7 +34,9 @@ Resources: ``` We recommend you use your application or main service as a metric namespace. -You can explicitly set a namespace name via `namespace` param or via `POWERTOOLS_METRICS_NAMESPACE` env var. This sets **namespace** key that will be used for all metrics. +You can explicitly set a namespace name via `namespace` param or via `POWERTOOLS_METRICS_NAMESPACE` env var. + +This sets **namespace** key that will be used for all metrics. You can also pass a service name via `service` param or `POWERTOOLS_SERVICE_NAME` env var. This will create a dimension with the service name. ```python:title=app.py @@ -67,7 +71,7 @@ metrics.add_metric(name="SuccessfulBooking", unit=MetricUnit.Count, value=1) `MetricUnit` enum facilitate finding a supported metric unit by CloudWatch. Alternatively, you can pass the value as a string if you already know them e.g. "Count". -CloudWatch EMF supports a max of 100 metrics. Metrics will automatically flush all metrics when adding the 100th metric, where subsequent metrics will be aggregated into a new EMF object. +CloudWatch EMF supports a max of 100 metrics. Metrics utility will flush all metrics when adding the 100th metric while subsequent metrics will be aggregated into a new EMF object, for your convenience. ## Creating a metric with a different dimension From ac0aba9cd9786cb53578ba2c07fa8a14ae6c942b Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Sun, 23 Aug 2020 19:47:00 +0200 Subject: [PATCH 6/8] docs: add blog post, and quick example Signed-off-by: heitorlessa --- docs/content/index.mdx | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/content/index.mdx b/docs/content/index.mdx index e5c2688ecc7..be10589f270 100644 --- a/docs/content/index.mdx +++ b/docs/content/index.mdx @@ -3,8 +3,15 @@ title: Homepage description: AWS Lambda Powertools Python --- +import Note from "../src/components/Note" + Powertools is a suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier. + + Looking for a quick run through of the core utilities?

+ Check out this detailed blog post with a practical example. +
+ ## Install Powertools is available in PyPi. You can use your favourite dependency management tool to install it @@ -12,6 +19,11 @@ Powertools is available in PyPi. You can use your favourite dependency managemen * [poetry](https://python-poetry.org/): `poetry add aws-lambda-powertools` * [pip](https://pip.pypa.io/en/latest/index.html): `pip install aws-lambda-powertools` +**Quick hello world example using SAM CLI** +```bash:title=hello_world.sh +sam init --location https://github.com/aws-samples/cookiecutter-aws-sam-python +``` + ## Features * [Tracing](./core/tracer) - Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions From e89fbea5689bbe2390d892984ad6ba77fe2b739f Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Sun, 23 Aug 2020 19:53:17 +0200 Subject: [PATCH 7/8] docs: use table for clarity Signed-off-by: heitorlessa --- docs/content/index.mdx | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/content/index.mdx b/docs/content/index.mdx index be10589f270..412cee68701 100644 --- a/docs/content/index.mdx +++ b/docs/content/index.mdx @@ -26,11 +26,13 @@ sam init --location https://github.com/aws-samples/cookiecutter-aws-sam-python ## Features -* [Tracing](./core/tracer) - Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions -* [Logging](./core/logger) - Structured logging made easier, and decorator to enrich structured logging with key Lambda context details -* [Metrics](./core/metrics) - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) -* [Bring your own middleware](./utilities/middleware_factory) - Decorator factory to create your own middleware to run logic before, and after each Lambda invocation -* [Parameters utility](./utilities/parameters) - Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time +Utility | Description +------------------------------------------------- | --------------------------------------------------------------------------------- +[Tracing](./core/tracer) | Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions +[Logging](./core/logger) | Structured logging made easier, and decorator to enrich structured logging with key Lambda context details +[Metrics](./core/metrics) | Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) +[Bring your own middleware](.//utilities/middleware_factory) | Decorator factory to create your own middleware to run logic before, and after each Lambda invocation +[Parameters utility](./utilities/parameters) | Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time ## Tenets @@ -49,10 +51,10 @@ _`*` Core utilities are Tracer, Logger and Metrics. Optional utilities may vary Environment variable | Description | Utility ------------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- -**POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | all +**POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | All **POWERTOOLS_METRICS_NAMESPACE** | Sets namespace used for metrics | [Metrics](./core/metrics) **POWERTOOLS_TRACE_DISABLED** | Disables tracing | [Tracing](./core/tracer) -**POWERTOOLS_TRACE_MIDDLEWARES** | Creates sub-segment for each custom middleware | [middleware_factory](./utilities/middleware_factory) +**POWERTOOLS_TRACE_MIDDLEWARES** | Creates sub-segment for each custom middleware | [Middleware factory](./utilities/middleware_factory) **POWERTOOLS_LOGGER_LOG_EVENT** | Logs incoming event | [Logging](./core/logger) **POWERTOOLS_LOGGER_SAMPLE_RATE** | Debug log sampling | [Logging](./core/logger) **LOG_LEVEL** | Sets logging level | [Logging](./core/logger) From efd38d823e707864a7c4e8e986cc414524f3a817 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Sun, 23 Aug 2020 19:56:10 +0200 Subject: [PATCH 8/8] docs: move tenets; remove extra space Signed-off-by: heitorlessa --- docs/content/index.mdx | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/content/index.mdx b/docs/content/index.mdx index 412cee68701..02b2846688d 100644 --- a/docs/content/index.mdx +++ b/docs/content/index.mdx @@ -10,7 +10,7 @@ Powertools is a suite of utilities for AWS Lambda Functions that makes tracing w Looking for a quick run through of the core utilities?

Check out this detailed blog post with a practical example. -
+ ## Install @@ -34,17 +34,6 @@ Utility | Description [Bring your own middleware](.//utilities/middleware_factory) | Decorator factory to create your own middleware to run logic before, and after each Lambda invocation [Parameters utility](./utilities/parameters) | Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time -## Tenets - -* **AWS Lambda only** – We optimise for AWS Lambda function environments and supported runtimes only. Utilities might work with web frameworks and non-Lambda environments, though they are not officially supported. -* **Eases the adoption of best practices** – The main priority of the utilities is to facilitate best practices adoption, as defined in the AWS Well-Architected Serverless Lens; all other functionality is optional. -* **Keep it lean** – Additional dependencies are carefully considered for security and ease of maintenance, and prevent negatively impacting startup time. -* **We strive for backwards compatibility** – New features and changes should keep backwards compatibility. If a breaking change cannot be avoided, the deprecation and migration process should be clearly defined. -* **We work backwards from the community** – We aim to strike a balance of what would work best for 80% of customers. Emerging practices are considered and discussed via Requests for Comment (RFCs) -* **Idiomatic** – Utilities follow programming language idioms and language-specific best practices. - -_`*` Core utilities are Tracer, Logger and Metrics. Optional utilities may vary across languages._ - ## Environment variables **Environment variables** used across suite of utilities. @@ -68,3 +57,14 @@ from aws_lambda_powertools.logging.logger import set_package_logger set_package_logger() ``` + +## Tenets + +* **AWS Lambda only** – We optimise for AWS Lambda function environments and supported runtimes only. Utilities might work with web frameworks and non-Lambda environments, though they are not officially supported. +* **Eases the adoption of best practices** – The main priority of the utilities is to facilitate best practices adoption, as defined in the AWS Well-Architected Serverless Lens; all other functionality is optional. +* **Keep it lean** – Additional dependencies are carefully considered for security and ease of maintenance, and prevent negatively impacting startup time. +* **We strive for backwards compatibility** – New features and changes should keep backwards compatibility. If a breaking change cannot be avoided, the deprecation and migration process should be clearly defined. +* **We work backwards from the community** – We aim to strike a balance of what would work best for 80% of customers. Emerging practices are considered and discussed via Requests for Comment (RFCs) +* **Idiomatic** – Utilities follow programming language idioms and language-specific best practices. + +_`*` Core utilities are Tracer, Logger and Metrics. Optional utilities may vary across languages._