Skip to content

Support structured JSON logging for errors #533

Open
@bensie

Description

@bensie

AWS released advanced logging controls for Lambda.

On the whole, it works great with the slog package in Go 1.21 on the provided.2023 runtime for logging from Go functions. However, aws-lambda-go handles the logging of function errors, which are written in their own format that was decided on long before AWS released structured logging options.

When JSON logging is enabled on a function, any unstructured log entry will be automatically reshaped to JSON and will set the log level to INFO. Function errors handled by aws-lambda-go have this problem, which means that the only way to see function errors in the logs is to set the application log level to INFO.

Here's the complete code that demonstrates the following.

In the CloudFormation for a function, we'll set the LoggingConfig and (for now) we'll set ApplicationLogLevel to INFO.

    LoggingConfig:
      ApplicationLogLevel: INFO
      LogFormat: JSON
      SystemLogLevel: INFO

Given the following code, we expect the usual Lambda system log entries, plus an application-generated info message, warn message, error message, and the panic log entry.

package main

import (
	"context"
	"fmt"
	"log/slog"
	"os"

	"github.com/aws/aws-lambda-go/lambda"
	"github.com/aws/aws-lambda-go/lambdacontext"
)

func main() {
	lambda.StartHandlerFunc(handlerWithLambdaLogging(handler))
}

func handler(ctx context.Context, event any) (any, error) {
	slog.DebugContext(ctx, "debug message", "event", event)
	slog.InfoContext(ctx, "info message", "event", event)
	slog.WarnContext(ctx, "warn message", "event", event)
	slog.ErrorContext(ctx, "error message", "event", event)

	a := []string{"hey"}
	fmt.Println(a[1]) // panic

	return nil, nil
}

func handlerWithLambdaLogging[E, R any](handler func(context.Context, E) (R, error)) func(context.Context, E) (R, error) {
	var level slog.Level
	switch os.Getenv("AWS_LAMBDA_LOG_LEVEL") {
	case "DEBUG":
		level = slog.LevelDebug
	case "INFO":
		level = slog.LevelInfo
	case "WARN":
		level = slog.LevelWarn
	case "ERROR":
		level = slog.LevelError
	default:
		level = slog.LevelInfo
	}

	return func(ctx context.Context, event E) (R, error) {
		lc, _ := lambdacontext.FromContext(ctx)
		logHandler := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: level})).With("requestId", lc.AwsRequestID)
		slog.SetDefault(logHandler)

		return handler(ctx, event)
	}
}

This works as expected, but notice that the level for the errors are INFO.

CleanShot 2023-11-17 at 11 29 26@2x

Now we'll update the LoggingConfig to set the application log level to ERROR (and we'll set the system level to warn to quiet things down).

    LoggingConfig:
      ApplicationLogLevel: ERROR
      LogFormat: JSON
      SystemLogLevel: WARN

Re-run the function and take a look at the CloudWatch Logs. Since the error output logging is getting shoved into INFO level, it's not shown at all. We know that the function errored from the platform-level log, but won't be able to see what happened.

CleanShot 2023-11-17 at 11 33 45@2x

In addition to the AWS_LAMBDA_LOG_LEVEL env var demonstrated above, Lambda now also exposes AWS_LAMBDA_LOG_FORMAT, which could be used here in aws-lambda-go to determine whether or not to write the log entry with a predefined JSON structure without being a breaking change to this library.

In summary, it'd be great to be able to use ERROR level logging in Lambda applications and be able to see errors from aws-lambda-go. Thank you!

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions