Bug: Response compression doesn't work on 2.28 lib version

[dacianf]

Expected Behaviour

Once compress flag is set to true on either Response object or in the router annotation the lambda should compress the response and return it as an encoded base 64 string.

Current Behaviour

It throws an error saying: a bytes-like object is required, not 'list'

Code snippet

from typing import List

import requests
from pydantic import BaseModel, Field

from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.utilities.typing import LambdaContext

tracer = Tracer()
logger = Logger()
app = APIGatewayRestResolver(debug=True)



class Todo(BaseModel):
    userId: int
    id_: int = Field(alias="id")
    title: str
    completed: bool


@app.get("/todos", compress=True)
def get_todos() -> List[Todo]:
    todo = requests.get("https://jsonplaceholder.typicode.com/todos")
    todo.raise_for_status()

    return todo.json()


def lambda_handler(event: dict, context: LambdaContext) -> dict:
    return app.resolve(event, context)

Possible Solution

No response

Steps to Reproduce

1. Use any API Request (either REST or HTML) you have that returns a json and add compress=True on the router. (if you want to return a Response as an object, you can do that too; add the compress flag on the response object as well)
2. Create a test event to call that route and include "Accept-Encoding": "gzip, deflate, br" in the request header
3. Call the function with that event

Powertools for AWS Lambda (Python) version

2.28.0

AWS Lambda function runtime

3.11

Packaging format used

Lambda Layers

Debugging logs

Function Logs
ogger to suppress child logger records to bubble up
2023-11-27 21:57:35,739 aws_lambda_powertools.logging.logger [DEBUG] Adding filter in root logger to suppress child logger records to bubble up
2023-11-27 21:57:35,739 aws_lambda_powertools.logging.logger [DEBUG] Marking logger service_undefined as preconfigured
[DEBUG]	2023-11-27T21:57:35.739Z		Marking logger service_undefined as preconfigured
2023-11-27 21:57:35,741 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /todos and methods: GET
[DEBUG]	2023-11-27T21:57:35.741Z		Adding route using rule /todos and methods: GET
START RequestId: 4091d393-77ec-4e22-841d-f42423544ed8 Version: $LATEST
{"version":"1.0","resource":"/todos","path":"/todos","httpMethod":"GET","headers":{"Origin":"https://aws.amazon.com","Accept-Encoding":"gzip, deflate, br"},"multiValueHeaders":{},"queryStringParameters":{},"multiValueQueryStringParameters":{},"requestContext":{"accountId":"123456789012","apiId":"id","authorizer":{"claims":null,"scopes":null},"domainName":"id.execute-api.us-east-1.amazonaws.com","domainPrefix":"id","extendedRequestId":"request-id","httpMethod":"GET","path":"/todos","protocol":"HTTP/1.1","requestId":"id=","requestTime":"04/Mar/2020:19:15:17 +0000","requestTimeEpoch":1583349317135,"resourceId":null,"resourcePath":"/todos/1","stage":"$default"},"pathParameters":null,"stageVariables":null,"body":"","isBase64Encoded":false}
2023-11-27 21:57:35,744 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway REST API contract
[DEBUG]	2023-11-27T21:57:35.744Z	4091d393-77ec-4e22-841d-f42423544ed8	Converting event to API Gateway REST API contract
2023-11-27 21:57:35,744 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
[DEBUG]	2023-11-27T21:57:35.744Z	4091d393-77ec-4e22-841d-f42423544ed8	Found a registered route. Calling function
2023-11-27 21:57:35,744 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Building middleware stack: []
[DEBUG]	2023-11-27T21:57:35.744Z	4091d393-77ec-4e22-841d-f42423544ed8	Building middleware stack: []
Processing Route:::get_todos (/todos)
Middleware Stack:
=================
_registered_api_adapter
=================
2023-11-27 21:57:35,744 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [_registered_api_adapter] next call chain is _registered_api_adapter -> get_todos
[DEBUG]	2023-11-27T21:57:35.744Z	4091d393-77ec-4e22-841d-f42423544ed8	MiddlewareFrame: [_registered_api_adapter] next call chain is _registered_api_adapter -> get_todos
2023-11-27 21:57:35,745 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Calling API Route Handler: {}
[DEBUG]	2023-11-27T21:57:35.745Z	4091d393-77ec-4e22-841d-f42423544ed8	Calling API Route Handler: {}
[DEBUG]	2023-11-27T21:57:36.107Z	4091d393-77ec-4e22-841d-f42423544ed8	Simple response detected, serializing return before constructing final response
2023-11-27 21:57:36,107 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Simple response detected, serializing return before constructing final response
[ERROR] TypeError: a bytes-like object is required, not 'list'
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 32, in lambda_handler
    return app.resolve(event, context)
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 1736, in resolve
    response = self._resolve().build(self.current_event, self._cors)
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 792, in build
    self._route(event, cors)
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 788, in _route
    self._compress()
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 773, in _compress
    self.response.body = gzip.compress(self.response.body) + gzip.flush()END RequestId: 4091d393-77ec-4e22-841d-f42423544ed8

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[heitorlessa]

hey @dacianf you're missing enable_validation=True parameter at the constructor, otherwise Event Handler won't understand how to handle a Pydantic model.

Could you try that and let us know, please? :)

[heitorlessa]

Docs just in case: https://docs.powertools.aws.dev/lambda/python/latest/core/event_handler/api_gateway/#data-validation

[dacianf]

With or without that flag, the lambda returns the same error.

Code:

from typing import List

import requests
from pydantic import BaseModel, Field

from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.utilities.typing import LambdaContext

tracer = Tracer()
logger = Logger()
app = APIGatewayRestResolver(enable_validation=True, debug=True)



class Todo(BaseModel):
    userId: int
    id_: int = Field(alias="id")
    title: str
    completed: bool


@app.get("/todos", compress=True)
def get_todos() -> List[Todo]:
    todo = requests.get("https://jsonplaceholder.typicode.com/todos")
    todo.raise_for_status()

    return todo.json()


def lambda_handler(event: dict, context: LambdaContext) -> dict:
    return app.resolve(event, context)

Logs:

Test Event Name
test_post

Response
{
  "errorMessage": "a bytes-like object is required, not 'list'",
  "errorType": "TypeError",
  "requestId": "4b16bbe7-9cdb-40d8-a2a9-186d3489cc70",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 32, in lambda_handler\n    return app.resolve(event, context)\n",
    "  File \"/opt/python/aws_lambda_powertools/event_handler/api_gateway.py\", line 1736, in resolve\n    response = self._resolve().build(self.current_event, self._cors)\n",
    "  File \"/opt/python/aws_lambda_powertools/event_handler/api_gateway.py\", line 792, in build\n    self._route(event, cors)\n",
    "  File \"/opt/python/aws_lambda_powertools/event_handler/api_gateway.py\", line 788, in _route\n    self._compress()\n",
    "  File \"/opt/python/aws_lambda_powertools/event_handler/api_gateway.py\", line 773, in _compress\n    self.response.body = gzip.compress(self.response.body) + gzip.flush()\n"
  ]
}

Function Logs
eryStringParameters":{},"requestContext":{"accountId":"123456789012","apiId":"id","authorizer":{"claims":null,"scopes":null},"domainName":"id.execute-api.us-east-1.amazonaws.com","domainPrefix":"id","extendedRequestId":"request-id","httpMethod":"GET","path":"/todos","protocol":"HTTP/1.1","requestId":"id=","requestTime":"04/Mar/2020:19:15:17 +0000","requestTimeEpoch":1583349317135,"resourceId":null,"resourcePath":"/todos/1","stage":"$default"},"pathParameters":null,"stageVariables":null,"body":"","isBase64Encoded":false}
2023-11-27 23:18:21,410 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway REST API contract
[DEBUG]	2023-11-27T23:18:21.410Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	Converting event to API Gateway REST API contract
2023-11-27 23:18:21,411 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
[DEBUG]	2023-11-27T23:18:21.411Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	Found a registered route. Calling function
2023-11-27 23:18:21,411 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Building middleware stack: [<aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIValidationMiddleware object at 0x7fa0757450>]
[DEBUG]	2023-11-27T23:18:21.411Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	Building middleware stack: [<aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIValidationMiddleware object at 0x7fa0757450>]
Processing Route:::get_todos (/todos)
Middleware Stack:
=================
OpenAPIValidationMiddleware
_registered_api_adapter
=================
2023-11-27 23:18:21,411 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIValidationMiddleware] next call chain is OpenAPIValidationMiddleware -> _registered_api_adapter
[DEBUG]	2023-11-27T23:18:21.411Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	MiddlewareFrame: [OpenAPIValidationMiddleware] next call chain is OpenAPIValidationMiddleware -> _registered_api_adapter
2023-11-27 23:18:21,411 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIValidationMiddleware handler
[DEBUG]	2023-11-27T23:18:21.411Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	OpenAPIValidationMiddleware handler
[DEBUG]	2023-11-27T23:18:21.415Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	MiddlewareFrame: [_registered_api_adapter] next call chain is _registered_api_adapter -> get_todos
2023-11-27 23:18:21,415 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [_registered_api_adapter] next call chain is _registered_api_adapter -> get_todos
2023-11-27 23:18:21,415 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Calling API Route Handler: {}
[DEBUG]	2023-11-27T23:18:21.415Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	Calling API Route Handler: {}
2023-11-27 23:18:21,835 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Simple response detected, serializing return before constructing final response
[DEBUG]	2023-11-27T23:18:21.835Z	4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	Simple response detected, serializing return before constructing final response
[ERROR] TypeError: a bytes-like object is required, not 'list'
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 32, in lambda_handler
    return app.resolve(event, context)
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 1736, in resolve
    response = self._resolve().build(self.current_event, self._cors)
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 792, in build
    self._route(event, cors)
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 788, in _route
    self._compress()
  File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 773, in _compress
    self.response.body = gzip.compress(self.response.body) + gzip.flush()END RequestId: 4b16bbe7-9cdb-40d8-a2a9-186d3489cc70
REPORT RequestId: 4b16bbe7-9cdb-40d8-a2a9-186d3489cc70	Duration: 589.01 ms	Billed Duration: 590 ms	Memory Size: 128 MB	Max Memory Used: 74 MB	Init Duration: 848.86 ms

Request ID
4b16bbe7-9cdb-40d8-a2a9-186d3489cc70

[rubenfonseca]

Thank you for opening this! We found the problem and we're going to patch it in the next few minutes.