Bug: HTTP API with router event handler fails on whitespace in path parameter #1098

sthuber90 · 2022-04-08T08:14:24Z

Expected Behaviour

Calling GET /pets/{name} or GET /pets/{name}/vets should return a successful response with status code 200 when called with "Chester Mall" or "Chester%20Mall" (without ") as name. API Gateway will already decode encoded whitespaces as you can see in the example events.

GET /pets/Fiffi

  {
      "version": "2.0",
      "routeKey": "GET /pets/{name}",
      "rawPath": "/pets/Fiffi",
      "rawQueryString": "",
      "headers": {
          "accept": "*/*",
          "accept-encoding": "gzip, deflate, br",
          "cache-control": "no-cache",
          "content-length": "0",
          "x-forwarded-port": "443",
          "x-forwarded-proto": "https"
      },
      "requestContext": {
          "http": {
              "method": "GET",
              "path": "/pets/Fiffi",
              "protocol": "HTTP/1.1"
          },
          "requestId": "QQCtFipSIAMEPNQ=",
          "routeKey": "GET /pets/{name}",
          "stage": "$default",
          "time": "08/Apr/2022:07:31:05 +0000",
          "timeEpoch": 1649403065999
      },
      "pathParameters": {
          "name": "Fiffi"
      },
      "isBase64Encoded": false
  }

GET /pets/Fiffi/vets

  {
          "version": "2.0",
          "routeKey": "GET /pets/{name}/vets",
          "rawPath": "/pets/Fiffi/vets",
          "rawQueryString": "",
          "headers": {
              "accept": "*/*",
              "accept-encoding": "gzip, deflate, br",
              "cache-control": "no-cache",
              "content-length": "0",
              "x-forwarded-port": "443",
              "x-forwarded-proto": "https"
          },
          "requestContext": {
              "http": {
                  "method": "GET",
                  "path": "/pets/Fiffi/vets",
                  "protocol": "HTTP/1.1"
              },
              "requestId": "QQGmWiEcoAMEPqA=",
              "routeKey": "GET /pets/{name}/vets",
              "stage": "$default",
              "time": "08/Apr/2022:07:57:41 +0000",
              "timeEpoch": 1649404661221
          },
          "pathParameters": {
              "name": "Fiffi"
          },
          "isBase64Encoded": false
      }

GET /pets/Chester Mall

{
      "version": "2.0",
      "routeKey": "GET /pets/{name}",
      "rawPath": "/pets/Chester Mall",
      "rawQueryString": "",
      "headers": {
          "accept": "*/*",
          "accept-encoding": "gzip, deflate, br",
          "cache-control": "no-cache",
          "content-length": "0",
          "x-forwarded-port": "443",
          "x-forwarded-proto": "https"
      },
      "requestContext": {
          "http": {
              "method": "GET",
              "path": "/pets/Chester Mall",
              "protocol": "HTTP/1.1"
          },
          "requestId": "QQCu1ioiIAMEPlg=",
          "routeKey": "GET /pets/{name}",
          "stage": "$default",
          "time": "08/Apr/2022:07:31:17 +0000",
          "timeEpoch": 1649403077140
      },
      "pathParameters": {
          "name": "Chester Mall"
      },
      "isBase64Encoded": false
  }

GET /pets/Chester Mall/vets

 {
      "version": "2.0",
      "routeKey": "GET /pets/{name}/vets",
      "rawPath": "/pets/Chester Mall/vets",
      "rawQueryString": "",
      "headers": {
          "accept": "*/*",
          "accept-encoding": "gzip, deflate, br",
          "cache-control": "no-cache",
          "content-length": "0"
          "x-forwarded-port": "443",
          "x-forwarded-proto": "https"
      },
      "requestContext": {
          "http": {
              "method": "GET",
              "path": "/pets/Chester Mall/vets",
              "protocol": "HTTP/1.1"
          },
          "requestId": "QQGlpij8oAMEPig=",
          "routeKey": "GET /pets/{name}/vets",
          "stage": "$default",
          "time": "08/Apr/2022:07:57:36 +0000",
          "timeEpoch": 1649404656715
      },
      "pathParameters": {
          "name": "Chester Mall"
      },
      "isBase64Encoded": false
  }

Current Behaviour

Instead of a successful response a 404 "Not Found" is returned, with response body

{
    "statusCode": 404,
    "message": "Not found"
}

Code snippet

import aws_lambda_powertools.event_handler.api_gateway
from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayHttpResolver
from aws_lambda_powertools.logging import correlation_paths

logger = Logger()
app = APIGatewayHttpResolver()

# Add to code to handle whitespaces in path parameters
# aws_lambda_powertools.event_handler.api_gateway._UNSAFE_URI = "%<> \[\]{}|^"
# aws_lambda_powertools.event_handler.api_gateway._NAMED_GROUP_BOUNDARY_PATTERN = rf"(?P\1[{aws_lambda_powertools.event_handler.api_gateway._SAFE_URI}{aws_lambda_powertools.event_handler.api_gateway._UNSAFE_URI}\\w]+)"


@app.get("/pets/<name>")
def get_petname(name):
    return {"name": name}


@app.get("/pets/<name>/vets")
def get_petname(name):
    return {"pet_name": name, "vets": []}


@logger.inject_lambda_context(
    correlation_id_path=correlation_paths.API_GATEWAY_HTTP, log_event=True
)
def lambda_handler(event, context):
    return app.resolve(event, context)

Possible Solution

Change the _UNSAFE_URI constant in aws_lambda_powertools/event_handler/api_gateway from:

_UNSAFE_URI = "%<>\[\]{}|^"

to

# whitespace between <> and \[\]
_UNSAFE_URI = "%<> \[\]{}|^"

Steps to Reproduce

Deploy reproducible example (https://github.com/sthuber90/aws-python-http-api-project)
Call /pets/{name} and /pets/{name}/vets endpoint with different names

AWS Lambda Powertools for Python version

latest (layer version 16)

AWS Lambda function runtime

{"label"=>"3.9"}

Packaging format used

{"label"=>"Lambda Layers"}

Debugging logs

timestamp,message
1649404657592,"START RequestId: 46aafc05-0539-4035-83cb-cbd4268db189 Version: $LATEST
"
1649404657592,"2022-04-08 07:57:37,592 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /pets/<name> and methods: GET
"
1649404657593,"2022-04-08 07:57:37,592 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /pets/<name>/vets and methods: GET
"
1649404657593,"2022-04-08 07:57:37,593 aws_lambda_powertools.logging.logger [DEBUG] Decorator called with parameters
"
1649404657596,"2022-04-08 07:57:37,595 aws_lambda_powertools.logging.logger [DEBUG] Event received
"
1649404657596,"{""level"":""INFO"",""location"":""decorate:352"",""message"":{""version"":""2.0"",""routeKey"":""GET /pets/{name}/vets"",""rawPath"":""/pets/Chester Mall/vets"",""rawQueryString"":"""",""headers"":{""accept"":""*/*"",""accept-encoding"":""gzip, deflate, br"",""cache-control"":""no-cache"",""content-length"":""0"",""host"":""938e36howc.execute-api.us-east-1.amazonaws.com"",""postman-token"":""d44c4268-96ea-4e76-a308-ac9f2b0eb952"",""user-agent"":""PostmanRuntime/7.29.0"",""x-amzn-trace-id"":""Root=1-624feaf0-4b992fda408853251e402d93"",""x-forwarded-for"":""95.208.248.101"",""x-forwarded-port"":""443"",""x-forwarded-proto"":""https""},""requestContext"":{""http"":{""method"":""GET"",""path"":""/pets/Chester Mall/vets"",""protocol"":""HTTP/1.1"",""userAgent"":""PostmanRuntime/7.29.0""},""requestId"":""QQGlpij8oAMEPig="",""routeKey"":""GET /pets/{name}/vets"",""stage"":""$default"",""time"":""08/Apr/2022:07:57:36 +0000"",""timeEpoch"":1649404656715},""pathParameters"":{""name"":""Chester Mall""},""isBase64Encoded"":false},""timestamp"":""2022-04-08 07:57:37,596+0000"",""service"":""service_undefined"",""cold_start"":true,""function_name"":""aws-python-http-api-project-dev-hello"",""function_memory_size"":""1024"",""function_arn"":""arn:aws:lambda:us-east-1:373270804851:function:aws-python-http-api-project-dev-hello"",""function_request_id"":""46aafc05-0539-4035-83cb-cbd4268db189"",""correlation_id"":""QQGlpij8oAMEPig="",""xray_trace_id"":""1-624feaf0-1f7281e9141a9e970179de6f""}
"
1649404657596,"2022-04-08 07:57:37,596 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
"
1649404657596,"2022-04-08 07:57:37,596 aws_lambda_powertools.event_handler.api_gateway [DEBUG] No match found for path /pets/Chester Mall/vets and method GET
"
1649404657597,"END RequestId: 46aafc05-0539-4035-83cb-cbd4268db189
"
1649404657597,"REPORT RequestId: 46aafc05-0539-4035-83cb-cbd4268db189	Duration: 2.12 ms	Billed Duration: 3 ms	Memory Size: 1024 MB	Max Memory Used: 57 MB	Init Duration: 531.20 ms	
"
1649404661256,"START RequestId: 23baf67a-158c-4246-a950-4eea00efba7d Version: $LATEST
"
1649404661259,"2022-04-08 07:57:41,259 aws_lambda_powertools.logging.logger [DEBUG] Event received
"
1649404661259,"{""level"":""INFO"",""location"":""decorate:352"",""message"":{""version"":""2.0"",""routeKey"":""GET /pets/{name}/vets"",""rawPath"":""/pets/Fiffi/vets"",""rawQueryString"":"""",""headers"":{""accept"":""*/*"",""accept-encoding"":""gzip, deflate, br"",""cache-control"":""no-cache"",""content-length"":""0"",""host"":""938e36howc.execute-api.us-east-1.amazonaws.com"",""postman-token"":""7d1e6998-1851-44fc-b029-279c7cbce941"",""user-agent"":""PostmanRuntime/7.29.0"",""x-amzn-trace-id"":""Root=1-624feaf5-1cf0e266526a59a700b13d68"",""x-forwarded-for"":""95.208.248.101"",""x-forwarded-port"":""443"",""x-forwarded-proto"":""https""},""requestContext"":{""http"":{""method"":""GET"",""path"":""/pets/Fiffi/vets"",""protocol"":""HTTP/1.1"",""userAgent"":""PostmanRuntime/7.29.0""},""requestId"":""QQGmWiEcoAMEPqA="",""routeKey"":""GET /pets/{name}/vets"",""stage"":""$default"",""time"":""08/Apr/2022:07:57:41 +0000"",""timeEpoch"":1649404661221},""pathParameters"":{""name"":""Fiffi""},""isBase64Encoded"":false},""timestamp"":""2022-04-08 07:57:41,259+0000"",""service"":""service_undefined"",""cold_start"":false,""function_name"":""aws-python-http-api-project-dev-hello"",""function_memory_size"":""1024"",""function_arn"":""arn:aws:lambda:us-east-1:373270804851:function:aws-python-http-api-project-dev-hello"",""function_request_id"":""23baf67a-158c-4246-a950-4eea00efba7d"",""correlation_id"":""QQGmWiEcoAMEPqA="",""xray_trace_id"":""1-624feaf5-44c0297c56144d6e0eb23a2f""}
"
1649404661260,"2022-04-08 07:57:41,260 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
"
1649404661260,"2022-04-08 07:57:41,260 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
"
1649404661260,"2022-04-08 07:57:41,260 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Simple response detected, serializing return before constructing final response
"
1649404661260,"END RequestId: 23baf67a-158c-4246-a950-4eea00efba7d
"
1649404661260,"REPORT RequestId: 23baf67a-158c-4246-a950-4eea00efba7d	Duration: 1.85 ms	Billed Duration: 2 ms	Memory Size: 1024 MB	Max Memory Used: 57 MB	
"

The text was updated successfully, but these errors were encountered:

boring-cyborg · 2022-04-08T08:14:27Z

Thanks for opening your first issue here! We'll come back to you as soon as we can.

sthulb · 2022-04-08T09:20:46Z

I'll look into this today.

michaelbrewer · 2022-04-08T14:02:39Z

Awesome thanks for the detailed logs and examples. Should be easy enough to make a patch for this.

sthuber90 · 2022-04-08T14:04:28Z

@michaelbrewer PR was already merged by @heitorlessa. I'll leave it up to you to guys to close the issue. Hope that's alright

michaelbrewer · 2022-04-08T14:31:19Z

haha, @sthuber90 i just woke up and was catching up. Sorry about the bug and thanks for fixing it! When i originally created the handler i missed these test cases.

sthuber90 · 2022-04-08T14:33:06Z

No worries. Thank you for this great project and the awesome support ❤️

sthulb · 2022-04-08T15:08:56Z

@sthuber90 thanks for the contribution.

@michaelbrewer thanks for trying to shepherd this, but the project maintainers have already validated it and merged it

github-actions · 2022-04-29T15:39:39Z

This is now released under 1.25.10 version!

sthuber90 added bug Something isn't working triage Pending triage from maintainers labels Apr 8, 2022

sthulb self-assigned this Apr 8, 2022

sthuber90 mentioned this issue Apr 8, 2022

fix(api_gateway): allow whitespace in routes' path parameter #1099

Merged

7 tasks

sthulb added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/event_handlers and removed triage Pending triage from maintainers labels Apr 8, 2022

heitorlessa added the pending-release Fix or implementation already in dev waiting to be released label Apr 8, 2022

sthulb closed this as completed Apr 8, 2022

github-actions bot removed the pending-release Fix or implementation already in dev waiting to be released label Apr 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: HTTP API with router event handler fails on whitespace in path parameter #1098

Bug: HTTP API with router event handler fails on whitespace in path parameter #1098

sthuber90 commented Apr 8, 2022

boring-cyborg bot commented Apr 8, 2022

sthulb commented Apr 8, 2022

michaelbrewer commented Apr 8, 2022

sthuber90 commented Apr 8, 2022

michaelbrewer commented Apr 8, 2022

sthuber90 commented Apr 8, 2022

sthulb commented Apr 8, 2022

github-actions bot commented Apr 29, 2022

Bug: HTTP API with router event handler fails on whitespace in path parameter #1098

Bug: HTTP API with router event handler fails on whitespace in path parameter #1098

Comments

sthuber90 commented Apr 8, 2022

Expected Behaviour

Current Behaviour

Code snippet

Possible Solution

Steps to Reproduce

AWS Lambda Powertools for Python version

AWS Lambda function runtime

Packaging format used

Debugging logs

boring-cyborg bot commented Apr 8, 2022

sthulb commented Apr 8, 2022

michaelbrewer commented Apr 8, 2022

sthuber90 commented Apr 8, 2022

michaelbrewer commented Apr 8, 2022

sthuber90 commented Apr 8, 2022

sthulb commented Apr 8, 2022

github-actions bot commented Apr 29, 2022