[SOLVED] Kong 3.9.0 Declarative (db-less) startup throws error in ECS Fargate Docker container due to missing resty/ada library

[iAmAdamReid]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.9.0

Current Behavior

Summary

Hello, I'm encountering this issue when attempting to deploy a declarative db-less instance of Kong v3.9.0 to ECS Fargate. Container initialization throws an error due to missing resty/ada library. This is possibly due to the recent addition of ada-url as a dependency in #13120

Docker image is built using --platform linux/amd64 and debian:bookworm-slim as the base image. The kong_3.9.0_amd64.deb file is sourced from Cloudsmith. Docker entrypoint script is sourced from GitHub

Error Message

[error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/resty/ada/lib.lua:225: 
unable to load ada library - 
please make sure that it can be found in package.cpath or system library path

Stack Trace

nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/resty/ada/lib.lua:225: unable to load ada library - please make sure that it can be found in package.cpath or system library path
--
stack traceback:
[C]: in function 'error'
/usr/local/share/lua/5.1/resty/ada/lib.lua:225: in main chunk
[C]: in function 'require'
/usr/local/share/lua/5.1/resty/ada/search.lua:9: in main chunk
[C]: in function 'require'
/usr/local/share/lua/5.1/kong/pdk/service/request.lua:26: in main chunk
[C]: in function 'require'
/usr/local/share/lua/5.1/kong/pdk/init.lua:263: in function 'new'
/usr/local/share/lua/5.1/kong/global.lua:168: in function 'init_pdk'
/usr/local/share/lua/5.1/kong/init.lua:655: in function 'init'
init_by_lua(nginx-kong.conf:45):3: in main chunk

Resolution Attempts

All resolution attempts were made using a barebones kong.yaml file:

_format_version: '3.0'
_transform: true

No Effect

• use a barebones config with KONG_PLUGINS=bundled to rule out any other plugins that might be pulling ada as a dependency
• use a barebones config with KONG_PLUGINS=off to turn off all plugins
• modify KONG_LUA_PACKAGE_PATH to try and resolve potential path issue
• modify KONG_LUA_PACKAGE_CPATH to try and resolve potential cpath issue

The source code of the file that's throwing the error: https://github.com/bungle/lua-resty-ada/blob/main/lib/resty/ada/lib.lua

Right now I'm seeing if I can build ada-url from source in the Dockerfile and make it discoverable via LUA_CPATH. If you know any other solution, please let me know.

Expected Behavior

When starting a Docker container using the official Kong 3.9.0 image, it should be able to resolve all necessary runtime dependencies without additional configuration or manual installation.

Steps To Reproduce

1. Follow the instructions to build your own Docker image
2. Push the Docker image to AWS ECR repository
3. Attempt to run the Docker image in ECS Fargate
4. Task will fail with log output: [error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/resty/ada/lib.lua:225: unable to load ada library - please make sure that it can be found in package.cpath or system library path

Anything else?

No response

[ProBrian]

Hi @iAmAdamReid,
I've tried with your steps to build a kong image, but not running in AWS ECS, just on my local environment, and it works fine.
Could you try to run the image on some not ECS environment, e.g. your local machine or some VM, so that we could figure out if this is an image issue of just some environment specific issue?

[iAmAdamReid]

@ProBrian OK, I think I figured this out.

I tried again locally with the barebones config, and was able to run without issue. This confirmed it was something in the ECS environment. The culprit ended up being Elastic File System.

I am using EFS to share the kong.conf and kong.yaml files across containers. EFS was configured to mount at /usr/local/kong which ended up borking something with the CPATH resolution. Moving all config files into /usr/local/kong/config and mounting EFS to that directory appears to have resolved the issue.

Closing this as solved!