Skip to content

[lldb] Use the Python limited API with SWIG 4.2 or later #153119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 13, 2025

Conversation

JDevlieghere
Copy link
Member

Use the Python limited API when building with SWIG 4.2 or later.

Use the Python limited API when building with SWIG 4.2 or later.
@llvmbot
Copy link
Member

llvmbot commented Aug 12, 2025

@llvm/pr-subscribers-lldb

Author: Jonas Devlieghere (JDevlieghere)

Changes

Use the Python limited API when building with SWIG 4.2 or later.


Full diff: https://github.com/llvm/llvm-project/pull/153119.diff

1 Files Affected:

  • (modified) lldb/cmake/modules/LLDBConfig.cmake (+8-1)
diff --git a/lldb/cmake/modules/LLDBConfig.cmake b/lldb/cmake/modules/LLDBConfig.cmake
index a9679d63e010d..ca574a142b9dd 100644
--- a/lldb/cmake/modules/LLDBConfig.cmake
+++ b/lldb/cmake/modules/LLDBConfig.cmake
@@ -68,7 +68,6 @@ add_optional_dependency(LLDB_ENABLE_FBSDVMCORE "Enable libfbsdvmcore support in
 option(LLDB_USE_ENTITLEMENTS "When codesigning, use entitlements if available" ON)
 option(LLDB_BUILD_FRAMEWORK "Build LLDB.framework (Darwin only)" OFF)
 option(LLDB_ENABLE_PROTOCOL_SERVERS "Enable protocol servers (e.g. MCP) in LLDB" ON)
-option(LLDB_ENABLE_PYTHON_LIMITED_API "Force LLDB to only use the Python Limited API (requires SWIG 4.2 or later)" OFF)
 option(LLDB_NO_INSTALL_DEFAULT_RPATH "Disable default RPATH settings in binaries" OFF)
 option(LLDB_USE_SYSTEM_DEBUGSERVER "Use the system's debugserver for testing (Darwin only)." OFF)
 option(LLDB_SKIP_STRIP "Whether to skip stripping of binaries when installing lldb." OFF)
@@ -173,6 +172,14 @@ if (LLDB_ENABLE_PYTHON)
     "Embed PYTHONHOME in the binary. If set to OFF, PYTHONHOME environment variable will be used to to locate Python."
     ${default_embed_python_home})
 
+  if (SWIG_VERSION VERSION_GREATER_EQUAL "4.2")
+    set(default_enable_python_limited_api ON)
+  else()
+    set(default_enable_python_limited_api OFF)
+  endif()
+  option(LLDB_ENABLE_PYTHON_LIMITED_API "Force LLDB to only use the Python Limited API (requires SWIG 4.2 or later)"
+    ${default_enable_python_limited_api})
+
   include_directories(${Python3_INCLUDE_DIRS})
   if (LLDB_EMBED_PYTHON_HOME)
     get_filename_component(PYTHON_HOME "${Python3_EXECUTABLE}" DIRECTORY)

@JDevlieghere JDevlieghere merged commit c14ca45 into llvm:main Aug 13, 2025
11 checks passed
@JDevlieghere JDevlieghere deleted the use-limited-api branch August 13, 2025 00:51
@llvm-ci
Copy link
Collaborator

llvm-ci commented Aug 13, 2025

LLVM Buildbot has detected a new failure on builder lldb-x86_64-debian running on lldb-x86_64-debian while building lldb at step 4 "build".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/162/builds/28799

Here is the relevant piece of the build log for the reference
Step 4 (build) failure: build (failure)
...
449.180 [1511/72/5264] Building CXX object tools/lldb/source/Plugins/SymbolVendor/ELF/CMakeFiles/lldbPluginSymbolVendorELF.dir/SymbolVendorELF.cpp.o
449.360 [1510/72/5265] Building CXX object tools/lldb/source/Plugins/Platform/AIX/CMakeFiles/lldbPluginPlatformAIX.dir/PlatformAIX.cpp.o
449.412 [1509/72/5266] Building CXX object tools/lldb/source/Plugins/SymbolVendor/wasm/CMakeFiles/lldbPluginSymbolVendorWasm.dir/SymbolVendorWasm.cpp.o
449.460 [1508/72/5267] Building CXX object tools/lldb/source/Plugins/Process/scripted/CMakeFiles/lldbPluginScriptedProcess.dir/ScriptedProcess.cpp.o
449.498 [1507/72/5268] Building CXX object tools/lldb/source/Plugins/SymbolVendor/PECOFF/CMakeFiles/lldbPluginSymbolVendorPECOFF.dir/SymbolVendorPECOFF.cpp.o
449.537 [1506/72/5269] Building CXX object tools/clang/lib/Index/CMakeFiles/obj.clangIndex.dir/IndexBody.cpp.o
449.595 [1505/72/5270] Building CXX object tools/lldb/source/Plugins/Platform/Android/CMakeFiles/lldbPluginPlatformAndroid.dir/PlatformAndroid.cpp.o
449.639 [1504/72/5271] Linking CXX static library lib/libclangIndex.a
449.714 [1503/72/5272] Linking CXX static library lib/libclangCrossTU.a
450.368 [1502/72/5273] Building CXX object tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o
FAILED: tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o 
/usr/bin/clang++ -DGTEST_HAS_RTTI=0 -DHAVE_ROUND -DLLDB_PYTHON_EXE_RELATIVE_PATH=\"bin/python3\" -DLLDB_PYTHON_RELATIVE_LIBDIR=\"local/lib/python3.13/dist-packages\" -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/worker/2.0.1/lldb-x86_64-debian/build/tools/lldb/source/Plugins/ScriptInterpreter/Python -I/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python -I/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/include -I/home/worker/2.0.1/lldb-x86_64-debian/build/tools/lldb/include -I/home/worker/2.0.1/lldb-x86_64-debian/build/include -I/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/llvm/include -I/usr/include/python3.13 -I/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/llvm/../clang/include -I/home/worker/2.0.1/lldb-x86_64-debian/build/tools/lldb/../clang/include -I/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source -I/home/worker/2.0.1/lldb-x86_64-debian/build/tools/lldb/source -isystem /usr/include/libxml2 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-vla-extension -O3 -DNDEBUG -std=c++17  -fno-exceptions -funwind-tables -fno-rtti -UNDEBUG -MD -MT tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o -MF tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o.d -o tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o -c /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:102:22: error: use of undeclared identifier 'PyImport_Inittab'; did you mean 'PyImport_Import'?
  102 |       for (auto *p = PyImport_Inittab; p->name != nullptr; p++) {
      |                      ^~~~~~~~~~~~~~~~
      |                      PyImport_Import
/usr/include/python3.13/import.h:78:24: note: 'PyImport_Import' declared here
   78 | PyAPI_FUNC(PyObject *) PyImport_Import(PyObject *name);
      |                        ^
/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:102:41: error: member reference base type 'PyObject *(PyObject *)' (aka '_object *(_object *)') is not a structure or union
  102 |       for (auto *p = PyImport_Inittab; p->name != nullptr; p++) {
      |                                        ~^ ~~~~
/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:102:61: error: arithmetic on a pointer to the function type 'PyObject *(PyObject *)' (aka '_object *(_object *)')
  102 |       for (auto *p = PyImport_Inittab; p->name != nullptr; p++) {
      |                                                            ~^
/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:103:21: error: member reference base type 'PyObject *(PyObject *)' (aka '_object *(_object *)') is not a structure or union
  103 |         if (strcmp(p->name, "readline") == 0) {
      |                    ~^ ~~~~
/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:104:12: error: member reference base type 'PyObject *(PyObject *)' (aka '_object *(_object *)') is not a structure or union
  104 |           p->initfunc = initlldb_readline;
      |           ~^ ~~~~~~~~
5 errors generated.
450.509 [1502/71/5274] Building CXX object tools/lldb/source/Plugins/SymbolLocator/Debuginfod/CMakeFiles/lldbPluginSymbolLocatorDebuginfod.dir/SymbolLocatorDebuginfod.cpp.o
450.536 [1502/70/5275] Building CXX object tools/lldb/source/Plugins/SymbolLocator/Default/CMakeFiles/lldbPluginSymbolLocatorDefault.dir/SymbolLocatorDefault.cpp.o
450.579 [1502/69/5276] Building CXX object tools/lldb/source/Plugins/Platform/FreeBSD/CMakeFiles/lldbPluginPlatformFreeBSD.dir/PlatformFreeBSD.cpp.o
450.586 [1502/68/5277] Building CXX object tools/lldb/source/Plugins/LanguageRuntime/CPlusPlus/ItaniumABI/CMakeFiles/lldbPluginCXXItaniumABI.dir/ItaniumABILanguageRuntime.cpp.o
450.805 [1502/67/5278] Building CXX object tools/lldb/source/Plugins/SymbolFile/Breakpad/CMakeFiles/lldbPluginSymbolFileBreakpad.dir/SymbolFileBreakpad.cpp.o
450.809 [1502/66/5279] Building CXX object tools/lldb/source/Plugins/UnwindAssembly/x86/CMakeFiles/lldbPluginUnwindAssemblyX86.dir/UnwindAssembly-x86.cpp.o
451.138 [1502/65/5280] Building CXX object tools/lldb/source/Plugins/Process/minidump/CMakeFiles/lldbPluginProcessMinidump.dir/ProcessMinidump.cpp.o
451.208 [1502/64/5281] Building CXX object lib/TextAPI/BinaryReader/CMakeFiles/LLVMTextAPIBinaryReader.dir/DylibReader.cpp.o
451.316 [1502/63/5282] Building CXX object tools/lldb/source/Plugins/TraceExporter/ctf/CMakeFiles/lldbPluginTraceExporterCTF.dir/CommandObjectThreadTraceExportCTF.cpp.o
451.353 [1502/62/5283] Building CXX object tools/lldb/source/Plugins/TraceExporter/ctf/CMakeFiles/lldbPluginTraceExporterCTF.dir/TraceExporterCTF.cpp.o
451.412 [1502/61/5284] Building CXX object tools/lldb/source/Plugins/UnwindAssembly/InstEmulation/CMakeFiles/lldbPluginUnwindAssemblyInstEmulation.dir/UnwindAssemblyInstEmulation.cpp.o
451.418 [1502/60/5285] Building CXX object tools/lldb/source/Plugins/Platform/MacOSX/CMakeFiles/lldbPluginPlatformMacOSX.dir/PlatformDarwin.cpp.o
451.672 [1502/59/5286] Building CXX object tools/clang/lib/StaticAnalyzer/Core/CMakeFiles/obj.clangStaticAnalyzerCore.dir/FunctionSummary.cpp.o
451.947 [1502/58/5287] Building CXX object tools/lldb/source/Plugins/Platform/NetBSD/CMakeFiles/lldbPluginPlatformNetBSD.dir/PlatformNetBSD.cpp.o
452.443 [1502/57/5288] Building CXX object tools/lldb/source/Plugins/StructuredData/DarwinLog/CMakeFiles/lldbPluginStructuredDataDarwinLog.dir/StructuredDataDarwinLog.cpp.o
452.455 [1502/56/5289] Building CXX object tools/lldb/source/Plugins/Platform/Windows/CMakeFiles/lldbPluginPlatformWindows.dir/PlatformWindows.cpp.o
452.486 [1502/55/5290] Building CXX object tools/lldb/source/Plugins/Protocol/MCP/CMakeFiles/lldbPluginProtocolServerMCP.dir/Resource.cpp.o

JDevlieghere added a commit that referenced this pull request Aug 13, 2025
JDevlieghere added a commit that referenced this pull request Aug 13, 2025
…53327)

Reverts #153119 because with
`LLDB_USE_LIBEDIT_READLINE_COMPAT_MODULE`, we're using
`PyImport_Inittab` which isn't part of the stable API.
llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Aug 13, 2025
… later" (#153327)

Reverts llvm/llvm-project#153119 because with
`LLDB_USE_LIBEDIT_READLINE_COMPAT_MODULE`, we're using
`PyImport_Inittab` which isn't part of the stable API.
@mstorsjo
Copy link
Member

I see that this was reverted due to PyImport_Inittab; on Windows (mingw) builds, I also ran into another issue with this change:
https://github.com/mstorsjo/llvm-mingw/actions/runs/16924804977/job/47967307644

/home/runner/work/llvm-mingw/llvm-mingw/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:119:5: error: unknown type name 'PyConfig'
  119 |     PyConfig config;
      |     ^
/home/runner/work/llvm-mingw/llvm-mingw/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:916:23: error: member access into incomplete type 'PyThreadState' (aka 'struct _ts')
  916 |       long tid = state->thread_id;
      |                       ^
/home/runner/work/llvm-mingw/llvm-mingw/install/llvm-mingw/python/include/python3.12/pytypedefs.h:24:16: note: forward declaration of '_ts'
   24 | typedef struct _ts PyThreadState;
      |                ^
2 errors generated.

If you're aiming to reland this after fixing the issue you ran into on the build bots, it'd be appreciated if you could look into this issue as well.

@JDevlieghere
Copy link
Member Author

@mstorsjo Thanks for the heads up. I added PyThreadState to #151617. The PyConfig issue will be fixed by account for LLDB_EMBED_PYTHON_HOME when picking the default for LLDB_ENABLE_PYTHON_LIMITED_API.

JDevlieghere added a commit to JDevlieghere/llvm-project that referenced this pull request Aug 13, 2025
Use the Python limited API when building with SWIG 4.2 or later.
JDevlieghere added a commit that referenced this pull request Aug 14, 2025
…153472)

Use the Python limited API when building with SWIG 4.2 or later.
mdenson pushed a commit to mdenson/llvm-project that referenced this pull request Aug 16, 2025
llvm#153472)

Use the Python limited API when building with SWIG 4.2 or later.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants