The SQLAlchemy 1.4 C extension fails to compile on Python 3.13 #11499

musicinmybrain · 2024-06-16T12:36:15Z

Describe the bug

The C extension for SQLAlchemy 1.4 uses _PyArg_NoKeywords(), an undocumented private API that was moved to the pycore_modsupport.h internal C API in Python 3.13.

As a result, the C extension fails to build on Python 3.13.

  lib/sqlalchemy/cextension/resultproxy.c: In function ‘tuplegetter_new’:
  lib/sqlalchemy/cextension/resultproxy.c:831:10: error: implicit declaration of function ‘_PyArg_NoKeywords’ [-Wimplicit-function-declaration]
    831 |     if (!_PyArg_NoKeywords("tuplegetter", kwds))
        |          ^~~~~~~~~~~~~~~~~

Optional link from https://docs.sqlalchemy.org which documents the behavior that is expected

No response

SQLAlchemy Version in Use

1.4.52 (or, rel_1_4 branch, currently at b13cdd1)

DBAPI (i.e. the database driver)

N/A

Database Vendor and Major Version

N/A

Python Version

3.13.0b1

Operating system

Fedora 40

To Reproduce

$ gh repo clone sqlalchemy/sqlalchemy
$ cd sqlalchemy
$ git checkout rel_1_4
$ python3.13 -m venv _e
(_e) $ pip install build
(_e) $ REQUIRE_SQLALCHEMY_CEXT=1 python -m build

Error

gcc -fno-strict-overflow -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -fcf-protection -fexceptions -fcf-protection -fexceptions -fcf-protection -fexceptions -O3 -fPIC -I/tmp/build-env-l1j4cq8g/include -I/usr/include/python3.13 -c lib/sqlalchemy/cextension/resultproxy.c -o build/temp.linux-x86_64-cpython-313/lib/sqlalchemy/cextension/resultproxy.o -Wundef -Werror=implicit-function-declaration
lib/sqlalchemy/cextension/resultproxy.c: In function ‘tuplegetter_new’:
lib/sqlalchemy/cextension/resultproxy.c:831:10: error: implicit declaration of function ‘_PyArg_NoKeywords’ [-Wimplicit-function-declaration]
  831 |     if (!_PyArg_NoKeywords("tuplegetter", kwds))
      |          ^~~~~~~~~~~~~~~~~
***************************************************************************
NOTE: C extension build is required because REQUIRE_SQLALCHEMY_CEXT is set, and the build has failed; will not degrade to non-C extensions
***************************************************************************
Traceback (most recent call last):
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/unixccompiler.py", line 188, in _compile
    self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
    spawn(cmd, dry_run=self.dry_run, **kwargs)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/spawn.py", line 68, in spawn
    raise DistutilsExecError(f"command {cmd!r} failed with exit code {exitcode}")
distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 74, in build_extension
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/command/build_ext.py", line 252, in build_extension
    _build_ext.build_extension(self, ext)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 560, in build_extension
    objects = self.compiler.compile(
        sources,
    ...<5 lines>...
        depends=ext.depends,
    )   
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/unixccompiler.py", line 190, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ben/Downloads/sqlalchemy/_e/lib64/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
    main()
    ~~~~^^
  File "/home/ben/Downloads/sqlalchemy/_e/lib64/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
                             ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ben/Downloads/sqlalchemy/_e/lib64/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 271, in build_wheel
    return _build_backend().build_wheel(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        wheel_directory, config_settings, metadata_directory
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/build_meta.py", line 410, in build_wheel
    return self._build_with_temp_dir(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        ['bdist_wheel'],
        ^^^^^^^^^^^^^^^^
    ...<3 lines>...
        self._arbitrary_args(config_settings),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
    self.run_setup()
    ~~~~~~~~~~~~~~^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/build_meta.py", line 487, in run_setup
    super().run_setup(setup_script=setup_script)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/build_meta.py", line 311, in run_setup
    exec(code, locals())
    ~~~~^^^^^^^^^^^^^^^^
  File "<string>", line 166, in <module>
  File "<string>", line 146, in run_setup
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/__init__.py", line 103, in setup
    return distutils.core.setup(**attrs)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/core.py", line 184, in setup
    return run_commands(dist)
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
    dist.run_commands()
    ~~~~~~~~~~~~~~~~~^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
    ~~~~~~~~~~~~~~~~^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/dist.py", line 968, in run_command
    super().run_command(command)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
    ~~~~~~~~~~~^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/wheel/bdist_wheel.py", line 368, in run
    self.run_command("build")
    ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/dist.py", line 968, in run_command
    super().run_command(command)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
    ~~~~~~~~~~~^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/command/build.py", line 132, in run
    self.run_command(cmd_name)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/dist.py", line 968, in run_command
    super().run_command(command)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
    ~~~~~~~~~~~^^
  File "<string>", line 68, in run
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/command/build_ext.py", line 91, in run
    _build_ext.run(self)
    ~~~~~~~~~~~~~~^^^^^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
    self.build_extensions()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
    self._build_extensions_serial()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/tmp/build-env-l1j4cq8g/lib64/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
    self.build_extension(ext)
    ~~~~~~~~~~~~~~~~~~~~^^^^^
  File "<string>", line 76, in build_extension
BuildFailed

ERROR Backend subprocess exited when trying to invoke build_wheel

Additional context

No response

The text was updated successfully, but these errors were encountered:

See: python/cpython#110964 Fixes: sqlalchemy#11499

CaselIT · 2024-06-16T12:41:39Z

Hi,

Thanks for reporting.
Do you have a suggested replacement function that can be used?

zzzeek · 2024-06-16T12:45:18Z

Well we might have to use version specific defines to handle this, there's already a bunch of that in the old C extensions

We probably need to set up a python version EOL for 1.4 at some point

musicinmybrain · 2024-06-16T12:46:46Z

Hi,

Thanks for reporting. Do you have a suggested replacement function that can be used?

The best suggestion I have is to just imitate its implementation, which is straightforward and uses only functions that are still in the public C API. I plant to submit a PR for that approach.

CaselIT · 2024-06-16T12:51:35Z

Thanks, that would be helpful

musicinmybrain · 2024-06-16T12:53:20Z

We probably need to set up a python version EOL for 1.4 at some point

This is probably true. On the other hand, speaking as a Fedora packager (but not a primary maintainer of any of the SQLAlchemy packages in Fedora) we still have a python-sqlalchemy1.4 compat package, and there are some significant packages that haven’t been able to migrate to SQLAlchemy 2.0 yet, so we have good reason to keep 1.4 around at least for Python 3.13, and perhaps a bit beyond.

zzzeek · 2024-06-16T13:10:55Z

oh the function was private, in that case yes we might want to just vendor it hoping it's not a very long function

zzzeek · 2024-06-16T13:11:54Z

yes we are keeping 1.4 around longer than usual for sure. but it's going to have to stop at some point, it's still supporting python 2.7 and every release of it has to work on 2.7

sqla-tester · 2024-06-20T17:18:19Z

Benjamin A. Beasley has proposed a fix for this issue in the rel_1_4 branch:

SQLAlchemy 1.4: Fix building the C extension on Python 3.13 https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/5339

Adjustments to the C extensions, which are specific to the SQLAlchemy 1.x series, to work under Python 3.13. Pull request courtesy Ben Beasley. Fixes: #11499 Closes: #11500 Pull-request: #11500 Pull-request-sha: 8a5888b Change-Id: I1943eb387f9b075bf07e179f7a24762236e234bf

zzzeek · 2024-06-23T18:18:22Z

fixed in fc66fda

musicinmybrain added the requires triage New issue that requires categorization label Jun 16, 2024

musicinmybrain added a commit to musicinmybrain/sqlalchemy that referenced this issue Jun 16, 2024

Work around _PyArg_NoKeywords moving to a private header in CPython 3.13

8a5888b

See: python/cpython#110964 Fixes: sqlalchemy#11499

CaselIT added bug Something isn't working cextensions issues with the C/cython extensions and removed requires triage New issue that requires categorization labels Jun 16, 2024

CaselIT added this to the 1.4.x milestone Jun 16, 2024

musicinmybrain mentioned this issue Jun 16, 2024

SQLAlchemy 1.4: Fix building the C extension on Python 3.13 #11500

Closed

3 tasks

zzzeek closed this as completed Jun 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The SQLAlchemy 1.4 C extension fails to compile on Python 3.13 #11499

The SQLAlchemy 1.4 C extension fails to compile on Python 3.13 #11499

musicinmybrain commented Jun 16, 2024

CaselIT commented Jun 16, 2024 •

edited

Loading

zzzeek commented Jun 16, 2024

musicinmybrain commented Jun 16, 2024

CaselIT commented Jun 16, 2024

musicinmybrain commented Jun 16, 2024

zzzeek commented Jun 16, 2024

zzzeek commented Jun 16, 2024

sqla-tester commented Jun 20, 2024

zzzeek commented Jun 23, 2024

The SQLAlchemy 1.4 C extension fails to compile on Python 3.13 #11499

The SQLAlchemy 1.4 C extension fails to compile on Python 3.13 #11499

Comments

musicinmybrain commented Jun 16, 2024

Describe the bug

Optional link from https://docs.sqlalchemy.org which documents the behavior that is expected

SQLAlchemy Version in Use

DBAPI (i.e. the database driver)

Database Vendor and Major Version

Python Version

Operating system

To Reproduce

Error

Additional context

CaselIT commented Jun 16, 2024 • edited Loading

zzzeek commented Jun 16, 2024

musicinmybrain commented Jun 16, 2024

CaselIT commented Jun 16, 2024

musicinmybrain commented Jun 16, 2024

zzzeek commented Jun 16, 2024

zzzeek commented Jun 16, 2024

sqla-tester commented Jun 20, 2024

zzzeek commented Jun 23, 2024

CaselIT commented Jun 16, 2024 •

edited

Loading