Skip to content

Support for macOS cross-architecture builds #225

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dnicolodi opened this issue Nov 22, 2022 · 3 comments
Closed

Support for macOS cross-architecture builds #225

dnicolodi opened this issue Nov 22, 2022 · 3 comments

Comments

@dnicolodi
Copy link
Member

Python wheels are often cross-compiled on macOS to arm64 hosts from x86_64 build machines. This is enabled by the macOS compilers allowing to cross-compile for amd64 and x86_64 with the -arch amd64 and -arch x86_64 command line arguments. setuptools invented the ARCHFLAGS environment variable to specify this arguments. The platform string returned by Python sysconfig.get_platform() can be ovrerwritten via the (undocumented) _PYTHON_HOST_PLATFORM environment variable. setuptools requires both these variables to be set correctly to generate the right object files in a wheel archive with the right tags. Other build tools like scikit-build and maturin rely only on the ARCHFLAGS environment variable. Established tools to build redistributable wheels such as cibuildwheel set both variables. However, the requirement to have to set two redundant variables in the correct way is a bit of a nuisance. (@henryiii is this a correct summary?)

The Meson way to setup a cross-build is via a cross-build-definition file https://mesonbuild.com/Cross-compilation.html#cross-compilation because it most commonly requires specifying the path (or at least the name) of the cross-compilers to be used. The situation for the macOS toolchain however, could be supported simply putting the right -arch flags in CFLAGS and similar variables.

I think there is some impedance matching here. At a minimum we need to support overwriting the wheel tag.

As an interim solution I would like to support overwriting the platform tag with the _PYTHON_HOST_PLATFORM environment variable. In combination with setting CFLAGS would allow to build arm64 wheels from x86_64 build machines.

@henryiii
Copy link
Contributor

henryiii commented Nov 22, 2022

Scikit-build-core (and I think Maturin) only uses the ARCHFLAGS, since it would be a bug to set _PYTHON_HOST_PLATFORM and ARCHFLAGS differently. Most other tools use ARCHFLAGS to control the target architecture, but also use wheel, which is controlled by _PYTHON_HOST_PLATFORM.

It's fine to be able to specify a cross file explicitly, but the existing Python way of setting things should be supported.

@dnicolodi
Copy link
Member Author

Meson delegates computing the extension filename suffix to the sysconfig module, thus there is no way around setting the _PYTHON_HOST_PLATFORM environment variable.

@rgommers rgommers changed the title Suport for macOS cross-architecture builds Support for macOS cross-architecture builds Nov 24, 2022
@dnicolodi
Copy link
Member Author

Fixed in #226.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants