From e76ad052be867e39ded186776f787d79503c05ab Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 21 Mar 2024 10:30:27 +0800 Subject: [PATCH 1/5] Add notes about htmlview and htmllive targets to the README. --- README.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.rst b/README.rst index f8f7ff2519..fde4a6c277 100644 --- a/README.rst +++ b/README.rst @@ -27,3 +27,11 @@ Render HTML To render the devguide to HTML under ``_build/html``, run:: make html + +To render the devguide to HTML, and open the result in a browser, run:: + + make htmlview + +To maintain a live view of edits as they are saved, run:: + + make htmllive From b79f0755d21527fb188f3e09578814f8d68c25e7 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 21 Mar 2024 10:30:41 +0800 Subject: [PATCH 2/5] Add an iOS developer guide. --- getting-started/setup-building.rst | 169 ++++++++++++++++++++++++++++- 1 file changed, 168 insertions(+), 1 deletion(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index fd1bce86da..b0cf8b8ef1 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -446,6 +446,155 @@ used in ``python.sh``: .. _wasmtime: https://wasmtime.dev .. _WebAssembly: https://webassembly.org +iOS +--- + +Compiling Python for iOS requires a macOS machine, on a recent version of macOS, +running a recent version of Xcode. Apple expects developers to keep their +operating systems and tools up-to-date; if your macOS version is more than one +major release out of date, or your Xcode version is more than a couple of minor +versions out of date, you'll likely encounter difficulties. It is not possible +to compile for iOS using Windows or Linux as a build machine. + +A complete build for Python on iOS requires compiling CPython 4 times: once for +macOS; then once for each of the three underlying platforms used by iOS: + +* An ARM64 device (an iPhone or iPad); +* An ARM64 simulator running on a recent macOS machine; and +* An x86_64 simulator running on older macOS machine. + +The macOS build is required because building Python involves running some Python +code. On a normal desktop build of Python, you can compile a Python interpreter +and then use that interpreter to run Python code. However, the binaries produced +for iOS won't run on macOS, so you need to provide an external Python +interpreter. From the root of a CPython code checkout, run the following: + +.. code-block:: shell + + $ ./configure --prefix=$(pwd)/cross-build/macOS + $ make -j4 all + $ make install + +This will build and install Python for macOS into the ``cross-build/macOS`` +directory. + +The CPython build system can compile a single platform at a time. It is possible +to *test* a single platform at a time; however, for distribution purposes, you +must compile all three, and merge the results. See the `iOS README +`__ +for details on this merging process. + +The following instructions will build CPython for iOS with all extensions +enabled, provided you have installed the build dependencies XZ, BZip2, OpenSSL +and libFFI in subfolders of the ``cross-build`` folder. See :ref:`the iOS +section on installing build dependencies ` for details on +how to obtain these dependencies. These dependencies are all strictly optional - +however, including libFFI is *highly* recommended, as it is required by the +:py:mod:`ctypes` module which is used on iOS to support accessing native system APIs. + +.. tab:: ARM64 Device + + .. code-block:: shell + + $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" + $ ./configure \ + LIBLZMA_CFLAGS="-I$(pwd)/cross-build/iphoneos.arm64/xz/include" \ + LIBLZMA_LIBS="-L$(pwd)/cross-build/iphoneos.arm64/xz/lib -llzma" \ + BZIP2_CFLAGS="-I$(pwd)/cross-build/iphoneos.arm64/bzip2/include" \ + BZIP2_LIBS="-L$(pwd)/cross-build/iphoneos.arm64/bzip2/lib -lbz2" \ + LIBFFI_CFLAGS="-I$(pwd)/cross-build/iphoneos.arm64/libffi/include" \ + LIBFFI_LIBS="-L$(pwd)/cross-build/iphoneos.arm64/libffi/lib -lffi" \ + --with-openssl="$(pwd)/cross-build/iphoneos.arm64/openssl" \ + --host=arm64-apple-ios12.0 \ + --build=arm64-apple-darwin \ + --with-build-python=$(pwd)/cross-build/macOS/bin/python3.13 \ + --without-ensurepip \ + --enable-framework + $ make -j4 all + $ make install + +.. tab:: ARM64 Simulator + + .. code-block:: shell + + $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" + $ ./configure \ + LIBLZMA_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.arm64/xz/include" \ + LIBLZMA_LIBS="-L$(pwd)/cross-build/iphonesimulator.arm64/xz/lib -llzma" \ + BZIP2_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.arm64/bzip2/include" \ + BZIP2_LIBS="-L$(pwd)/cross-build/iphonesimulator.arm64/bzip2/lib -lbz2" \ + LIBFFI_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.arm64/libffi/include" \ + LIBFFI_LIBS="-L$(pwd)/cross-build/iphonesimulator.arm64/libffi/lib -lffi" \ + --with-openssl="$(pwd)/cross-build/iphonesimulator.arm64/openssl" \ + --host=arm64-apple-ios12.0-simulator \ + --build=arm64-apple-darwin \ + --with-build-python=$(pwd)/cross-build/macOS/bin/python3.13 \ + --without-ensurepip \ + --enable-framework + $ make -j4 all + $ make install + +.. tab:: x86-64 Simulator + + .. code-block:: shell + + $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" + $ ./configure \ + LIBLZMA_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.x86_64/xz/include" \ + LIBLZMA_LIBS="-L$(pwd)/cross-build/iphonesimulator.x86_64/xz/lib -llzma" \ + BZIP2_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.x86_64/bzip2/include" \ + BZIP2_LIBS="-L$(pwd)/cross-build/iphonesimulator.x86_64/bzip2/lib -lbz2" \ + LIBFFI_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.x86_64/libffi/include" \ + LIBFFI_LIBS="-L$(pwd)/cross-build/iphonesimulator.x86_64/libffi/lib -lffi" \ + --with-openssl="$(pwd)/cross-build/iphonesimulator.x86_64/openssl" \ + --host=x86_64-apple-ios12.0-simulator \ + --build=arm64-apple-darwin \ + --with-build-python=$(pwd)/cross-build/macOS/bin/python3.13 \ + --without-ensurepip \ + --enable-framework + $ make -j4 all + $ make install + +These instructions modify your ``PATH`` before the build. As iOS and macOS share +a hardware architecture (ARM64), it is easy for a macOS ARM64 binary to be +accidentally linked into your iOS build. This is especially common when Homebrew +is present on the build system. The most reliable way to avoid this problem is +to remove any potential source of other libraries from your ``PATH``. + +However, the ``PATH`` is not completely bare - it includes the +``iOS/Resources/bin`` folder. This folder contains a collection of scripts that +wrap the invocation of the Xcode :program:`xcrun` tool, removing user- and +version-specific paths from the values encoded in the :py:mod:`sysconfig` +module. Copies of these scripts are included in the final build products. + +Once this build completes, the ``iOS/Frameworks`` folder will contain a +``Python.framework`` that can be used for testing. + +To run the test suite on iOS, complete a build for a *simulator* platform, +ensure the path modifications from the build are still in effect, and run: + +.. code-block:: shell + + $ make testios + +The full test suite takes approximately 12 minutes to run on a 2022 M1 MacBook +Pro, plus a couple of extra minutes to build the testbed application and boot +the simulator. There will be an initial burst of console output while the Xcode +test project is compiled; however, while the test suite is running, there is no +console output or progress. This is a side effect of how Xcode operates when +executed at the command line. You should see an iOS simulator appear during the +testing process; the simulator will booth to an iOS landing screen, the testbed +app will be installed, and then started. The screen of the simulator will be +black while the test suite is running. When the test suite completes, success or +failure will be reported at the command line. In the case of failure, you will +see the full log of CPython test suite output. + +You can also run the test suite in Xcode itself. This is required if you want to +run on a physical device; it is also the easiest approach if you need to run a +single test, or a subset of tests. See the `iOS README +`__ +for details. + .. _build-dependencies: .. _deps-on-linux: .. _macOS and OS X: @@ -455,7 +604,7 @@ Install dependencies ==================== This section explains how to install additional extensions (e.g. ``zlib``) -on Linux and macOS. +on Linux, macOS and iOS. .. tab:: Linux @@ -597,6 +746,24 @@ on Linux and macOS. On Windows, extensions are already included and built automatically. +.. tab:: iOS + + As with CPython itself, the dependencies for CPython must be compiled for + each of the hardware architectures that iOS supports. Consult the + documentation for `XZ `__, `BZip2 + `__, `OpenSSL `__ and + `libFFI `__ for details on how to configure + the project for cross-platform iOS builds. + + Alternatively, the `BeeWare Project `__ maintains a + `project for building iOS dependencies + `__, and distributes + `pre-compiled binaries + `__ for each + of the dependencies. If you use this project to build the dependencies + yourself, the subfolders of the ``install`` folder can be used to configure + CPython. If you use the pre-compiled binaries, you should unpack each tarball + into a separate folder, and use that folder as the configuration target. .. _regenerate_configure: From 6c562a0a9501f2203e5b880340b6d1011c6432f5 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 21 Mar 2024 11:48:20 +0800 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Ezio Melotti --- getting-started/setup-building.rst | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index b0cf8b8ef1..f53efb30f9 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -467,13 +467,11 @@ The macOS build is required because building Python involves running some Python code. On a normal desktop build of Python, you can compile a Python interpreter and then use that interpreter to run Python code. However, the binaries produced for iOS won't run on macOS, so you need to provide an external Python -interpreter. From the root of a CPython code checkout, run the following: +interpreter. From the root of a CPython code checkout, run the following:: -.. code-block:: shell - - $ ./configure --prefix=$(pwd)/cross-build/macOS - $ make -j4 all - $ make install + $ ./configure --prefix=$(pwd)/cross-build/macOS + $ make -j4 all + $ make install This will build and install Python for macOS into the ``cross-build/macOS`` directory. @@ -488,13 +486,13 @@ The following instructions will build CPython for iOS with all extensions enabled, provided you have installed the build dependencies XZ, BZip2, OpenSSL and libFFI in subfolders of the ``cross-build`` folder. See :ref:`the iOS section on installing build dependencies ` for details on -how to obtain these dependencies. These dependencies are all strictly optional - +how to obtain these dependencies. These dependencies are all strictly optional, however, including libFFI is *highly* recommended, as it is required by the :py:mod:`ctypes` module which is used on iOS to support accessing native system APIs. .. tab:: ARM64 Device - .. code-block:: shell + .. code-block:: console $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" $ ./configure \ @@ -515,7 +513,7 @@ however, including libFFI is *highly* recommended, as it is required by the .. tab:: ARM64 Simulator - .. code-block:: shell + .. code-block:: console $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" $ ./configure \ @@ -536,7 +534,7 @@ however, including libFFI is *highly* recommended, as it is required by the .. tab:: x86-64 Simulator - .. code-block:: shell + .. code-block:: console $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" $ ./configure \ @@ -561,7 +559,7 @@ accidentally linked into your iOS build. This is especially common when Homebrew is present on the build system. The most reliable way to avoid this problem is to remove any potential source of other libraries from your ``PATH``. -However, the ``PATH`` is not completely bare - it includes the +However, the ``PATH`` is not completely bare --- it includes the ``iOS/Resources/bin`` folder. This folder contains a collection of scripts that wrap the invocation of the Xcode :program:`xcrun` tool, removing user- and version-specific paths from the values encoded in the :py:mod:`sysconfig` @@ -571,11 +569,9 @@ Once this build completes, the ``iOS/Frameworks`` folder will contain a ``Python.framework`` that can be used for testing. To run the test suite on iOS, complete a build for a *simulator* platform, -ensure the path modifications from the build are still in effect, and run: - -.. code-block:: shell +ensure the path modifications from the build are still in effect, and run:: - $ make testios + $ make testios The full test suite takes approximately 12 minutes to run on a 2022 M1 MacBook Pro, plus a couple of extra minutes to build the testbed application and boot From 8c9e5354d6c5ee2e8312acabd8b1e1c2b8f2fc91 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Fri, 22 Mar 2024 05:29:14 +0800 Subject: [PATCH 4/5] Apply suggestions from code review Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- getting-started/setup-building.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index f53efb30f9..d6737df355 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -456,7 +456,7 @@ major release out of date, or your Xcode version is more than a couple of minor versions out of date, you'll likely encounter difficulties. It is not possible to compile for iOS using Windows or Linux as a build machine. -A complete build for Python on iOS requires compiling CPython 4 times: once for +A complete build for Python on iOS requires compiling CPython four times: once for macOS; then once for each of the three underlying platforms used by iOS: * An ARM64 device (an iPhone or iPad); @@ -490,7 +490,7 @@ how to obtain these dependencies. These dependencies are all strictly optional, however, including libFFI is *highly* recommended, as it is required by the :py:mod:`ctypes` module which is used on iOS to support accessing native system APIs. -.. tab:: ARM64 Device +.. tab:: ARM64 device .. code-block:: console @@ -511,7 +511,7 @@ however, including libFFI is *highly* recommended, as it is required by the $ make -j4 all $ make install -.. tab:: ARM64 Simulator +.. tab:: ARM64 simulator .. code-block:: console @@ -532,7 +532,7 @@ however, including libFFI is *highly* recommended, as it is required by the $ make -j4 all $ make install -.. tab:: x86-64 Simulator +.. tab:: x86-64 simulator .. code-block:: console @@ -746,9 +746,9 @@ on Linux, macOS and iOS. As with CPython itself, the dependencies for CPython must be compiled for each of the hardware architectures that iOS supports. Consult the - documentation for `XZ `__, `BZip2 + documentation for `XZ `__, `bzip2 `__, `OpenSSL `__ and - `libFFI `__ for details on how to configure + `libffi `__ for details on how to configure the project for cross-platform iOS builds. Alternatively, the `BeeWare Project `__ maintains a From dbcee2b3f01049e636b3ecad746ef8bc873cef88 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 27 Mar 2024 15:33:35 +0800 Subject: [PATCH 5/5] Remove --without-ensurepip; this is now implied by the default build configuration on iOS. --- getting-started/setup-building.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index d6737df355..30b7567088 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -506,7 +506,6 @@ however, including libFFI is *highly* recommended, as it is required by the --host=arm64-apple-ios12.0 \ --build=arm64-apple-darwin \ --with-build-python=$(pwd)/cross-build/macOS/bin/python3.13 \ - --without-ensurepip \ --enable-framework $ make -j4 all $ make install @@ -527,7 +526,6 @@ however, including libFFI is *highly* recommended, as it is required by the --host=arm64-apple-ios12.0-simulator \ --build=arm64-apple-darwin \ --with-build-python=$(pwd)/cross-build/macOS/bin/python3.13 \ - --without-ensurepip \ --enable-framework $ make -j4 all $ make install @@ -548,7 +546,6 @@ however, including libFFI is *highly* recommended, as it is required by the --host=x86_64-apple-ios12.0-simulator \ --build=arm64-apple-darwin \ --with-build-python=$(pwd)/cross-build/macOS/bin/python3.13 \ - --without-ensurepip \ --enable-framework $ make -j4 all $ make install