Merge main into release/6.2
#214
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Defaulted `swiftVersion` to "6.1-RELEASE".
- Added EndToEndTests for Ubuntu and RHEL for Swift61, including some additional tests for Ubuntu Jammy.
- Fixed missing /lib and /lib64 directories in Ubuntu Noble when building from deb packages. If these directories don't exist, they are symlinked to the correct directories in /usr, which is consistent to what happens when copying from the Docker containers.
- Added the fix for downloading Swift for the target that adds the platform name to the cache file, to avoid corruption of the artifacts:
* <img width="375" alt="image" src="https://github.com/user-attachments/assets/b18efd38-2cbe-44e1-a6fd-2fbb1a13002b" />
ALSO: I did not include EndToEndTests for Fedora39 for Swift 6.1 since the container is non-existent, likely because it's EOL.
Utilities/soundness.sh was deleted in #199 which moved CI to GitHub Actions. Credit to @heckj, who found the same problem in apple/swift-container-plugin#96.
This completes #116 and can be used to generate Debian Swift SDKs either by downloading dependencies from mirrors directly or from containers. - Added Debian `bullseye` and `bookworm` to `LinuxDistribution` and modded `SwiftSDKGenerator+Download` to work downloading both Ubuntu and Debian packages from their respective mirrors. - Swift SDKs can be generated from Docker using the `swift:*-bookworm` container for Debian 12, but for Debian 11 you must create the container yourself/manually put the Swift toolchain into it. - Added special handling in `VersionsConfiguration` so that if Debian 11 (bullseye) is selected, the Ubuntu 20.04 toolchain is downloaded. If Debian 12 (bookworm) is selected for Swift 5.9.* or Swift 5.10, the Ubuntu 22.04 toolchain is used instead. https://askubuntu.com/a/445496 - Added tests for the special handling in `LinuxRecipeTests.testItemsToDownloadForDebianTargets()` to ensure the correct toolchain is selected. To use: ```bash $ swift run swift-sdk-generator make-linux-sdk --linux-distribution-name debian --linux-distribution-version 11 $ swift run swift-sdk-generator make-linux-sdk --linux-distribution-name debian --linux-distribution-version 12 $ swift run swift-sdk-generator make-linux-sdk --linux-distribution-name debian --linux-distribution-version 12 --with-docker ``` I also added EndToEndTests for all Debian supported combinations, resulting in another 30GB of generated Swift SDKs: ``` 2.1 GiB [####################] /debian_12_x86_64_6.0.3-RELEASE_with-docker.artifactbundle 2.1 GiB [################### ] /debian_12_aarch64_6.0.3-RELEASE_with-docker.artifactbundle 2.0 GiB [################## ] /debian_12_x86_64_5.10.1-RELEASE_with-docker.artifactbundle 1.9 GiB [################# ] /debian_12_x86_64_6.0.3-RELEASE.artifactbundle 1.9 GiB [################# ] /debian_12_aarch64_6.0.3-RELEASE.artifactbundle 1.9 GiB [################# ] /debian_11_x86_64_6.0.3-RELEASE.artifactbundle 1.9 GiB [################# ] /debian_12_aarch64_5.10.1-RELEASE_with-docker.artifactbundle 1.9 GiB [################# ] /debian_11_aarch64_6.0.3-RELEASE.artifactbundle 1.8 GiB [################ ] /debian_12_x86_64_5.10.1-RELEASE.artifactbundle 1.7 GiB [################ ] /debian_12_aarch64_5.10.1-RELEASE.artifactbundle 1.7 GiB [################ ] /debian_11_x86_64_5.10.1-RELEASE.artifactbundle 1.7 GiB [############### ] /debian_11_aarch64_5.10.1-RELEASE.artifactbundle 1.7 GiB [############### ] /debian_12_x86_64_5.9.2-RELEASE.artifactbundle 1.7 GiB [############### ] /debian_12_aarch64_5.9.2-RELEASE.artifactbundle 1.7 GiB [############### ] /debian_11_x86_64_5.9.2-RELEASE.artifactbundle 1.6 GiB [############### ] /debian_11_aarch64_5.9.2-RELEASE.artifactbundle ``` To make this work properly, I modded the `targetSwift` path in `DownloadableArtifacts` to use the swiftPlatform name in the file name to avoid the issues I saw with the EndToEndTests, where the *.tar.gz files would get corrupted when trying to download a different version of the target Swift on top of the existing file. Now, they look like this: ``` target_swift_5.10.1-RELEASE_debian12_aarch64.tar.gz target_swift_5.9.2-RELEASE_ubuntu22.04_aarch64.tar.gz target_swift_5.10.1-RELEASE_debian12_x86_64.tar.gz target_swift_5.9.2-RELEASE_ubuntu22.04_x86_64.tar.gz target_swift_5.10.1-RELEASE_ubuntu20.04_aarch64.tar.gz target_swift_6.0.3-RELEASE_debian12_aarch64.tar.gz target_swift_5.10.1-RELEASE_ubuntu20.04_x86_64.tar.gz target_swift_6.0.3-RELEASE_debian12_x86_64.tar.gz target_swift_5.10-RELEASE_ubuntu20.04_aarch64.tar.gz target_swift_6.0.3-RELEASE_ubuntu20.04_aarch64.tar.gz target_swift_5.9.2-RELEASE_ubuntu20.04_aarch64.tar.gz target_swift_6.0.3-RELEASE_ubuntu20.04_x86_64.tar.gz target_swift_5.9.2-RELEASE_ubuntu20.04_x86_64.tar.gz ```
This seems to be a reasonable limit still readable on most devices, while reducing the amount of reformatting applied by `swift format` checks.
This allows targeting WASI with Embedded Swift, enabling use of a subset of Swift Concurrency in this mode, and also hiding most if not all of unsafe flags in toolsets. This way, users of Embedded Swift can build their packages with a similar `swift build --swift-sdk` command as the rest of users of WASI.
The generator incorrectly recorded absolute paths to Swift SDK metadata in bundle archive metadata, which is fragile and also can't be handled by SwiftPM.
Embedded Swift looks up Clang's compiler-rt static library in a different path than non-embedded. These paths could be normalized in the future, but for now the easiest fix seems to be just to symlink the directory the static library is located in.
bnbarham
approved these changes
May 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Explanation: This updates
release/6.2branch of the generator used when building 6.2 Swift 6.2 toolchain with recent changes. Most notably:// swift-tools-versionScope: Isolated to cross-compilation and only impacts existing Ubuntu toolchain job that generates Swift SDK for WASI.
Risk: Low, most of the changes LoC-wise were incubated on the
mainbranch for a month.Testing: Expanded test suite, corresponding toolchain builds are passing https://ci.swift.org/job/oss-swift-pr-test-crosscompile-wasm-ubuntu-20_04/
Issue: rdar://151475169
Reviewers: @kateinoigakukun @euanh