Implicitly support Darwin SDKs on macOS #6828

kabiroberai · 2023-08-18T03:36:17Z

This change makes it possible to run e.g. swift build --triple arm64-apple-ios on macOS to build for iOS.

Motivation:

With #6732 in place, it's possible to use swift-build for most Darwin targets on macOS with swift build --triple arm64-apple-ios --sdk $(xcrun --sdk iphoneos --show-sdk-path) etc. The SwiftSDK type already has support for computing a "default" SDK given a triple, so we can elide the --sdk argument by automatically inferring it on macOS.

Modifications:

Generalize hostSwiftSDK as a systemSwiftSDK function that additionally accepts a Darwin platform as an argument. On macOS, this allows us to compute a SwiftSDK that works for arbitrary Darwin targets, instead of just macosx.
Modify defaultSwiftSDK(for:) to return a non-macOS Darwin systemSwiftSDK if requested.
Modify the SwiftTool._targetToolchain computation to handle default SDKs passed via --swift-sdk, not just --triple. (Open to discussing whether this change is a good idea.)

Result:

Darwin targets get free Swift SDK support on macOS 🎉

Open questions:

Should default Swift SDKs be usable via --swift-sdk, and should they show up in swift sdk list? These two changes would ensure homogeneity; for example, if/when SourceKit-LSP supports SDK selection this would enable Darwin targets on macOS there too.

kabiroberai · 2023-08-18T04:19:56Z

Tests/PackageModelTests/SwiftSDKTests.swift

+
+        #if os(macOS)
+        let iOSTriple = try Triple("arm64-apple-ios")
+        let iOS = try XCTUnwrap(SwiftSDK.defaultSwiftSDK(for: iOSTriple, hostSDK: hostSDK))


Does the SwiftPM test suite have a way to mock shell invocations? This invokes xcrun so it would be great to 1) verify that xcrun is indeed invoked and 2) mock the result.

There's no such mocking available at the moment, unfortunately.

and of course this is precisely the test that fails on CI 😶 Looks like it can't find iPhoneOS.sdk — could that be because the iOS component isn't installed in the runner's Xcode?

Yes, that's quite likely, since we're not building for iOS on Swift CI. IMO better not to rely on that to be present.

@MaxDesiatov do you know if this is still the case? Worth triggering CI again to confirm? If so I could add some plumbing to inject xcrun as a mockable dependency instead.

Sources/PackageModel/SwiftSDK.swift

kabiroberai · 2023-08-20T00:04:08Z

Sources/Commands/Utilities/TestingSupport.swift

@@ -102,6 +102,15 @@ enum TestingSupport {
    ) throws -> [TestSuite] {
        // Run the correct tool.
        #if os(macOS)
+        let targetTriple = try swiftTool.getTargetToolchain().targetTriple
+        guard targetTriple.darwinPlatform == .macOS else {


@MaxDesiatov thought I'd point out that this is a good opportunity to use your matches method from #6819 — in fact the check here is currently not strong enough because swiftpm-xctest-helper can't handle x64 tests on arm64 macOS without explicitly requesting Rosetta translation (not to mention that AS-on-Intel is also not trivially possible). What do you think about renaming matches to something along the lines of isRuntimeCompatible(with:) since those are the emergent semantics?

Also, I wonder whether it's worth a spike to support iOS simulator testing on macOS — though iiuc we'd need a simulator swiftpm-xctest-helper binary for that.

What do you think about renaming matches to something along the lines of isRuntimeCompatible(with:) since those are the emergent semantics?

I like that idea, thanks! I've updated the corresponding PR 👍

though iiuc we'd need a simulator swiftpm-xctest-helper binary for that.

Probably, yah. Note that swiftpm-xctest-helper isn't part of SwiftPM itself anymore for a bit, so you would need to file a FB for that.

The --triple codepath in SwiftTool also sets this but that's happenstance; we don't want defaultSwiftSDK to ever return an SDK with the wrong targetTriple.

Preserve the `environmentName` part of triples when altering the OS version with `Triple.tripleString(forPlatformVersion:)` ### Motivation: Building for the iOS simulator (and presumably anything other triple with a fourth component) would fail because we would generate the wrong swiftc/clang `-target`, incorrectly excluding the `Triple.environmentName`. ### Modifications: - Modify `tripleString(forPlatformVersion:)` to include `self.environmentName` if it is non-empty. ### Result: You can now build for the iOS simulator with ```shell swift build --triple arm64-apple-ios-simulator --sdk "$(xcrun --sdk iphonesimulator --show-sdk-path)" ``` or if you also pull #6828, just ```shell swift build --triple arm64-apple-ios-simulator ```

Sources/CoreCommands/SwiftTool.swift

MaxDesiatov

Thank you!

MaxDesiatov · 2023-09-08T10:35:47Z

@swift-ci test

MaxDesiatov · 2023-09-19T19:12:48Z

@swift-ci test macos

kabiroberai · 2023-09-19T19:38:01Z

@MaxDesiatov fyi I don't think this will succeed on macOS until I update the tests to work around the lack of an iOS SDK (unless CI was updated with iOS recently). Will get to it soon, excuse the delay; been a bit busy.

kabiroberai · 2024-08-31T03:39:11Z

should be good to test now!

MaxDesiatov · 2024-08-31T07:24:19Z

@swift-ci test

MaxDesiatov · 2024-08-31T16:50:35Z

@swift-ci test windows

kabiroberai · 2024-09-01T01:46:28Z

@MaxDesiatov any idea what the macOS test failure is about? seems unrelated to my changes, something about cxx interop not understanding std::span 🤔

MaxDesiatov · 2024-09-01T08:08:09Z

@swift-ci test macos

I’m not sure if this is causing CI to fail but we should probably add these checks somewhere else where they can apply to swift-testing too. Not in the critical path so worth moving to a followup.

kabiroberai · 2024-09-02T02:21:26Z

@MaxDesiatov I pushed some minor tweaks but I don't think anything should affect the tests — still unsure what's up with the failing test

kabiroberai · 2024-09-02T04:13:34Z

Looks like the CI failure is a bug on the swiftlang/swift side — swiftlang/swift#75436 (comment)

Seems this should be fixed once swiftlang/swift#76207 lands

MaxDesiatov · 2024-09-03T10:29:25Z

@swift-ci test

MaxDesiatov · 2024-09-03T10:29:43Z

@swift-ci test windows

kabiroberai · 2024-09-04T05:50:11Z

@MaxDesiatov by the looks of it CI might finally be in working shape again after swiftlang/swift#76236

Edit: looks like CI runs are still failing for other PRs so maybe not?

kabiroberai · 2024-09-04T05:55:25Z

would this also need updates to the bootstrap script?

@peterkos (long time!) fyi for posterity I don't think we need to update the bootstrap script because that would only be necessary if we supported building the SwiftPM package on a non-macOS-but-still-Darwin host (which would be cool, but afaik we'd have numerous other hurdles in the way.) For cases like Swift Playgrounds — where iPadOS is the compiler host — I would assume libSwiftPM is cross-compiled from macOS to iOS rather than being bootstrapped on an iPadOS device.

MaxDesiatov · 2024-09-04T11:28:17Z

@swift-ci test macos

kabiroberai · 2024-09-04T18:27:54Z

CI is green! are we good to merge this?

MaxDesiatov · 2024-09-04T19:38:13Z

I'd like this to get a review from @dschaefer2 or someone on his team first.

dschaefer2 · 2024-10-23T15:19:32Z

@swift-ci please test

dschaefer2 · 2024-10-23T18:51:13Z

@swift-ci please test windows

kabiroberai · 2024-10-25T20:35:15Z

from the Windows CI log:

Test Case 'TestFileManager.test_emptyFilename' started at 2024-10-23 22:25:43.796

TestFoundation/TestFileManager.swift:1361: error: TestFileManager.test_emptyFilename : XCTAssertEqual failed: ("Optional(FoundationEssentials.CocoaError.Code(rawValue: 512))") is not equal to ("Optional(FoundationEssentials.CocoaError.Code(rawValue: 4))") - 

Test Case 'TestFileManager.test_emptyFilename' failed (0.004 seconds)

🤔 this seems unrelated to the PR

MaxDesiatov · 2024-10-27T08:16:25Z

@swift-ci test windows

dschaefer2 · 2024-10-28T14:16:12Z

Looks ready to go. Thanks!

kabiroberai added 2 commits August 17, 2023 23:22

Implicitly support all Darwin triples on macOS

9aa5ee8

comment

b82a400

kabiroberai requested review from abertelrud, neonichu, tomerd and MaxDesiatov as code owners August 18, 2023 03:36

kabiroberai added 3 commits August 18, 2023 00:13

Fix sdkPlatformFrameworkPath memoization

ca0822d

Add tests

a202320

a dash of determinism

46b65d1

kabiroberai commented Aug 18, 2023

View reviewed changes

MaxDesiatov reviewed Aug 19, 2023

View reviewed changes

Sources/PackageModel/SwiftSDK.swift Outdated Show resolved Hide resolved

MaxDesiatov added the cross-compilation label Aug 19, 2023

kabiroberai added 3 commits August 19, 2023 19:29

Feedback

d253fab

We can't run cross-compiled tests yet

ba32e4c

test suite -> tests

b273ba4

MaxDesiatov reviewed Aug 19, 2023

View reviewed changes

Sources/PackageModel/SwiftSDK.swift Outdated Show resolved Hide resolved

Reintroduce tuple API as deprecated

2f9adad

kabiroberai commented Aug 20, 2023

View reviewed changes

MaxDesiatov self-assigned this Aug 20, 2023

kabiroberai mentioned this pull request Aug 21, 2023

Preserve environment in tripleString(forPlatformVersion:) #6834

Merged

kabiroberai added 2 commits August 21, 2023 23:50

Set targetTriple for systemSwiftSDK

9e51703

The --triple codepath in SwiftTool also sets this but that's happenstance; we don't want defaultSwiftSDK to ever return an SDK with the wrong targetTriple.

Allow default SDKs via --swift-sdk

ed1513e

kabiroberai requested a review from MaxDesiatov September 7, 2023 08:59

MaxDesiatov reviewed Sep 7, 2023

View reviewed changes

Sources/CoreCommands/SwiftTool.swift Outdated Show resolved Hide resolved

feedback

c62ad42

MaxDesiatov approved these changes Sep 8, 2023

View reviewed changes

kabiroberai added 4 commits September 1, 2024 21:04

potential fixes

5560fd3

fix warning

79f582d

drop xctest platform checks

3fe9ac7

I’m not sure if this is causing CI to fail but we should probably add these checks somewhere else where they can apply to swift-testing too. Not in the critical path so worth moving to a followup.

use xcrunName for platformPath

24cfc46

MaxDesiatov removed the needs tests This change needs test coverage label Sep 3, 2024

MaxDesiatov approved these changes Sep 3, 2024

View reviewed changes

kabiroberai added 2 commits September 4, 2024 16:28

Use xcrunName for platform path override as well

8351d14

Merge branch 'main' into darwin-target-sdks

bb4f49c

dschaefer2 merged commit f78fc4e into swiftlang:main Oct 28, 2024
5 checks passed

kabiroberai mentioned this pull request Nov 2, 2024

Support Swift SDKs swiftlang/vscode-swift#1191

Merged

1 task

kabiroberai mentioned this pull request Jan 16, 2025

Support background indexing when cross-compiling swiftlang/sourcekit-lsp#1923

Merged

Implicitly support Darwin SDKs on macOS #6828

Implicitly support Darwin SDKs on macOS #6828

Uh oh!

Conversation

kabiroberai commented Aug 18, 2023 • edited by MaxDesiatov Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation:

Modifications:

Result:

Open questions:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kabiroberai Aug 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MaxDesiatov left a comment

Choose a reason for hiding this comment

Uh oh!

MaxDesiatov commented Sep 8, 2023

Uh oh!

MaxDesiatov commented Sep 19, 2023

Uh oh!

kabiroberai commented Sep 19, 2023

Uh oh!

kabiroberai commented Aug 31, 2024

Uh oh!

MaxDesiatov commented Aug 31, 2024

Uh oh!

MaxDesiatov commented Aug 31, 2024

Uh oh!

kabiroberai commented Sep 1, 2024

Uh oh!

MaxDesiatov commented Sep 1, 2024

Uh oh!

kabiroberai commented Sep 2, 2024

Uh oh!

kabiroberai commented Sep 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MaxDesiatov commented Sep 3, 2024

Uh oh!

MaxDesiatov commented Sep 3, 2024

Uh oh!

kabiroberai commented Sep 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kabiroberai commented Sep 4, 2024

Uh oh!

MaxDesiatov commented Sep 4, 2024

Uh oh!

kabiroberai commented Sep 4, 2024

Uh oh!

MaxDesiatov commented Sep 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dschaefer2 commented Oct 23, 2024

Uh oh!

dschaefer2 commented Oct 23, 2024

Uh oh!

kabiroberai commented Oct 25, 2024

Uh oh!

kabiroberai commented Aug 18, 2023 •

edited by MaxDesiatov

Loading

kabiroberai Aug 20, 2023 •

edited

Loading

kabiroberai commented Sep 2, 2024 •

edited

Loading

kabiroberai commented Sep 4, 2024 •

edited

Loading

MaxDesiatov commented Sep 4, 2024 •

edited

Loading