generator/linux: Copy ELF interpreter from architecture-specific locations #154
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@swift-ci test |
euanh
force-pushed
the
issue147-copy-ldlinux-aarch64
branch
from
e9b34c1 to
83850f4
Compare
|
@swift-ci test |
MaxDesiatov
approved these changes
Nov 25, 2024
euanh
force-pushed
the
issue147-copy-ldlinux-aarch64
branch
from
83850f4 to
e690ad4
Compare
|
@swift-ci test |
euanh
added a commit
to euanh/swift-sdk-generator
that referenced
this pull request
Nov 26, 2024
Building with a Swift 6.0 SDK fails because the new `_FoundationCShims`
framework headers can't be found:
<unknown>:0: error: missing required module '_FoundationCShims'
The necessary files are present in `$PLATFORM.sdk/usr/lib/swift`,
but are not part of the list which is copied to
`$PLATFORM.sdk/usr/include`. The Static Linux SDK and WASM SDK
generator don't copy these files into /usr/include; instead they
set the `swiftResourcesPath` and `swiftStaticResourcesPath` fields
in `swift-sdk.json`, which causes the build system to look in those
paths for framework headers.
We can do the same for Linux SDKs, however we must stop copying the
files to `/usr/include`, otherwise the build will fail because the
framework header definitions are duplicated:
error: redefinition of module 'DispatchIntrospection'
6 | }
7 |
8 | module DispatchIntrospection [system] [extern_c] {
| `- error: redefinition of module 'DispatchIntrospection'
9 | header "introspection.h"
10 | export *
This change fixes issue swiftlang#152. Compared to swiftlang#153, the end-to-end
tests now pass for all supported Swift versions, distributions and
architectures.
| SDK | Hello World | Foundation |
| ----------------------------------------- | ----------- | ---------- |
| ubuntu_aarch64_5.9.2-RELEASE | ok | ok |
| ubuntu_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| ubuntu_x86_64_5.9.2-RELEASE | ok | ok |
| ubuntu_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
FAIL1: cannot find /lib/ld-linux-aarch64.so.1 (swiftlang#147)
FAIL2: missing required module '_FoundationCShims' (swiftlang#152)
Fixes: swiftlang#152
Depends on: swiftlang#153, swiftlang#154
euanh
changed the title
euanh
force-pushed
the
issue147-copy-ldlinux-aarch64
branch
from
e690ad4 to
cc30535
Compare
|
@swift-ci test |
euanh
added a commit
to euanh/swift-sdk-generator
that referenced
this pull request
Nov 27, 2024
Building with a Swift 6.0 SDK fails because the new `_FoundationCShims`
framework headers can't be found:
<unknown>:0: error: missing required module '_FoundationCShims'
The necessary files are present in `$PLATFORM.sdk/usr/lib/swift`,
but are not part of the list which is copied to
`$PLATFORM.sdk/usr/include`. The Static Linux SDK and WASM SDK
generator don't copy these files into /usr/include; instead they
set the `swiftResourcesPath` and `swiftStaticResourcesPath` fields
in `swift-sdk.json`, which causes the build system to look in those
paths for framework headers.
We can do the same for Linux SDKs, however we must stop copying the
files to `/usr/include`, otherwise the build will fail because the
framework header definitions are duplicated:
error: redefinition of module 'DispatchIntrospection'
6 | }
7 |
8 | module DispatchIntrospection [system] [extern_c] {
| `- error: redefinition of module 'DispatchIntrospection'
9 | header "introspection.h"
10 | export *
This change fixes issue swiftlang#152. Compared to swiftlang#153, the end-to-end
tests now pass for all supported Swift versions, distributions and
architectures.
| SDK | Hello World | Foundation |
| ----------------------------------------- | ----------- | ---------- |
| ubuntu_aarch64_5.9.2-RELEASE | ok | ok |
| ubuntu_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| ubuntu_x86_64_5.9.2-RELEASE | ok | ok |
| ubuntu_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
FAIL1: cannot find /lib/ld-linux-aarch64.so.1 (swiftlang#147)
FAIL2: missing required module '_FoundationCShims' (swiftlang#152)
Fixes: swiftlang#152
Depends on: swiftlang#153, swiftlang#154
…tions Issue swiftlang#147 occurs because the ELF interpreter is found at different paths, and with different names, on different architectures. We happen to pick up the x86_64 interpreter because it is stored in /lib64, which we aready copy, but we miss the aarch64 interpreter. An ELF binary contains an interpreter path which can point anywhere, but there are defacto standard default locations. * According to https://refspecs.linuxbase.org/elf/x86_64-abi-0.99.pdf) the x86_64 / amd64 ABI specifies that the interpreter should be at /lib/ld64.so.1, but Linux overrides this and uses /lib64/ld-linux-x86-64.so.2. * I couldn't find a similar document for ARM64 / aarch64 - https://github.com/ARM-software/abi-aa/tree/main/aaelf64 is silent on the interpreter path - but most sources I could find place it at /lib/ld-linux-aarch64.so.1 - for example zulu-openjdk/zulu-openjdk#11 There is a pattern to the paths and filenames but it seems risky to try to construct them; instead this PR adds a new field to `Triple` which returns the default path for each architecture. For now we only support x86_64 and aarch64 - if new architectures are added in future we will need to add new entries for them. This change fixes issue swiftlang#147. Compared to swiftlang#153, all the basic 'hello world' tests now pass. The tests which import Foundation still fail on all Swift 6.0 SDKs. | SDK | Hello World | Foundation | | ----------------------------------------- | ----------- | ---------- | | ubuntu_aarch64_5.9.2-RELEASE | ok | ok | | ubuntu_aarch64_5.9.2-RELEASE_with-docker | ok | ok | | ubuntu_aarch64_5.10.1-RELEASE | ok | ok | | ubuntu_aarch64_5.10.1-RELEASE_with-docker | ok | ok | | ubuntu_aarch64_6.0.2-RELEASE | ok | FAIL2 | | ubuntu_aarch64_6.0.2-RELEASE_with-docker | ok | FAIL2 | | | | | | ubuntu_x86_64_5.9.2-RELEASE | ok | ok | | ubuntu_x86_64_5.9.2-RELEASE_with-docker | ok | ok | | ubuntu_x86_64_5.10.1-RELEASE | ok | ok | | ubuntu_x86_64_5.10.1-RELEASE_with-docker | ok | ok | | ubuntu_x86_64_6.0.2-RELEASE | ok | FAIL2 | | ubuntu_x86_64_6.0.2-RELEASE_with-docker | ok | FAIL2 | | | | | | rhel_aarch64_5.9.2-RELEASE_with-docker | ok | ok | | rhel_aarch64_5.10.1-RELEASE_with-docker | ok | ok | | rhel_aarch64_6.0.2-RELEASE_with-docker | ok | FAIL2 | | | | | | rhel_x86_64_5.9.2-RELEASE_with-docker | ok | ok | | rhel_x86_64_5.10.1-RELEASE_with-docker | ok | ok | | rhel_x86_64_6.0.2-RELEASE_with-docker | ok | FAIL2 | FAIL1: cannot find /lib/ld-linux-aarch64.so.1 FAIL2: missing required module '_FoundationCShims' Fixes: swiftlang#147
euanh
force-pushed
the
issue147-copy-ldlinux-aarch64
branch
from
cc30535 to
6fb3fbc
Compare
|
@swift-ci test |
euanh
added a commit
to euanh/swift-sdk-generator
that referenced
this pull request
Nov 27, 2024
Building with a Swift 6.0 SDK fails because the new `_FoundationCShims`
framework headers can't be found:
<unknown>:0: error: missing required module '_FoundationCShims'
The necessary files are present in `$PLATFORM.sdk/usr/lib/swift`,
but are not part of the list which is copied to
`$PLATFORM.sdk/usr/include`. The Static Linux SDK and WASM SDK
generator don't copy these files into /usr/include; instead they
set the `swiftResourcesPath` and `swiftStaticResourcesPath` fields
in `swift-sdk.json`, which causes the build system to look in those
paths for framework headers.
We can do the same for Linux SDKs, however we must stop copying the
files to `/usr/include`, otherwise the build will fail because the
framework header definitions are duplicated:
error: redefinition of module 'DispatchIntrospection'
6 | }
7 |
8 | module DispatchIntrospection [system] [extern_c] {
| `- error: redefinition of module 'DispatchIntrospection'
9 | header "introspection.h"
10 | export *
This change fixes issue swiftlang#152. Compared to swiftlang#153, the end-to-end
tests now pass for all supported Swift versions, distributions and
architectures.
| SDK | Hello World | Foundation |
| ----------------------------------------- | ----------- | ---------- |
| ubuntu_aarch64_5.9.2-RELEASE | ok | ok |
| ubuntu_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| ubuntu_x86_64_5.9.2-RELEASE | ok | ok |
| ubuntu_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
FAIL1: cannot find /lib/ld-linux-aarch64.so.1 (swiftlang#147)
FAIL2: missing required module '_FoundationCShims' (swiftlang#152)
Fixes: swiftlang#152
Depends on: swiftlang#153, swiftlang#154
euanh
added a commit
to euanh/swift-sdk-generator
that referenced
this pull request
Nov 28, 2024
Building with a Swift 6.0 SDK fails because the new `_FoundationCShims`
framework headers can't be found:
<unknown>:0: error: missing required module '_FoundationCShims'
The necessary files are present in `$PLATFORM.sdk/usr/lib/swift`,
but are not part of the list which is copied to
`$PLATFORM.sdk/usr/include`. The Static Linux SDK and WASM SDK
generator don't copy these files into /usr/include; instead they
set the `swiftResourcesPath` and `swiftStaticResourcesPath` fields
in `swift-sdk.json`, which causes the build system to look in those
paths for framework headers.
We can do the same for Linux SDKs, however we must stop copying the
files to `/usr/include`, otherwise the build will fail because the
framework header definitions are duplicated:
error: redefinition of module 'DispatchIntrospection'
6 | }
7 |
8 | module DispatchIntrospection [system] [extern_c] {
| `- error: redefinition of module 'DispatchIntrospection'
9 | header "introspection.h"
10 | export *
This change fixes issue swiftlang#152. Compared to swiftlang#153, the end-to-end
tests now pass for all supported Swift versions, distributions and
architectures.
| SDK | Hello World | Foundation |
| ----------------------------------------- | ----------- | ---------- |
| ubuntu_aarch64_5.9.2-RELEASE | ok | ok |
| ubuntu_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| ubuntu_x86_64_5.9.2-RELEASE | ok | ok |
| ubuntu_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
FAIL1: cannot find /lib/ld-linux-aarch64.so.1 (swiftlang#147)
FAIL2: missing required module '_FoundationCShims' (swiftlang#152)
Fixes: swiftlang#152
Depends on: swiftlang#153, swiftlang#154
euanh
added a commit
that referenced
this pull request
Nov 28, 2024
Building with a Swift 6.0 SDK fails because the new `_FoundationCShims`
framework headers can't be found:
<unknown>:0: error: missing required module '_FoundationCShims'
The necessary files are present in `$PLATFORM.sdk/usr/lib/swift`,
but are not part of the list which is copied to
`$PLATFORM.sdk/usr/include`. The Static Linux SDK and WASM SDK
generator don't copy these files into /usr/include; instead they
set the `swiftResourcesPath` and `swiftStaticResourcesPath` fields
in `swift-sdk.json`, which causes the build system to look in those
paths for framework headers.
We can do the same for Linux SDKs, however we must stop copying the
files to `/usr/include`, otherwise the build will fail because the
framework header definitions are duplicated:
error: redefinition of module 'DispatchIntrospection'
6 | }
7 |
8 | module DispatchIntrospection [system] [extern_c] {
| `- error: redefinition of module 'DispatchIntrospection'
9 | header "introspection.h"
10 | export *
This change fixes issue #152. Compared to #153, the end-to-end
tests now pass for all supported Swift versions, distributions and
architectures.
| SDK | Hello World | Foundation |
| ----------------------------------------- | ----------- | ---------- |
| ubuntu_aarch64_5.9.2-RELEASE | ok | ok |
| ubuntu_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE | ok | ok |
| ubuntu_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE | ok | ok |
| ubuntu_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| ubuntu_x86_64_5.9.2-RELEASE | ok | ok |
| ubuntu_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE | ok | ok |
| ubuntu_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE | ok | ok |
| ubuntu_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_aarch64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_aarch64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_aarch64_6.0.2-RELEASE_with-docker | ok | ok |
| | | |
| rhel_x86_64_5.9.2-RELEASE_with-docker | ok | ok |
| rhel_x86_64_5.10.1-RELEASE_with-docker | ok | ok |
| rhel_x86_64_6.0.2-RELEASE_with-docker | ok | ok |
FAIL1: cannot find /lib/ld-linux-aarch64.so.1 (#147)
FAIL2: missing required module '_FoundationCShims' (#152)
Fixes: #152
Depends on: #153, #154
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #147 occurs because the ELF interpreter is found at different
paths, and with different names, on different architectures. We
happen to pick up the x86_64 interpreter because it is stored in /lib64,
which we aready copy, but we miss the aarch64 interpreter.
An ELF binary contains an interpreter path which can point anywhere, but
there are defacto standard default locations.
According to https://refspecs.linuxbase.org/elf/x86_64-abi-0.99.pdf)
the x86_64 / amd64 ABI specifies that the interpreter should be
at /lib/ld64.so.1, but Linux overrides this and uses
/lib64/ld-linux-x86-64.so.2.
I couldn't find a similar document for ARM64 / aarch64 -
https://github.com/ARM-software/abi-aa/tree/main/aaelf64 is silent
on the interpreter path - but most sources I could find place it
at /lib/ld-linux-aarch64.so.1 - for example
ELF interpreter should be /lib/ld-linux-aarch64.so.1 instead of /lib64/ld-linux-aarch64.so.1 on Ubuntu LTS 16.04 for aarch64 zulu-openjdk/zulu-openjdk#11
There is a pattern to the paths and filenames but it seems risky to
try to construct them; instead this PR adds a new field to
Triplewhich returns the default path for each architecture. For now we
only support x86_64 and aarch64 - if new architectures are added in
future we will need to add new entries for them.
This change fixes issue #147. Compared to #153, all the basic
'hello world' tests now pass. The tests which import Foundation
still fail on all Swift 6.0 SDKs.
FAIL1: cannot find /lib/ld-linux-aarch64.so.1 (#147)
FAIL2: missing required module '_FoundationCShims' (#152)
Fixes: #147
Depends on: #153