Skip to content

Linker error/regression introduced with rustc 1.48.0 on aarch64-unknown-linux-musl target #79791

Closed
@acatangiu

Description

@acatangiu

Hi all!

First of all congrats on #76420! We at firecracker are using the aarch64-unknown-linux-musl target and are very excited to see it promoted to Tier 2.

Unfortunately, while trying to move to 1.48.0 I am encountering a linker error for which I've yet to find a solution. Hope you can help here!

Code

cargo build --target aarch64-unknown-linux-musl --release:

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-Wl,--eh-frame-hdr" "-nostartfiles" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crt1.o" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crti.o" "-L" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib" "-L" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained" "/firecracker/build/cargo_target/aarch64-unknown-linux-musl/release/deps/firecracker-8bf1ef43b7fbfc39.firecracker.5v4bbyky-cgu.11.rcgu.o" "-o" "/firecracker/build/cargo_target/aarch64-unknown-linux-musl/release/deps/firecracker-8bf1ef43b7fbfc39" "-Wl,--gc-sections" "-static" "-no-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/firecracker/build/cargo_target/aarch64-unknown-linux-musl/release/deps" "-L" "/firecracker/build/cargo_target/release/deps" "-L" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/tmp/rustcRAyWtZ/libunwind-1271d5b7d3b7bb97.rlib" "/tmp/rustcRAyWtZ/liblibc-99a9d4afa9ae21ce.rlib" "-Wl,--end-group" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-08b97ac48e48a161.rlib" "-lgcc" "-lfdt" "-Wl,-Bdynamic" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_sw.o): In function `fdt_property':
          (.text+0x4d4): undefined reference to `__stack_chk_guard'
          (.text+0x4e0): undefined reference to `__stack_chk_guard'
          (.text+0x518): undefined reference to `__stack_chk_guard'
          (.text+0x530): undefined reference to `__stack_chk_fail'
          <ignored output lines>
          collect2: error: ld returned 1 exit status

Meta

rustc --version --verbose:

rustc 1.48.0 (7eac88abb 2020-11-16)
binary: rustc
commit-hash: 7eac88abb2e57e752f3302f02be5f3ce3d7adfb4
commit-date: 2020-11-16
host: aarch64-unknown-linux-gnu
release: 1.48.0
LLVM version: 11.0

Issue is present with nightly as well:

rustc 1.50.0-nightly (0f6f2d681 2020-12-06)
binary: rustc
commit-hash: 0f6f2d681b39c5f95459cd09cb936b6ceb27cd82
commit-date: 2020-12-06
host: aarch64-unknown-linux-gnu
release: 1.50.0-nightly

The error is specific to aarch64-unknown-linux-musl. The issue is not present when compiling for aarch64-unknown-linux-gnu.

System info:

$ uname -a
Linux cd2a1aab8486 4.15.0-1088-aws #93~16.04.1-Ubuntu SMP Wed Nov 18 01:34:22 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

$ apt-cache policy libfdt-dev
libfdt-dev:
  Installed: 1.4.5-3
  Candidate: 1.4.5-3
  Version table:
 *** 1.4.5-3 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main arm64 Packages
        100 /var/lib/dpkg/status

Error output

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-Wl,--eh-frame-hdr" "-nostartfiles" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crt1.o" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crti.o" "-L" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib" "-L" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained" "/firecracker/build/cargo_target/aarch64-unknown-linux-musl/release/deps/firecracker-8bf1ef43b7fbfc39.firecracker.5v4bbyky-cgu.11.rcgu.o" "-o" "/firecracker/build/cargo_target/aarch64-unknown-linux-musl/release/deps/firecracker-8bf1ef43b7fbfc39" "-Wl,--gc-sections" "-static" "-no-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/firecracker/build/cargo_target/aarch64-unknown-linux-musl/release/deps" "-L" "/firecracker/build/cargo_target/release/deps" "-L" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/tmp/rustcRAyWtZ/libunwind-1271d5b7d3b7bb97.rlib" "/tmp/rustcRAyWtZ/liblibc-99a9d4afa9ae21ce.rlib" "-Wl,--end-group" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-08b97ac48e48a161.rlib" "-lgcc" "-lfdt" "-Wl,-Bdynamic" "/usr/local/rust/toolchains/1.48.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_sw.o): In function `fdt_property':
          (.text+0x4d4): undefined reference to `__stack_chk_guard'
          (.text+0x4e0): undefined reference to `__stack_chk_guard'
          (.text+0x518): undefined reference to `__stack_chk_guard'
          (.text+0x530): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_sw.o): In function `fdt_finish':
          (.text+0x550): undefined reference to `__stack_chk_guard'
          (.text+0x55c): undefined reference to `__stack_chk_guard'
          (.text+0x628): undefined reference to `__stack_chk_guard'
          (.text+0x640): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_rw.o): In function `fdt_set_name':
          (.text+0x5c4): undefined reference to `__stack_chk_guard'
          (.text+0x5d0): undefined reference to `__stack_chk_guard'
          (.text+0x658): undefined reference to `__stack_chk_guard'
          (.text+0x670): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_rw.o): In function `fdt_setprop_placeholder':
          (.text+0x694): undefined reference to `__stack_chk_guard'
          (.text+0x6a0): undefined reference to `__stack_chk_guard'
          (.text+0x714): undefined reference to `__stack_chk_guard'
          (.text+0x728): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_rw.o): In function `fdt_setprop':
          (.text+0x79c): undefined reference to `__stack_chk_guard'
          (.text+0x7a8): undefined reference to `__stack_chk_guard'
          (.text+0x7e4): undefined reference to `__stack_chk_guard'
          (.text+0x7fc): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_rw.o): In function `fdt_appendprop':
          (.text+0x81c): undefined reference to `__stack_chk_guard'
          (.text+0x828): undefined reference to `__stack_chk_guard'
          (.text+0x8cc): undefined reference to `__stack_chk_guard'
          (.text+0x8e4): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_rw.o): In function `fdt_delprop':
          (.text+0x940): undefined reference to `__stack_chk_guard'
          (.text+0x94c): undefined reference to `__stack_chk_guard'
          (.text+0x98c): undefined reference to `__stack_chk_guard'
          (.text+0x9a0): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_rw.o): In function `fdt_add_subnode_namelen':
          (.text+0x9e0): undefined reference to `__stack_chk_guard'
          (.text+0x9ec): undefined reference to `__stack_chk_guard'
          (.text+0xae8): undefined reference to `__stack_chk_guard'
          (.text+0xafc): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_rw.o): In function `fdt_open_into':
          (.text+0xbe0): undefined reference to `__stack_chk_guard'
          (.text+0xbec): undefined reference to `__stack_chk_guard'
          (.text+0xca4): undefined reference to `__stack_chk_guard'
          (.text+0xcbc): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt.o): In function `fdt_next_node':
          (.text+0x2a4): undefined reference to `__stack_chk_guard'
          (.text+0x2b4): undefined reference to `__stack_chk_guard'
          (.text+0x324): undefined reference to `__stack_chk_guard'
          (.text+0x33c): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt.o): In function `fdt_first_subnode':
          (.text+0x3a0): undefined reference to `__stack_chk_guard'
          (.text+0x3a4): undefined reference to `__stack_chk_guard'
          (.text+0x3d0): undefined reference to `__stack_chk_guard'
          (.text+0x3e4): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt.o): In function `fdt_next_subnode':
          (.text+0x400): undefined reference to `__stack_chk_guard'
          (.text+0x418): undefined reference to `__stack_chk_guard'
          (.text+0x460): undefined reference to `__stack_chk_guard'
          (.text+0x474): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `_nextprop':
          (.text+0x20): undefined reference to `__stack_chk_guard'
          (.text+0x30): undefined reference to `__stack_chk_guard'
          (.text+0x74): undefined reference to `__stack_chk_guard'
          (.text+0x8c): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_subnode_offset_namelen':
          (.text+0x194): undefined reference to `__stack_chk_guard'
          (.text+0x1a0): undefined reference to `__stack_chk_guard'
          (.text+0x268): undefined reference to `__stack_chk_guard'
          (.text+0x280): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_get_phandle':
          (.text+0x624): undefined reference to `__stack_chk_guard'
          (.text+0x630): undefined reference to `__stack_chk_guard'
          (.text+0x690): undefined reference to `__stack_chk_guard'
          (.text+0x6a4): undefined reference to `__stack_chk_fail'
          (.text+0x6bc): undefined reference to `__stack_chk_guard'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_get_path':
          (.text+0x934): undefined reference to `__stack_chk_guard'
          (.text+0x940): undefined reference to `__stack_chk_guard'
          (.text+0x980): undefined reference to `__stack_chk_guard'
          (.text+0x998): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_supernode_atdepth_offset':
          (.text+0xad8): undefined reference to `__stack_chk_guard'
          (.text+0xae4): undefined reference to `__stack_chk_guard'
          (.text+0xb6c): undefined reference to `__stack_chk_guard'
          (.text+0xb80): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_node_depth':
          (.text+0xbcc): undefined reference to `__stack_chk_guard'
          (.text+0xbd0): undefined reference to `__stack_chk_guard'
          (.text+0xbf4): undefined reference to `__stack_chk_guard'
          (.text+0xc14): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_node_offset_by_prop_value':
          (.text+0xc74): undefined reference to `__stack_chk_guard'
          (.text+0xc80): undefined reference to `__stack_chk_guard'
          (.text+0xcd4): undefined reference to `__stack_chk_guard'
          (.text+0xcec): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_stringlist_count':
          (.text+0xe7c): undefined reference to `__stack_chk_guard'
          (.text+0xe84): undefined reference to `__stack_chk_guard'
          (.text+0xea4): undefined reference to `__stack_chk_guard'
          (.text+0xebc): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_stringlist_search':
          (.text+0xf24): undefined reference to `__stack_chk_guard'
          (.text+0xf30): undefined reference to `__stack_chk_guard'
          (.text+0xf58): undefined reference to `__stack_chk_guard'
          (.text+0xf70): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_stringlist_get':
          (.text+0x1014): undefined reference to `__stack_chk_guard'
          (.text+0x1020): undefined reference to `__stack_chk_guard'
          (.text+0x10b4): undefined reference to `__stack_chk_guard'
          (.text+0x10cc): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_ro.o): In function `fdt_node_check_compatible':
          (.text+0x10fc): undefined reference to `__stack_chk_guard'
          (.text+0x1104): undefined reference to `__stack_chk_guard'
          (.text+0x112c): undefined reference to `__stack_chk_guard'
          (.text+0x1140): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_wip.o): In function `fdt_setprop_inplace_namelen_partial':
          (.text+0xc): undefined reference to `__stack_chk_guard'
          (.text+0x18): undefined reference to `__stack_chk_guard'
          (.text+0x44): undefined reference to `__stack_chk_guard'
          (.text+0x58): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_wip.o): In function `fdt_setprop_inplace':
          (.text+0xa4): undefined reference to `__stack_chk_guard'
          (.text+0xb0): undefined reference to `__stack_chk_guard'
          (.text+0xec): undefined reference to `__stack_chk_guard'
          (.text+0x100): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_wip.o): In function `fdt_nop_property':
          (.text+0x164): undefined reference to `__stack_chk_guard'
          (.text+0x168): undefined reference to `__stack_chk_guard'
          (.text+0x18c): undefined reference to `__stack_chk_guard'
          (.text+0x1a0): undefined reference to `__stack_chk_fail'
          /usr/lib/gcc/aarch64-linux-gnu/7/../../../aarch64-linux-gnu/libfdt.a(fdt_wip.o): In function `_fdt_node_end_offset':
          (.text+0x1d8): undefined reference to `__stack_chk_guard'
          (.text+0x1f0): undefined reference to `__stack_chk_guard'
          (.text+0x21c): undefined reference to `__stack_chk_guard'
          (.text+0x230): undefined reference to `__stack_chk_fail'
          collect2: error: ld returned 1 exit status

Activity

added
C-bugCategory: This is a bug.
I-ICEIssue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️
T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
on Dec 7, 2020
added
A-linkageArea: linking into static, shared libraries and binaries
O-muslTarget: The musl libc
on Dec 7, 2020
LeSeulArtichaut

LeSeulArtichaut commented on Dec 7, 2020

@LeSeulArtichaut
Contributor

Did this work before 1.48.0?

added
O-ArmTarget: 32-bit Arm processors (armv6, armv7, thumb...), including 64-bit Arm in AArch32 state
and removed
O-ArmTarget: 32-bit Arm processors (armv6, armv7, thumb...), including 64-bit Arm in AArch32 state
on Dec 7, 2020
acatangiu

acatangiu commented on Dec 7, 2020

@acatangiu
Author

Yes, it works fine with 1.47.0 and lower

added
I-prioritizeIssue: Indicates that prioritization has been requested for this issue.
on Dec 7, 2020
removed
I-ICEIssue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️
on Dec 7, 2020
camelid

camelid commented on Dec 7, 2020

@camelid
Member

(This isn't an ICE.)

added
P-highHigh priority
and removed
I-prioritizeIssue: Indicates that prioritization has been requested for this issue.
on Dec 9, 2020
apiraino

apiraino commented on Dec 9, 2020

@apiraino
Contributor

@rustbot ping cleanup

also @acatangiu I think it would interesting to have a smaller example to reproduce this. Are you straight compiling Firecracker?

rustbot

rustbot commented on Dec 9, 2020

@rustbot
Collaborator
added
E-needs-mcveCall for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example
on Dec 9, 2020
dillona

dillona commented on Dec 9, 2020

@dillona
Contributor

searched nightlies: from nightly-2020-07-27 to nightly-2020-12-08
regressed nightly: nightly-2020-08-01
searched commits: from cfc572c to 6e87bac
regressed commit: c058a8b

bisected with cargo-bisect-rustc v0.6.0

Host triple: aarch64-unknown-linux-gnu
Reproduce with:

cargo bisect-rustc aarch64-unknown-linux-musl -- build --target aarch64-unknown-linux-musl
peterwaller-arm

peterwaller-arm commented on Mar 24, 2022

@peterwaller-arm

Driveby comment, I perceive from the error message libfdt is from aarch64-linux-gnu and the linker being invoked is cc. Is your configuration correct? It seems you need to specify your linker as musl-gcc, or similar, to have the musl libraries linked (and not the GNU ones). See also:
#90846 (comment), #94864

acatangiu

acatangiu commented on Mar 25, 2022

@acatangiu
Author

Driveby comment, I perceive from the error message libfdt is from aarch64-linux-gnu and the linker being invoked is cc. Is your configuration correct? It seems you need to specify your linker as musl-gcc, or similar, to have the musl libraries linked (and not the GNU ones). See also: #90846 (comment), #94864

This works, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-linkageArea: linking into static, shared libraries and binariesC-bugCategory: This is a bug.E-needs-mcveCall for participation: This issue has a repro, but needs a Minimal Complete and Verifiable ExampleICEBreaker-Cleanup-CrewHelping to "clean up" bugs with minimal examples and bisectionsO-muslTarget: The musl libcP-highHigh priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @dillona@apiraino@acatangiu@camelid@LeSeulArtichaut

        Issue actions

          Linker error/regression introduced with rustc 1.48.0 on `aarch64-unknown-linux-musl` target · Issue #79791 · rust-lang/rust