Skip to content

Linking error on Rust 1.70 aarch64-unknown-linux-musl toolchain #113104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
geauxvirtual opened this issue Jun 27, 2023 · 35 comments
Open

Linking error on Rust 1.70 aarch64-unknown-linux-musl toolchain #113104

geauxvirtual opened this issue Jun 27, 2023 · 35 comments
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug. P-high High priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@geauxvirtual
Copy link

I have a project that currently compiles and links with Rust 1.67 on the following toolchains:

  • x86_64-unknown-linux-musl (built in rust:1.67-alpine3.17)
  • aarch64-unknown-linux-musl (built in rust:1.67-alpine3.17)
  • aarch64-apple-darwin

The project successfully builds with all three toolchains for Rust 1.69.

I was attempting to build the project with Rust 1.70. The project compiles successfully on the following toolchains:

  • x86_64-unknown-linux-musl (built in rust:1.70-alpine.3.17 and rust:1.70-alpine3.18)
  • aarch64-apple-darwin

The project fails to compile and link on the following toolchain:

  • aarch64-unknown-linux-musl (fails in rust:1.70-alpine3.17 and rust:1.70-alpine3.18)

I've been trying to track down which dependency of the project could be causing the following errors to be thrown since the project has no specific hardware requirements, but I also wanted to open an issue since this is failing for one specific toolchain starting with 1.70.

Building with --verbose does not yield any additional information. This occurs for both debug and release builds.

error: linking with `cc` failed: exit status: 1
= note: /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: GFp_armcap_P: undefined version:
        /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: failed to set dynamic section sizes: bad value
        collect2: error: ld returned 1 exit status
@saethlin
Copy link
Member

I think this is another manifestation of whatever is causing #111888

The root cause predates my alignment checks PR, but the non-unwinding panics that I added provoke it more often.

@jyn514 jyn514 added A-linkage Area: linking into static, shared libraries and binaries regression-from-stable-to-stable Performance or correctness regression from one stable version to another. O-musl Target: The musl libc C-bug Category: This is a bug. O-AArch64 Armv8-A or later processors in AArch64 mode labels Jun 28, 2023
@rustbot rustbot added the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label Jun 28, 2023
@apiraino
Copy link
Contributor

apiraino commented Jun 28, 2023

WG-prioritization assigning priority (Zulip discussion) and nominating for T-compiler meeting to try getting more eyes on this class of linkage error we're experiencing since 1.70

@rustbot label -I-prioritize +P-high I-compiler-nominated +T-compiler

@rustbot rustbot added I-compiler-nominated Nominated for discussion during a compiler team meeting. P-high High priority T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Jun 28, 2023
@pnkfelix
Copy link
Member

I've carved out some time for this today and tomorrow.

@apiraino
Copy link
Contributor

Discussed in T-compiler triage meeting on Zulip (notes).

@rustbot label -I-compiler-nominated

@rustbot rustbot removed the I-compiler-nominated Nominated for discussion during a compiler team meeting. label Jun 29, 2023
@pnkfelix
Copy link
Member

pnkfelix commented Jul 14, 2023

@geauxvirtual I'm feeling dense: Can you provide a link to a reproduction, even its just the whole project you are trying to build?

I.e. you say "the project" many times in the bug description, but there's no link.

If its not open-source, then without a separate minimized reproduction provided, we won't be able to ensure that whatever fix we put in actually addresses your problem.

@pnkfelix pnkfelix added the E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example label Jul 14, 2023
@geauxvirtual
Copy link
Author

I'll see if I can put together a minimal reproduction today because the project isn't open source.

@geauxvirtual
Copy link
Author

I'm still working on trying to produce a minimal reproducible version of the issue, but in the process, I have been able to get the project to build on aarch64-unknown-linux-musl (rust:1.70-alpine3.18) by changing how some dependencies were being included (and removing some dependencies that weren't being used) in the crate that was failing to link. I know above someone had mentioned it's possibly related to ring, but I haven't be able to confirm that. However by removing a dependency from the crate that was including sys-info, and adding mysql_async to the crate instead of re-exporting it through a second crate, everything compiles fine now, including all the other crates that include the dependency that has sys-info as a dependency.

Example dependency structure below from the way the project compiled fine up to Rust 1.69 on aarch64-unknown-linux-musl, to the small changes I made to now get it to compile with Rust 1.70 on aarch64-unknown-linux-musl. The peculiar part below is that Crate B, Crate C, and Crate D compiled and linked fine. Crate A was only using a re-export of mysql_async from Crate B. The only other info about Crate A is that it's used to define some procedural macros, and I couldn't find anything from syn, quote, and proc-macro2 with regards to issues on Rust 1.70 and aarch64-unknown-linux-musl.

Project A (dependency structure failing to link)
    |-- Crate A (failing to link)
         |-- Crate B
    |-- Crate B
         |-- Crate C
         |-- mysql_async
    |-- Crate C
         |-- Crate D
              |-- sys-info

Project A (dependency structure that builds and links successfully)
    |-- Crate A
         |-- mysql_async
    |-- Crate B
         |-- Crate C
         |-- mysql_async
    |-- Crate C
         |-- Crate D
              |-- sys-info

@badboy
Copy link
Member

badboy commented Jul 21, 2023

After hitting that locally I'm now also getting this in CI for Firefox, targetting x86_64-unknown-linux-gnu.
I'll get you more details shortly and see if I can reproduce with anything simpler.

@badboy
Copy link
Member

badboy commented Jul 24, 2023

I have yet to reproduce that in anything outside of mozilla-central (the firefox repository).
Running the following on a checkout of https://hg.mozilla.org/try/rev/ca38fbc66ca72744143d604e66810f3b47094189 produces the error. Not sure how helpful this is.

cargo build --manifest-path /home/jer/src/mozilla/gecko/Cargo.toml --package uniffi-bindgen-gecko-js

Full details:

$ rustc -vV
rustc 1.71.0 (8ede3aae2 2023-07-12)
binary: rustc
commit-hash: 8ede3aae28fe6e4d52b38157d7bfe0d3bceef225
commit-date: 2023-07-12
host: x86_64-unknown-linux-gnu
release: 1.71.0
LLVM version: 16.0.5

Error:

$ cargo build --manifest-path /home/jer/src/mozilla/gecko/Cargo.toml --package uniffi-bindgen-gecko-js
    Updating crates.io index
   Compiling uniffi_macros v0.24.1
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/jer/.pyenv/shims:/home/jer/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/store/9c03r86hcdn43dm3hsgjirifvyzfkhwh-python3-3.10.12/bin:/nix/store/ngpj89qvilv4n1li30l2zl7jifnzf7z0-tmuxp-1.28.1/bin:/nix/store/prijbwb5cgwbznzganfwjdpkv8xd4a2g-python3.10-kaptan-0.5.12/bin:/home/jer/.pyenv/bin:/home/jer/bin:/home/jer/.cargo/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/jer/.dotnet/tools:/var/lib/snapd/snap/bin:/home/jer/.gem/ruby/2.7.0/bin:/home/jer/code/go/bin:/home/jer/.mozbuild/git-cinnabar:/home/jer/Android/android-studio/bin:/home/jer/Android/Sdk/platform-tools:/home/jer/.local/bin:/home/jer/.fly/bin" VSLANG="1033" "cc" "-Wl,--version-script=/tmp/rustc62ohwS/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustc62ohwS/symbols.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.0.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.1.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.10.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.11.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.12.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.13.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.14.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.15.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.2.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.3.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.4.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.5.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.6.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.7.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.8.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.uniffi_macros.c09f0bf8631e3ca0-cgu.9.rcgu.o" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.1ujverqzotmpqsgs.rcgu.rmeta" "/home/jer/src/mozilla/gecko/target/debug/deps/uniffi_macros-1ebbc199f08291aa.25z4o1bn5v109hju.rcgu.o" "-Wl,--as-needed" "-L" "/home/jer/src/mozilla/gecko/target/debug/deps" "-L" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/jer/src/mozilla/gecko/target/debug/deps/libuniffi_build-896d013fdb7375b1.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libuniffi_bindgen-1b8b63ef5d9a34f8.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libglob-a85420e1f74a62ca.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libgoblin-debb8ebc56382946.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libplain-60abe002dc486d7e.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libscroll-6987499de7d462ad.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libweedle-10a64121bbcada62.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libnom-5b089e1cd9825d51.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libmemchr-77ecca6e3d8530b9.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libuniffi_testing-900a279d1948846c.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libcargo_metadata-1bf75b2a3434b0b6.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libserde_json-7e5f4df2c6670659.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libryu-bf1761a090e0d244.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libitoa-31083e990354a5eb.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libthiserror-d69bce00f6aad8f1.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libcargo_platform-c0e918f0d912cdee.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libsemver-3add90b556d5ea7b.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libheck-5d274367d6cd4a11.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libaskama-139d13e0e052591b.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libaskama_escape-cee00719578c85c2.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libtoml-758df12d318006a3.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libonce_cell-1eddf36a5857174d.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libfs_err-ed8775f3ced91ffe.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libuniffi_meta-a9272bac2e3b0f6c.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libsiphasher-3f19e0de6b638cbe.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libuniffi_core-7fa367cf19af87f2.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libstatic_assertions-09bb1be294d27063.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/liblog-10ad0f5a1f418df9.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libcfg_if-c5cd52f2f953356d.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libbytes-2260469161279e09.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libanyhow-5a625291d2b834e9.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libsyn-64ffbf436832c4c1.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libquote-bb589a7647e0b23f.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libproc_macro2-b0dba7c929f125c2.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libunicode_ident-a248ba30253e17af.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libproc_macro-7ede36ba902b7649.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libcamino-1ab45bb9e9e73468.rlib" "/home/jer/src/mozilla/gecko/target/debug/deps/libserde-fe0cfa60f053fdab.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-0d91c78a7710ed2e.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-9bfeb974ba4dc4e7.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-7dd2d47ddb5fff81.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-18d5ce8e8a320b85.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-69d2770595dc6161.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-b156a020cc470e38.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-84091f15e468b1ee.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-fbde6acb28b510ca.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e9de5d8ee4e7a3dd.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-9023252e4f119830.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-fcbb01769e88af40.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-f69e84994e245fea.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-27b55c02caca49ea.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-579b26075cbe9eca.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-54c30397d4b33d3b.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-65fb576691133eee.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-3fff6412017c0b89.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-fd3918c72578db43.rlib" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-2a597573799b576f.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/jer/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/jer/src/mozilla/gecko/target/debug/deps/libuniffi_macros-1ebbc199f08291aa.so" "-Wl,--gc-sections" "-shared" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: /usr/bin/ld: uniffi_rustbuffer_reserve: undefined version:
          /usr/bin/ld: uniffi_rustbuffer_from_bytes: undefined version:
          /usr/bin/ld: uniffi_rustbuffer_free: undefined version:
          /usr/bin/ld: uniffi_rustbuffer_alloc: undefined version:
          /usr/bin/ld: uniffi_foreign_executor_callback_set: undefined version:
          /usr/bin/ld: failed to set dynamic section sizes: bad value
          collect2: error: ld returned 1 exit status

uniffi_rustbuffer_from_bytes and the others are no_mangle extern "C" functions in the uniffi_core create, which is nested somewhere in the dep tree there. Those symbols should not be used in that part of the binary anyway.
Indeed the solution to make this compile is dropping the uniffi dependency of the uniffi-bindgen-gecko-js crate.

@badboy
Copy link
Member

badboy commented Jul 24, 2023

Still no small reproduction, but I extracted the crate and tested it in a clean Docker environment too: https://github.com/badboy/uniffi-bindgen-gecko-js-rustc-bug/

Note: removing the profile.dev.build-override in the Cargo.toml makes this compile just fine then.

@glandium
Copy link
Contributor

Still no small reproduction, but I extracted the crate and tested it in a clean Docker environment too: https://github.com/badboy/uniffi-bindgen-gecko-js-rustc-bug/

What's happening on this one is that rust creates a symbol versioning script for the uniffi-macros proc macro with the contents:

{
  global:
    __rustc_proc_macro_decls_cf019372984507f5__;
    rust_metadata_uniffi_macros_cf019372984507f5;
    uniffi_foreign_executor_callback_set;
    uniffi_rustbuffer_alloc;
    uniffi_rustbuffer_free;
    uniffi_rustbuffer_from_bytes;
    uniffi_rustbuffer_reserve;

  local:
    *;
};

IOW, it wants to export those symbols, although they are irrelevant for a proc macro.

@glandium
Copy link
Contributor

Fundamentally, though, the uniffi problem is that the proc macro is depending on uniffi_core, via uniffi_meta, and uniffi_meta only needs the metadata module of uniffi_core, but uniffi_core contains much more stuff, including those ffi exports. If that module was a separate crate (or duplicated in uniffi_meta), the problem wouldn't happen.

All in all the uniffi problem may or may not be related to this issue.

@saethlin
Copy link
Member

That sounds exactly like another source of these linker errors: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/linker.20errs.20from.20wasm.20exposed.20by.20mir.20align.20checks.20.23111888

I'm more convinced now that there is just one bug here.

@badboy
Copy link
Member

badboy commented Jul 25, 2023

Fundamentally, though, the uniffi problem is that the proc macro is depending on uniffi_core, via uniffi_meta, and uniffi_meta only needs the metadata module of uniffi_core, but uniffi_core contains much more stuff, including those ffi exports. If that module was a separate crate (or duplicated in uniffi_meta), the problem wouldn't happen.

All in all the uniffi problem may or may not be related to this issue.

True.
Though as mentioned in #111888 there were some changes adding no-undefined-version to the linker invocation and manually stripping that out makes my builds work, which makes me believe it's more or less the same underlying issue here (we will work around this issue in some other way in m-c)

@pnkfelix
Copy link
Member

pnkfelix commented Aug 9, 2023

@geauxvirtual can you check if the problem is resolved in the latest nightly, now that #114470 has landed?

(or maybe @badboy can check against the reproducer that they concocted...)

@geauxvirtual
Copy link
Author

I just tried building against the latest nightly, and the same linking error above is seen with the latest nightly compiler.

de-sh pushed a commit to bytebeamio/uplink that referenced this issue Aug 15, 2023
@mmastrac
Copy link

We just got a report of a similar bug building Deno on aarch64 using Ubuntu 16.04 (glibc 2.26.1):

#18 179.0    Compiling deno_cache_dir v0.5.0
#18 180.5 error: linking with `cc` failed: exit status: 1
#18 180.5   |
#18 180.5   = note: LC_ALL="C" PATH="/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/bin:/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "cc" "-Wl,--version-script=/tmp/rustcRUJo55/list" "-Wl,--no-undefined-version" "/tmp/rustcRUJo55/symbols.o" "/deno/target/release/deps/deno_cache_dir-8cd1e1be67d058cd.deno_cache_dir.bf5665095537e8b5-cgu.0.rcgu.o" "/deno/target/release/deps/deno_cache_dir-8cd1e1be67d058cd.4qp5mhu02gfvfcfl.rcgu.o" "-Wl,--as-needed" "-L" "/deno/target/release/deps" "-L" "/deno/target/release/build/ring-6c8cedad92a22f15/out" "-L" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/deno/target/release/deps/libserde_json-e03c70682fa7e370.rlib" "/deno/target/release/deps/libryu-f5b0e8b3015522f8.rlib" "/deno/target/release/deps/libitoa-0d042663236bb011.rlib" "/deno/target/release/deps/liblog-fa937a2f3f3ab897.rlib" "/deno/target/release/deps/libparking_lot-d4926d1f42203871.rlib" "/deno/target/release/deps/libparking_lot_core-450cdaa49b2afb0a.rlib" "/deno/target/release/deps/libcfg_if-38889dd09f4a491e.rlib" "/deno/target/release/deps/libsmallvec-6d1aebb7a8998e19.rlib" "/deno/target/release/deps/liblock_api-d7d91fcb547223e4.rlib" "/deno/target/release/deps/libscopeguard-f50f1aef82cc20bf.rlib" "/deno/target/release/deps/libindexmap-e6bad887b381ff97.rlib" "/deno/target/release/deps/libequivalent-8cb2cf8bd8c60702.rlib" "/deno/target/release/deps/libhashbrown-5b842ca54ce03c1f.rlib" "/deno/target/release/deps/libdeno_media_type-0ef0bf74110a6814.rlib" "/deno/target/release/deps/libdata_url-090a63930aa396ce.rlib" "/deno/target/release/deps/libthiserror-02a335d27a569383.rlib" "/deno/target/release/deps/libring-5a3bcffb9fc73ee5.rlib" "/deno/target/release/deps/libspin-1129cfebe598ed67.rlib" "/deno/target/release/deps/libuntrusted-a5dcc30bcb4f8a25.rlib" "/deno/target/release/deps/libonce_cell-980fd0d198baddfa.rlib" "/deno/target/release/deps/liblibc-abb1a91c0c67b773.rlib" "/deno/target/release/deps/liburl-12800c7d33fe26d0.rlib" "/deno/target/release/deps/libidna-426e9ae2e2003357.rlib" "/deno/target/release/deps/libunicode_normalization-64d503e613bf6c92.rlib" "/deno/target/release/deps/libtinyvec-5655d95fed51f64c.rlib" "/deno/target/release/deps/libtinyvec_macros-cfde70f85c067b2a.rlib" "/deno/target/release/deps/libunicode_bidi-3264b114da7c7b69.rlib" "/deno/target/release/deps/libform_urlencoded-d0e256eac59b13e6.rlib" "/deno/target/release/deps/libpercent_encoding-b9b191ae89425e92.rlib" "/deno/target/release/deps/libserde-6c2a55a9c90e51d6.rlib" "/deno/target/release/deps/libanyhow-da72a87212c1a104.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd-49f405f32420fd6a.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libpanic_unwind-2b2d0fc0b0e1e1ec.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libobject-7cc15c5f0c23404e.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libmemchr-424f9a552277b8de.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libaddr2line-be9e9c3394ea9d59.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libgimli-650021992b21b3a2.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_demangle-f3c8c0feb6f7f8aa.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd_detect-d7ab6e8d31d95747.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libhashbrown-29bef78daca585c7.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-0bb4fc54e64e1928.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libminiz_oxide-dcacb73e0fc82c6c.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libadler-5978245ddc0addd7.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libunwind-fd9fde115279cb3c.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcfg_if-50edd0de903dc776.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-74bb96f0ab3e26e0.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/liballoc-60a4032889f46560.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_core-140890b14fd90465.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcore-67fe04957e868f3a.rlib" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcompiler_builtins-f261afcc57c36219.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/root/.rustup/toolchains/1.71.0-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-o" "/deno/target/release/deps/libdeno_cache_dir-8cd1e1be67d058cd.so" "-Wl,--gc-sections" "-shared" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
#18 180.5   = note: /usr/bin/ld: GFp_armcap_P: undefined version:
#18 180.5           /usr/bin/ld: failed to set dynamic section sizes: Bad value

@saethlin
Copy link
Member

The symbol with a missing version is in ring, which is a direct dependency of deno_cache_dir, so I suspect this is not the same issue that @pnkfelix recently fixed with proc macros. Following up on the deno issue.

@saethlin
Copy link
Member

saethlin commented Aug 20, 2023

I am looking into this. Mostly hoping to get an MCVE out of ring.

@saethlin saethlin self-assigned this Aug 20, 2023
@saethlin saethlin removed O-musl Target: The musl libc O-AArch64 Armv8-A or later processors in AArch64 mode labels Aug 20, 2023
@pnkfelix
Copy link
Member

After some awesome effort from @saethlin to produce an MCVE, it looks to me like the MCVE itself is fixed by #115114 (which ended up in the nightly-2023-08-24).... and PR #115114 does seem like something that could have an impact on this kind of thing...

so ... I hate to ask again but ... can @geauxvirtual and @mmastrac check if their problem still occurs against the latest nightly? :)

@mmastrac
Copy link

I was able to successfully build with both cargo +nightly build --release and cargo +nightly build in the docker container from https://github.com/LukeChannings/deno-arm64

@geauxvirtual
Copy link
Author

I was able to build my project with the latest nightly release.

@saethlin
Copy link
Member

My MCVE was a bit more fragile than I hoped, and it was "fixed" by the linked PR, but the deno-arm64 build was not fixed. I've improved the MCVE, it now still encounters the linker error on nightly; the trick is that deno_cache_dir has crate-type = ["lib", "cdylib"].

This Dockerfile will continue to reproduce the linker error on any host:

FROM ubuntu:16.04

ENV PATH=$PATH:/root/.cargo/bin/

RUN apt-get update
RUN apt-get install -y git gcc curl
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain=nightly --profile=minimal -y
RUN git clone https://github.com/saethlin/ring --branch=minimize && \
    cd ring && \
    ./build.sh

@saethlin saethlin removed the E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example label Sep 14, 2023
@saethlin saethlin removed their assignment Sep 14, 2023
@saethlin
Copy link
Member

I unassigned myself because I have provided the MCVE and in general these kinds of problems are beyond my skills, but I think there are people on the compiler team who are able to tackle this.

@wiiznokes
Copy link

I thinks i'm having the same error on 1.74:

= note: /usr/bin/ld: /home/lenaic/Documents/fan-control/target/debug/deps/libsensors_sys-9eee00e2dd017631.rlib(data.ao): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
          /usr/bin/ld: failed to set dynamic section sizes: bad value
          collect2: error: ld returned 1 exit status

let me know if you need more information

@saethlin
Copy link
Member

@wiiznokes Are you also hitting this only on Ubuntu 16? If not, I'd love to have a reproducer I can run so that I can minimize it to see if this is the same effect that this issue evolved into.

@wiiznokes
Copy link

@saethlin well, i was running a fresh install of Fedora 39. And this was this softwre, but it is quit big

@saethlin
Copy link
Member

Up-thread I minimized part of deno down to 12 lines of code, I'm not afraid. Yet.

@saethlin
Copy link
Member

It looks like this project only builds in debug mode. If I try to cargo run --release like BUILD.md says I get a handful of compile errors that I don't get with cargo build. Were you seeing this error in an unoptimized build? Or is there a particular commit I need to check out?

@saethlin
Copy link
Member

Wait a second, you said the error is:

relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE

which is very suspicious. Have you tried following the linker's advice? This looks like a mismatch between the flags used for a C or C++ project and a Rust project.

@wiiznokes
Copy link

I hadn't tested it, no. I just did this and I still get the same error. I really don't know if this is related to this issue. I don't know much in this level

@saethlin
Copy link
Member

The linker error that this issue is about looks like

note: ld: <some symbol name>: undefined version:
ld: failed to set dynamic section sizes: Bad value

Which is a different error message than you have. If your project builds on Rust 1.73 but not Rust 1.74, please open a new issue for the regression. If your project doesn't build on 1.70 but builds on 1.69 you might be seeing a very different manifestation of the underlying cause of this issue.

@briansmith
Copy link
Contributor

briansmith commented Nov 21, 2023

Hi friends. I just found this issue today. I wish somebody would have let me know about it before, so I could address it. I will be looking into it soon.

The thing that's special about this symbol is that it is defined as #[no_mangle] static mut GFp_armcap_P which gives it non-hidden visibility. This symbol is referenced from C code in a static library (ring-core) built/linked by ring's build.rs.

ring's build.rs doesn't pass '-fpie` but how would it know when it needs to do so? Edit: This is irrelevant and off-topic as those comments in this issue are about some other problem.

@briansmith
Copy link
Contributor

@saethlin wrote:

RUN git clone https://github.com/saethlin/ring --branch=minimize

Thanks. I appreciate you doing this minimization. I am planning to work around the issue in briansmith/ring#1808.

@briansmith
Copy link
Contributor

/usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: GFp_armcap_P: undefined version:
/usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: failed to set dynamic section sizes: bad value

Back on July 25th, @badboy wrote:

Though as mentioned in #111888 there were some changes adding no-undefined-version

Indeed, 1.70.0 was the first version that included the change in PR #108017, which adds --no-undefined-version to the linker command line. So that's very likely what broke this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug. P-high High priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests