Skip to content

rustc_trans: Link archives with --whole-archive #19183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

klutzy
Copy link
Contributor

@klutzy klutzy commented Nov 21, 2014

If archives are passed to linker, linker does not add all object files
but instead scans each object in archive then adds it only if it
defines some symbols which has been undefined.
It is problematic when rlib needs to be contained in dylib, since
some portion of rlib is stripped out if unused by dylib itself.

This patch uses --whole-archive flag which changes linker behavior
that the linker just adds all objects in archive.
Unfortunately, ld does not accept rlib as-is because it contains
non-object files, so they are manually removed before passed to ld.

Also, this change can be problematic if several archives actually
contain same symbol. For example, this patch breaks code like:

    #![crate_type = "rlib"]

    // `libcfoo.a` contains `foo`.
    #![link(name = "cfoo", kind = "static")]
    extern {}

    // `foo` is also defined here!
    #[no_mangle]
    pub fn foo() {}

    // so this rlib archive has two `foo` symbols.

Previously it worked because linker stripped out cfoo.

Fixes #14344

If archives are passed to linker, linker does not add all object files
but instead scans each object in archive then adds it only if it
defines some symbols which has been undefined.
It is problematic when rlib needs to be contained in dylib, since
some portion of rlib is stripped out if unused by dylib itself.

This patch uses `--whole-archive` flag which changes linker behavior
that the linker just adds all objects in archive.
Unfortunately, ld does not accept rlib as-is because it contains
non-object files, so they are manually removed before passed to ld.

Also, this change can be problematic if several archives actually
contain same symbol. For example, this patch breaks code like:

```Rust
    #![crate_type = "rlib"]

    // `libcfoo.a` contains `foo`.
    #![link(name = "cfoo", kind = "static")]
    extern {}

    // `foo` is also defined here!
    #[no_mangle]
    pub fn foo() {}

    // so this rlib archive has two `foo` symbols.
```

Previously it worked because linker stripped out `cfoo`.

Fixes rust-lang#14344
@alexcrichton
Copy link
Member

Can you profile the compile time of a before/after hello world? In the past I've seen the "altering rlib" step take an inordinately large amount of time compared to just passing it through to the linker. I'd want to make sure that our link times don't regress too much as a result of this change.

Additionally, The --whole-archive flag is -force_load on OSX, so you may need some platform-specific detection in that regard.

An alternative solution would be to generate references to symbols to all upstream libraries from within the object file itself. That would guarantee that the linker would at least consider all rlib object files, and we'd rely on --gc-sections to strip out all the extra cruft (I can go into more detail on this if you'd like).

@klutzy
Copy link
Contributor Author

klutzy commented Nov 27, 2014

Can you profile the compile time of a before/after hello world? In the past I've seen the "altering rlib" step take an inordinately large amount of time compared to just passing it through to the linker. I'd want to make sure that our link times don't regress too much as a result of this change.

time: 0.003     LLVM passes
  time: 0.021   altering a.rlib
  time: 0.098   altering std-4e7c5e5c.rlib
  time: 0.104   altering sync-4e7c5e5c.rlib
  time: 0.017   altering rustrt-4e7c5e5c.rlib
  time: 0.023   altering collections-4e7c5e5c.rlib
  time: 0.019   altering alloc-4e7c5e5c.rlib
  time: 0.015   altering libc-4e7c5e5c.rlib
  time: 0.016   altering rand-4e7c5e5c.rlib
  time: 0.026   altering unicode-4e7c5e5c.rlib
  time: 0.052   altering core-4e7c5e5c.rlib
  time: 0.167   running linker
time: 0.564     linking

So linking time has increased 0.167 -> 0.564, and it would be more if there are more dependencies. Right now I'm not sure if this is a right approach..

I also started to think this is not a right approach for static C libraries, since it may be intended that unused symbols are stripped out; the alternative solution seems more plausible since it would not affect non-rlib libraries.

@klutzy klutzy closed this Nov 27, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Linking an rlib with a dylib may not actually link the rlib
2 participants