Skip to content

Turn link-args into advanced-linking, add upstream crate linking and static linking #25685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/doc/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -1620,6 +1620,10 @@ The type of a function declared in an extern block is `extern "abi" fn(A1, ...,
An) -> R`, where `A1...An` are the declared types of its arguments and `R` is
the declared return type.

It is valid to add the `link` attribute on an empty extern block. You can use
this to satisfy the linking requirements of extern blocks elsewhere in your code
(including upstream crates) instead of adding the attribute to each extern block.

## Visibility and Privacy

These two terms are often used interchangeably, and what they are attempting to
Expand Down
2 changes: 1 addition & 1 deletion src/doc/trpl/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
* [No stdlib](no-stdlib.md)
* [Intrinsics](intrinsics.md)
* [Lang items](lang-items.md)
* [Link args](link-args.md)
* [Advanced linking](advanced-linking.md)
* [Benchmark Tests](benchmark-tests.md)
* [Box Syntax and Patterns](box-syntax-and-patterns.md)
* [Slice Patterns](slice-patterns.md)
Expand Down
150 changes: 150 additions & 0 deletions src/doc/trpl/advanced-linking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
% Advanced Linking

The common cases of linking with Rust have been covered earlier in this book,
but supporting the range of linking possibilities made available by other
languages is important for Rust to achieve seamless interaction with native
libraries.

# Link args

There is one other way to tell rustc how to customize linking, and that is via
the `link_args` attribute. This attribute is applied to `extern` blocks and
specifies raw flags which need to get passed to the linker when producing an
artifact. An example usage would be:

``` no_run
#![feature(link_args)]

#[link_args = "-foo -bar -baz"]
extern {}
# fn main() {}
```

Note that this feature is currently hidden behind the `feature(link_args)` gate
because this is not a sanctioned way of performing linking. Right now rustc
shells out to the system linker (`gcc` on most systems, `link.exe` on MSVC),
so it makes sense to provide extra command line
arguments, but this will not always be the case. In the future rustc may use
LLVM directly to link native libraries in which case `link_args` will have no
meaning. You can achieve the same effect as the `link-args` attribute with the
`-C link-args` argument to `rustc`.

It is highly recommended to *not* use this attribute, and rather use the more
formal `#[link(...)]` attribute on `extern` blocks instead.

# Static linking

Static linking refers to the process of creating output that contain all
required libraries and so don't need libraries installed on every system where
you want to use your compiled project. Rust libraries are statically linked by
default so you can use Rust-created binaries and libraries without installing
the Rust runtime everywhere. By contrast, native libraries are dynamically
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This last sentence doesn't quite 100% agree with reality I think because the "Rust runtime" doesn't actually exist. I think this paragraph may want to be re-worded a bit to emphasize that Rust dependencies are statically linked by default but dependencies like libc/librt/libm are all dynamically linked, and this section is specifically about statically linking only the system dependencies.

linked by default, but sometimes it can be useful to change this.

Linking is a very platform dependant topic - on some platforms, static linking
may not be possible at all! This section assumes some basic familiarity with
linking on your platform of choice.

## Linux

By default, all Rust programs on Linux will link to the system libc along with
a number of other libraries. Let's look at an example on a 64-bit linux machine
with GCC and glibc (by far the most common libc on Linux):

``` text
$ cat example.rs
fn main() {}
$ rustc example.rs
$ ldd example
linux-vdso.so.1 => (0x00007ffd565fd000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa81889c000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa81867e000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa818475000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa81825f000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa817e9a000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa818cf9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa817b93000)
```

Dynamic linking on Linux can be undesirable if you wish to target older
machines as applications compiled aginst newer versions glibc are not
guaranteed to run against older versions.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This statement may not quite be true as Rust doesn't really compile against a particular glibc because we don't use their headers. All Rust code using the standard library is compatible with glibc 2.18 by default, so most Rust programs are compatible with older glibc versions automatically.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's more targeted at external native libraries which may use exciting new features.
When I replace this section with musl I'll elaborate that it's possible (even likely) that external native libraries will need to be recompiled with musl to be linkable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked over this bit again. Here's what I'm talking about:

$ cat /etc/issue | head -n 1
Ubuntu 14.04.2 LTS \n \l
$ /lib/x86_64-linux-gnu/libc.so.6 | head -n 1
GNU C Library (Ubuntu EGLIBC 2.19-0ubuntu6.6) stable release version 2.19, by Roland McGrath et al.
$ cat example.rs 
fn main() {}
$ rustc example.rs
$ nm example | grep GLIBC | grep thread_atexit
                 w __cxa_thread_atexit_impl@@GLIBC_2.18
$ docker run -it --rm -v $(pwd):/r centos:centos6 bash
[root@6dec72837400 /]# cat /etc/issue | head -n 1
CentOS release 6.6 (Final)
[root@6dec72837400 /]# /lib64/libc.so.6 | head -n 1
GNU C Library stable release version 2.12, by Roland McGrath et al.
[root@6dec72837400 /]# /r/example
/r/example: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /r/example)
/r/example: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /r/example)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah that's somewhat different, it's that you linked against a newer version of libc but when running it against an older version not all symbols were available. This is solved with static linking because there aren't any dependencies, but it doesn't mean that dynamic linking should be avoided (just that it should be compiled against an older libc)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, if it's somewhat different to what you took away from the sentence then it clearly needs rephrasing :)
The thrust is that I'd consider static linking an easier solution than finding/compiling an older libc and then trying to compile against it. It also helps in more exotic situations, like non-glibc distros or even distros without a dynamic libc.

Reading again, I see the mistake is in my sloppy wording - it should be 'linked against' rather than 'compiled against'.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

glibc is actually pretty clever in its linkage in that you can link against a newer version but remain compatible with much older versions (so long as you avoid the "only new" apis). Overall I believe that static linking is largely only motivated by being able to use new APIs on all systems, or being able to run on all systems that may not even have the dependencies in question.


Static linking supported via an alternative libc, musl - this must be enabled
at rust compile-time with some prerequisites available. You can compile your
own version of rust with musl enabled and install it into a custom directory
with the instructions below:

```
$ mkdir musldist
$ PREFIX=$(pwd)/musldist
$
$ # Build musl
$ wget http://www.musl-libc.org/releases/musl-1.1.10.tar.gz
[...]
$ tar xf musl-1.1.10.tar.gz
$ cd musl-1.1.10/
musl-1.1.10 $ ./configure --disable-shared --prefix=$PREFIX
[...]
musl-1.1.10 $ make
[...]
musl-1.1.10 $ make install
[...]
musl-1.1.10 $ cd ..
$ du -h musldist/lib/libc.a
2.2M musldist/lib/libc.a
$
$ # Build libunwind.a
$ wget http://llvm.org/releases/3.6.1/llvm-3.6.1.src.tar.xz
$ tar xf llvm-3.6.1.src.tar.xz
$ cd llvm-3.6.1.src/projects/
llvm-3.6.1.src/projects $ svn co http://llvm.org/svn/llvm-project/libcxxabi/trunk/ libcxxabi
llvm-3.6.1.src/projects $ svn co http://llvm.org/svn/llvm-project/libunwind/trunk/ libunwind
llvm-3.6.1.src/projects $ sed -i 's#^\(include_directories\).*$#\0\n\1(../libcxxabi/include)#' libunwind/CMakeLists.txt
llvm-3.6.1.src/projects $ mkdir libunwind/build
llvm-3.6.1.src/projects $ cd libunwind/build
llvm-3.6.1.src/projects/libunwind/build $ cmake -DLLVM_PATH=../../.. -DLIBUNWIND_ENABLE_SHARED=0 ..
llvm-3.6.1.src/projects/libunwind/build $ make
llvm-3.6.1.src/projects/libunwind/build $ cp lib/libunwind.a $PREFIX/lib/
llvm-3.6.1.src/projects/libunwind/build $ cd cd ../../../../
$ du -h musldist/lib/libunwind.a
164K musldist/lib/libunwind.a
$
$ # Build musl-enabled rust
$ git clone https://github.com/rust-lang/rust.git muslrust
$ cd muslrust
muslrust $ ./configure --target=x86_64-unknown-linux-musl --musl-root=$PREFIX --prefix=$PREFIX
muslrust $ make
muslrust $ make install
muslrust $ cd ..
$ du -h musldist/bin/rustc
12K musldist/bin/rustc
```

You now have a build of a musl-enabled rust! Because we've installed it to a
custom prefix we need to make sure our system can the binaries and appropriate
libraries when we try and run it:

```
$ export PATH=$PREFIX/bin:$PATH
$ export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH
```

Let's try it out!

```
$ echo 'fn main() { println!("hi!"); panic!("failed"); }' > example.rs
$ rustc --target=x86_64-unknown-linux-musl example.rs
$ ldd example
not a dynamic executable
$ ./example
hi!
thread '<main>' panicked at 'failed', example.rs:1
```

Success! This binary can be copied to almost any Linux machine with the same
machine architecture and run without issues.

`cargo build` also permits the `--target` option so you should be able to build
your crates as normal. However, you may need to recompile your native libraries
against musl before they can be linked against.
25 changes: 0 additions & 25 deletions src/doc/trpl/link-args.md

This file was deleted.