Cargo integration in Meson via an "unstable-cargo" module #2616

nirbheek · 2017-11-12T17:25:17Z

This was made primarily so we can build librsvg with Meson, which now works: https://git.gnome.org/browse/librsvg/log/?h=wip/meson, and it actually required fewer hacks than I expected it to.

The integration can and should definitely be better, but I think the best way to get there is to have something that works right now and slowly improve Cargo till we have everything we need. Cargo is not going to magically grow API for giving greater control to build systems; we have to work towards it.

I have been talking to @alexcrichton about the things that Meson would need to integrate better with Cargo, and he has been very receptive and encouraging of our efforts, and in the need for adding features to Cargo that allow Meson to control what it does a bit better.

Here's what you can do right now:

'Extract' build outputs from Cargo.toml files in your source tree with the unstable-cargo module and mostly use them like you would any other target. For example:

cargo = import('unstable-cargo')
# Build the staticlib
librsvg_internals = cargo.static_library('rsvg_internals', toml : 'Cargo.toml')

librsvg = library('rsvg-' + rsvg_api_major, librsvg_sources,
                  dependencies : [librsvg_deps, librsvg_internals])

You can also specify the list of sources if you wish:

cargo.executable('mesonbin',
                 toml : 'Cargo.toml',
                 # This is optional, since ninja keeps track of dependencies on source files.
                 sources : ['src/main.rs'],
                 # Yes, installation works
                 install : true)

The sources list is optional because cargo will output a nice depfile for us and ninja can then keep track of when to rebuild the target.

Meson TODO:

Cross-compilation
Cargo tests and benchmarks
cdylibs or rust-specific library types
ninja dist does not support vendoring crate sources yet (needed for librsvg)

Cargo TODO:

Fix dep info showing up with a build script rust-lang/cargo#4711
CARGO_TARGET_DIR should be an argument --target-dir
Options to set the output library/executable name (prefix/suffix); perhaps also in Cargo.toml
In general, the ability to more accurately control build outputs from via cargo arguments
- This is where our work comes in; something half-working is the best way to figure out what is needed to make it work well ;)
Dry-run downloading of sources by cargo vendor so we can ship a list of files instead of the sources themselves (cc @alexlarsson)

Cargo long-term TODO:

Output rustc invocations needed to build a crate so we can run everything ourselves
Overall, Meson should be able to output build artifacts exactly where it wants, and with the rustc flags that we want.

This is not exposed to build files and should not be. However, it is useful for modules that want to run external commands that only take certain inputs via environment variables.

It is better to send it to stdout so that the user sees progress on long-running commands. It also gives better feedback to users, f.ex. when a command requires capture: true and the user forgets to pass it.

Change the check for `/` to checking for `..` to prevent custom targets from fetching outputs from outside the target directory, but allow fetching outputs from arbitrarily-nested subdirs. Also disallow absolute paths for outputs for similar reasons.

Long-running commands such as `cargo build` would show no output at all without this. This feature is precisely for this use-case, so expose it. Of course, build files do not have access to this yet.

Needs the meson branch that adds a cargo module for building library and executable targets: mesonbuild/meson#2616 TODO: * gtk-doc documentation * symbol visibility: must use symbol prototype macros instead of relying on libtool symbol regex or a linker script - MSVC support also needs this * introspection bindings * vala bindings * LC_MESSAGES localization

ignatenkobrain · 2017-11-12T17:26:29Z

mesonbuild/backend/ninjabackend.py

+    def custom_target_needs_serialization(self, target, cmd):
+        if target.capture or target.envvars or any('\n' in c for c in cmd) or \
+           ((mesonlib.is_windows() or mesonlib.is_cygwin()) and
+            self.determine_windows_extra_paths(target.command[0])):


[Flake8]

visually indented line with same indent as next logical line

ignatenkobrain · 2017-11-12T17:26:30Z

mesonbuild/modules/unstable_cargo.py

+
+import os
+import json
+from pathlib import PurePath


[Flake8]

'pathlib.PurePath' imported but unused

ignatenkobrain · 2017-11-12T17:26:30Z

mesonbuild/modules/unstable_cargo.py

+        self._check_cargo_dep_info_bug(md)
+        # Get the list of outputs that cargo will create matching the specified name
+        ctkwargs['output'], ctkwargs['depfile'], cargo_args = \
+                self._get_cargo_outputs(name, md, env, cargo_target_type)


[Flake8]

continuation line over-indented for visual indent

Currently only supports building staticlib libraries and binaries with Cargo. All configuration must be in the Cargo.toml file. You should use cargo 0.23 because it fixes a bug in emitting dep-info FIXMEs: * Cross compilation is broken. We do not pass the `--target TRIPLE` flag to cargo, so it always builds targetting the build machine. * tests and benches are currently not supported, but can be added * cdylibs are not supported because it is not clear if anyone uses them and if they even work properly in Rust * `ninja dist` does not yet run `cargo vendor` to add crate sources * `cargo clean` is not called on `ninja clean` * We cannot handle adding system libraries that are needed by the staticlib while linking because it's outputted at build time by `cargo build`. You must handle that yourself. * We do not pass on RUSTFLAGS from the env during configure to cargo build.

nirbheek · 2017-11-12T17:31:22Z

Gosh darnit, travis is still flagging my PRs as "abuse". Sigh. Moving this to a new PR.

QuLogic · 2017-11-12T19:38:18Z

I don't understand; there are Travis results there.

nirbheek · 2017-11-12T22:51:07Z

That's because of the new PR that I created by pushing to mesonbuild/meson instead of centricular/meson. See this:

nirbheek added 4 commits November 12, 2017 20:49

custom target: Serialize env vars that are set on the target

35d737d

This is not exposed to build files and should not be. However, it is useful for modules that want to run external commands that only take certain inputs via environment variables.

meson_exe.py: Don't capture stdout unless requested

054910a

It is better to send it to stdout so that the user sees progress on long-running commands. It also gives better feedback to users, f.ex. when a command requires capture: true and the user forgets to pass it.

custom targets: Allow setting the ninja pool for commands

545d79b

Long-running commands such as `cargo build` would show no output at all without this. This feature is precisely for this use-case, so expose it. Of course, build files do not have access to this yet.

ignatenkobrain reviewed Nov 12, 2017

View reviewed changes

nirbheek closed this Nov 12, 2017

nirbheek deleted the cargo-module branch November 12, 2017 17:31

eli-schwartz added the language:rust label Mar 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Cargo integration in Meson via an "unstable-cargo" module #2616

Cargo integration in Meson via an "unstable-cargo" module #2616

Uh oh!

nirbheek commented Nov 12, 2017

Uh oh!

ignatenkobrain Nov 12, 2017

Uh oh!

ignatenkobrain Nov 12, 2017

Uh oh!

ignatenkobrain Nov 12, 2017

Uh oh!

nirbheek commented Nov 12, 2017

Uh oh!

QuLogic commented Nov 12, 2017

Uh oh!

nirbheek commented Nov 12, 2017

Uh oh!

Uh oh!

Uh oh!

Cargo integration in Meson via an "unstable-cargo" module #2616

Cargo integration in Meson via an "unstable-cargo" module #2616

Uh oh!

Conversation

nirbheek commented Nov 12, 2017

Uh oh!

ignatenkobrain Nov 12, 2017

Choose a reason for hiding this comment

Uh oh!

ignatenkobrain Nov 12, 2017

Choose a reason for hiding this comment

Uh oh!

ignatenkobrain Nov 12, 2017

Choose a reason for hiding this comment

Uh oh!

nirbheek commented Nov 12, 2017

Uh oh!

QuLogic commented Nov 12, 2017

Uh oh!

nirbheek commented Nov 12, 2017

Uh oh!

Uh oh!