a zig build mode intended to be used when packaging an application or library for a system distribution

[andrewrk]

This proposal is accepted; please see these comments for the criteria for closing the issue:

• a zig build mode intended to be used when packaging an application or library for a system distribution #14281 (comment)
• a zig build mode intended to be used when packaging an application or library for a system distribution #14281 (comment)

---

Extracted from #14265.

The use case is as follows:

Someone wants to take an upstream application, and package it for a Linux distribution, or another kind of distribution such as Homebrew or Nix.

The application knows exactly what versions of each dependency it wants to build against, but distributions only provide a limited set of versions. Furthermore, the system distribution wants to be in charge of providing all dependencies, building the application from source without access to any files outside of those provided by the system itself, via its own dependency management system.

For this use case, instead of fetching dependencies from the URLs provided by zig build.zig.zon files, instead there should be some kind of override that makes the dependencies fetched from the system package manager.

Related: Stay Together For The Kids: Why System Package Managers and Language Package Managers Struggle to Cooperate

[daurnimator]

For this use case, instead of fetching dependencies from the URLs provided by zig build.zig.toml files, instead there should be some kind of override that makes the dependencies fetched from the system package manager. Perhaps Zig can provide tooling that makes an application support being built in both contexts.

As long as zig can be pointed at a local directory for each dependency instead of ever fetching itself, then the external packaging systems (and/or the tooling around it) can be made to work.

[andrewrk]

I suspect that, in this mode, it will be necessary for zig to shell out to the system C/C++ compiler toolchain instead of relying on its own embedded clang library code. This is a very large can of worms, but, if we accept the foundational premise that zig build is intended to be truly used as a competitor to other build systems, this point is non-negotiable.

[motiejus]

Edit: moved to #14283 (comment)

[nektro]

i think something akin to CMAKE_PREFIX_PATH could help a lot here. in a system package manager environment, it could be populated by them, and if its empty zig could fill the value itself

https://cmake.org/cmake/help/latest/envvar/CMAKE_PREFIX_PATH.html

[andrewrk]

The equivalent parameter in zig build is --search-prefix

[nektro]

yes but i mean a variable different but similar for finding zig packages instead of cmake packages

[andrewrk]

Note what's happening over at NixOS/nixpkgs#241741.

They are creating common logic for packaging Zig applications on NixOS and have this line:

zig build -Drelease-safe -Dcpu=baseline $zigBuildFlags

This is a bit of a problem because -Drelease-safe, while commonly exposed,

• Is not always exposed because the build script might build multiple things and a single choice like this might not make sense
• The upstream author might expose only -Drelease because they want to choose the safety/size tradeoff.

Furthermore, -Dcpu is also not necessarily exposed, for similar reasons. There might be different things being built, with different CPUs. Anything could happen.

What we need is a way for nix and other systems to be able to specify their true intent, which is this:

• Make a release build, not a debug build.
• Any binary artifacts that otherwise would be compiled for the native host, compile them for this target/cpu instead.

We should make both of these things easy to specify, regardless of what is in the upstream application's build.zig script.

Here is my concrete proposal:

• Add a --release flag to zig build. So they can change -Drelease-safe to --release. In build.zig, upstream authors can observe this boolean, and pick a release mode, or, if they don't pick, an error will be emitted, something like, "upstream package supports fast, safe, and small. Please choose: --prefer-fast, --prefer-safe, or --prefer-small." Those options are also options that zig build will unconditionally support. So, probably Nix will have this: zig build --release --prefer-safe.
• Add --override-host-target <target> and --override-host-cpu <cpu> flags to zig build. These are always supported, and do what they say on the tin: any time the host target or cpu would be used, the provided one is used instead.

Finally, we need to also tackle one more problem, which is that zig packages need to know when they are being built in a mode that means they should try to depend on system libraries rather than fetching dependencies via the zig build system. For this I think we will need one or more flags, but I haven't quite thought through all the ramifications and how exactly this will be integrated into systems like Debian or Nix, but that concern should not block the implementation of these other flags, which would help Nix right now.

[andrewrk]

Accepting the proposal now. When closing the issue, there should be 2 follow-up issues opened:

• ability to provide package dependencies by system package manager rather than downloaded by the zig build system
  • related: reconciliation of dependency versions
• ability to build C/C++ source files by the system toolchain rather than by zig's C compiler.

[andrewrk]

In addition to those two flags specified in #14281 (comment), this proposal also introduces a --system <zig-package-dir> flag that does the following:

• makes zig build entirely skip fetching dependency packages to the global zig cache.
• the build runner exposes system mode to build scripts
• build scripts observe system mode and, where appropriate, avoid calling dependency(), instead linking directly to system libraries.
  • note that this provides an opportunity for build scripts to re-use logic for doing this for individual components via standard build options, e.g. -Duse-system-ffmpeg.
    • example: https://github.com/MasonRemaley/2Pew/blob/a912fcf64b04ffdcb0935f31b87fe3ae2be4ff09/build.zig#L34 - here we simply add a b.isSystemMode() or ... clause.
• for dependencies on zig modules, dependency() is called like normal, and those packages are found in <zig-package-dir>. Note that only these dependencies, and not the previous bullet point, need to be populated inside <zig-package-dir>. It is the responsibility of the system package manager to ensure this directory is populated with the necessary packages before invoking zig build in system mode.

As for reconciliation of dependency versions, it will be the responsibility of build scripts to probe the system and decide if it is acceptable to proceed. Zig build system API will be provided for common tasks to aid this endeavor, such as checking the major version of a shared library.

As for ability to build C/C++ source files by the system toolchain rather than by zig's toolchain, the decision to do this can be done automatically when in system mode, and the logic for being a system toolchain driver can go directly into the zig build system.

[ifreund]

Overall the proposed --system flag and proposed behavior sounds like a straightforward and effective way of solving this that is relatively easy to use and understand for zig developers and distro packagers alike.

* build scripts observe system mode and, where appropriate, avoid calling `dependency()`, instead linking directly to system libraries.

My only concern with this proposal is that the default and path of least resistance here for zig projects seems to be calling dependency and doing nothing further to handle the possibility of that dependency being provided by the system. I see this resulting in distro maintainers needing to patch the build.zig files of many projects to fix this.

Perhaps it would be possible to handle the simple case of this automatically by adding more declarative metadata to zig packages providing system libraries (e.g. the ffmpeg zig package could declare that it is equivalent to linkSystemLibrary("ffmpeg") in some way). I'm not clear on the details here though and such an approach may not be worth the complexity.

As for reconciliation of dependency versions, it will be the responsibility of build scripts to probe the system and decide if it is acceptable to proceed. Zig build system API will be provided for common tasks to aid this endeavor, such as checking the major version of a shared library.

A build.zig API for requiring a specific shared library soversion would be very much appreciated.

As for ability to build C/C++ source files by the system toolchain rather than by zig's toolchain, the decision to do this can be done automatically when in system mode, and the logic for being a system toolchain driver can go directly into the zig build system.

I assume this also covers linking the system libc version instead of the zig provided one?

[BratishkaErik]

build scripts observe system mode and, where appropriate, avoid calling dependency(), instead linking directly to system libraries.

Perhaps related #3671 . There can be additional field like zig_artifact which will point to artifact of some dependency, and this field can be used with other fields from that proposal, like this:

const mach_glfw_dep = b.dependency("mach_glfw", .{});

exe.linkSomeLibrary(.{
    .system_artifact = .{
        .so_name = "glfw",
        .pkgconf_name = "glfw3",
        .vcpkg_name = "glfw3",
    },
    .zig_artifact = mach_glfw_dep.artifact("glfw"),
});

Then it uses either system_artifact or zig_artifact, depending on current mode.

[mitchellh]

The ability for a library link to specify a dynamic counterpart (if any) to source from the system would be very welcome indeed, amongst the bigger picture proposed here.

As a bullet point, the way Ghostty does this today is with a flag -Dstatic=false (default true). This defaults to true so that for a new dev cloning out the project, all they need to build Ghostty is zig and git and we'll statically compile the world (except for stuff that can't be, like OpenGL, X11, etc.). But that's not a healthy way to build a release so we provide -Dstatic=false which will link most dependencies dynamically.

This is done very manually with a giant if in build.zig. I'd be super happy with the proposals above to standardize this on the library itself as well as stanardize modes and CPU targets and so on.

Looking forward to it.