package manager: no dependency named 'X' in 'build.zig.zon'; but it is #16354
Comments
emidoots
added
the
bug
andrewrk
added
the
zig build system
|
Apologies, I think I know why this is happening now and believe it's my own fault. I'm confirming now. |
|
Even if it ends up being ultimately on your end, feel free to suggest an enhancement to the zig toolchain that could have helped you diagnose the problem more easily 🙂 |
|
Confirmed, this was my fault / not a bug in Zig. Here's the summary:
Ultimately I think this is caused by abusing (1), which is probably unintended behavior of the package manager, and there should instead be a different/better way for a package to expose build-time code. |
|
Ability to import directly the dependency in build.zig (and have the import root be the package's build.zig file) is intentional behavior, and it sounds like it's useful given that several projects have already discovered it and started using it. I'm not sure I understand your second point - where would |
|
I do think there is a bug with how dependencies are discovered (or replicated when the same dependency is present multiple time within a project dependency tree). I'm trying to build mach-gpu, https://github.com/hexops/mach-gpu/tree/1266bd935c20d539d74390380364ec566be4eec9 this commit exactly but changed to use mach_gpu as a zig dependency instead of a git submodule. Thus, this was added in the .mach_gpu_dawn = .{
.url = "https://github.com/hexops/mach-gpu-dawn/archive/a33e4030dcc0d6fa6c0abf3419000b504074511a.tar.gz",
.hash = "122078af196fd94860b4050edb55605e3cd4332405b8edb56718767e2433dd47975d",
},By modifying .{
.{.name = "mach_gpu_dawn.mach_glfw.glfw.vulkan_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.glfw.wayland_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.glfw.x11_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.glfw", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.direct3d_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.vulkan_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.wayland_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.x11_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw", .is_pub = true},
.{.name = "mach_gpu_dawn.glfw", .is_pub = true},
.{.name = "mach_gpu_dawn.direct3d_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.vulkan_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.wayland_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.x11_headers", .is_pub = true},
.{.name = "mach_gpu_dawn", .is_pub = true},
.{.name = "mach_glfw.glfw", .is_pub = true},
.{.name = "mach_glfw.direct3d_headers", .is_pub = true},
.{.name = "mach_glfw.vulkan_headers", .is_pub = true},
.{.name = "mach_glfw.wayland_headers", .is_pub = true},
.{.name = "mach_glfw.x11_headers", .is_pub = true},
.{.name = "mach_glfw", .is_pub = true},
.{.name = "glfw.vulkan_headers", .is_pub = true},
.{.name = "glfw.wayland_headers", .is_pub = true},
.{.name = "glfw.x11_headers", .is_pub = true},
.{.name = "glfw", .is_pub = true},
.{.name = "direct3d_headers", .is_pub = true},
.{.name = "vulkan_headers", .is_pub = true},
.{.name = "wayland_headers", .is_pub = true},
.{.name = "x11_headers", .is_pub = true}
}(result of Since the "dependency resolution" seemed to have worked fine in Or maybe that the result is not replicated when the The issue with my idea is that, even if the dependencies werent duplicated for .{
.{.name = "mach_gpu_dawn.mach_glfw.glfw.vulkan_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.glfw.wayland_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.glfw.x11_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.glfw", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.direct3d_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.vulkan_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.wayland_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw.x11_headers", .is_pub = true},
.{.name = "mach_gpu_dawn.mach_glfw", .is_pub = true},
[...]
.{.name = "mach_glfw.glfw", .is_pub = true},
.{.name = "mach_glfw.direct3d_headers", .is_pub = true},
.{.name = "mach_glfw.vulkan_headers", .is_pub = true},
.{.name = "mach_glfw.wayland_headers", .is_pub = true},
.{.name = "mach_glfw.x11_headers", .is_pub = true},
.{.name = "mach_glfw", .is_pub = true},
[...]
}Edit: I thought that maybe it was an issue with the commit/hash of the packages, so I went around and ensured that all of the commits and hashs of each package in every build.zig.zon files matched, but the issue still happens. |
|
Ah, yes, looking at the code, there is a bug there. I have no idea why it worked for Fixing this leads to a design question for @andrewrk: should recursive dependencies between packages be allowed? (If so, we'll need to move away from the FQN thing, since |
You're right, I didn't think this through enough before drawing my conclusion.
Yes, I'm not sure either - but the behavior did change when I stopped importing the |
|
I don't have a reproduction, but this issue is blocking me from being able to update to use the package manager. I have multiple triangle-shaped dependencies. Concretely, this is what it looks like. All of the transitive dependencies (i.e. zlib) currently match exact hashes in all repositories. More info below the code block.
And the So it seems: once you have two dependencies that share a transitive dependency grandchild-level deep, something breaks. I can't find a workaround for this scenario. @slimsag shared a workaround with me that works for the mach dependency (I hit the original issue too) but it doesn't work for my freetype scenario. |
|
I've found a workaround based on what @slimsag suggested and it looks like this. Yikes! const freetype_dep = b.dependency("freetype", .{ .target = step.target, .optimize = step.optimize });
const libpng_dep = b.dependency("freetype.libpng", .{
.target = step.target,
.optimize = step.optimize,
});
const zlib_dep = b.dependency("freetype.libpng.zlib", .{
.target = step.target,
.optimize = step.optimize,
});What I'd rather do which probably is correct once this bug is fixed: It's possible as I continue going further down the transitive path here that I may not be able to work around it. I'll update this if so. |
|
I am running into this as well when trying to build |
|
Same here. The weird thing is that |
|
I'm seeing this issue in a slightly different context, when using This is roughly what my project looks like:
.{
.name = "common",
.version = "0.1.0",
.dependencies = .{
.httpz = .{
.url = "https://github.com/karlseguin/http.zig/archive/3b15f2034f7b412b168c7d623bcd1a11d95ee6c6.tar.gz",
.hash = "12204d254505fd33404d95da79024bcac378bfd558987a8f04bd2920af53e7aed768",
},
.zigimg = .{
.url = "https://github.com/zigimg/zigimg/archive/40ddb16fd246174545b7327a12dce7f0889ada7a.tar.gz",
.hash = "12208c69b00540fe3f38893800fb6c89a9fe27b4a687de5d0462e498a43db4caeec0",
},
},
}In pub fn build(b: *std.build.Builder) !void
{
// ...
const zigimg = b.dependency("zigimg", .{
.target = target,
.optimize = optimize,
});
const zigimgModule = zigimg.module("zigimg");
const appModule = b.addModule("common-app", .{
.source_file = .{.path = "src/app/app.zig"},
.dependencies = &[_]std.build.ModuleDependency {
// ...
.{.name = "zigimg", .module = zigimgModule},
},
});
// ...
}Then in my top-level pub fn build(b: *std.build.Builder) !void
{
// ...
const common = b.anonymousDependency("deps/common", @import("deps/common/build.zig"), .{
.target = target,
.optimize = optimize,
});
// define exe target, blabla ...
exe.addModule("common-app", common .module("common-app"))
}For most cases, I would probably pull this library from a URL with a top-level |
…cept
The new `@depedencies` module contains generated code like the
following:
```zig
pub const root_deps = [_]struct { []const u8, []const u8 }{
.{ "foo", "abc123" },
};
pub const packages = struct {
pub const abc123 = struct {
pub const build_root = "/home/mlugg/.cache/zig/blah/abc123";
pub const build_zig = @import("abc123");
pub const deps = [_]struct { []const u8, []const u8 }{
.{ "bar", "abc123" },
.{ "name", "ghi789" },
};
};
};
```
Each package contains a build root string, the build.zig import, and a
mapping from dependency names to package hashes. There is also such a
mapping for the root package dependencies.
Resolves: ziglang#16354
Resolves: ziglang#17135
…cept
The new `@depedencies` module contains generated code like the
following (where strings like "abc123" represent hashes):
```zig
pub const root_deps = [_]struct { []const u8, []const u8 }{
.{ "foo", "abc123" },
};
pub const packages = struct {
pub const abc123 = struct {
pub const build_root = "/home/mlugg/.cache/zig/blah/abc123";
pub const build_zig = @import("abc123");
pub const deps = [_]struct { []const u8, []const u8 }{
.{ "bar", "abc123" },
.{ "name", "ghi789" },
};
};
};
```
Each package contains a build root string, the build.zig import, and a
mapping from dependency names to package hashes. There is also such a
mapping for the root package dependencies.
In theory, we could now remove the `dep_prefix` field from `std.Build`,
since its main purpose is now handled differently. I believe this is a
desirable goal, as it doesn't really make sense to assign a single FQN
to any package (because it may appear in many different places in the
package hierarchy). This commit does not remove that field, as it's used
non-trivially in a few places in the build runner and compiler tests:
this will be a future enhancement.
Resolves: ziglang#16354
Resolves: ziglang#17135
…cept
The new `@depedencies` module contains generated code like the
following (where strings like "abc123" represent hashes):
```zig
pub const root_deps = [_]struct { []const u8, []const u8 }{
.{ "foo", "abc123" },
};
pub const packages = struct {
pub const abc123 = struct {
pub const build_root = "/home/mlugg/.cache/zig/blah/abc123";
pub const build_zig = @import("abc123");
pub const deps = [_]struct { []const u8, []const u8 }{
.{ "bar", "abc123" },
.{ "name", "ghi789" },
};
};
};
```
Each package contains a build root string, the build.zig import, and a
mapping from dependency names to package hashes. There is also such a
mapping for the root package dependencies.
In theory, we could now remove the `dep_prefix` field from `std.Build`,
since its main purpose is now handled differently. I believe this is a
desirable goal, as it doesn't really make sense to assign a single FQN
to any package (because it may appear in many different places in the
package hierarchy). This commit does not remove that field, as it's used
non-trivially in a few places in the build runner and compiler tests:
this will be a future enhancement.
Resolves: ziglang#16354
Resolves: ziglang#17135
…cept
The new `@depedencies` module contains generated code like the
following (where strings like "abc123" represent hashes):
```zig
pub const root_deps = [_]struct { []const u8, []const u8 }{
.{ "foo", "abc123" },
};
pub const packages = struct {
pub const abc123 = struct {
pub const build_root = "/home/mlugg/.cache/zig/blah/abc123";
pub const build_zig = @import("abc123");
pub const deps = [_]struct { []const u8, []const u8 }{
.{ "bar", "abc123" },
.{ "name", "ghi789" },
};
};
};
```
Each package contains a build root string, the build.zig import, and a
mapping from dependency names to package hashes. There is also such a
mapping for the root package dependencies.
In theory, we could now remove the `dep_prefix` field from `std.Build`,
since its main purpose is now handled differently. I believe this is a
desirable goal, as it doesn't really make sense to assign a single FQN
to any package (because it may appear in many different places in the
package hierarchy). This commit does not remove that field, as it's used
non-trivially in a few places in the build runner and compiler tests:
this will be a future enhancement.
Resolves: ziglang#16354
Resolves: ziglang#17135
…cept
The new `@depedencies` module contains generated code like the
following (where strings like "abc123" represent hashes):
```zig
pub const root_deps = [_]struct { []const u8, []const u8 }{
.{ "foo", "abc123" },
};
pub const packages = struct {
pub const abc123 = struct {
pub const build_root = "/home/mlugg/.cache/zig/blah/abc123";
pub const build_zig = @import("abc123");
pub const deps = [_]struct { []const u8, []const u8 }{
.{ "bar", "abc123" },
.{ "name", "ghi789" },
};
};
};
```
Each package contains a build root string, the build.zig import, and a
mapping from dependency names to package hashes. There is also such a
mapping for the root package dependencies.
In theory, we could now remove the `dep_prefix` field from `std.Build`,
since its main purpose is now handled differently. I believe this is a
desirable goal, as it doesn't really make sense to assign a single FQN
to any package (because it may appear in many different places in the
package hierarchy). This commit does not remove that field, as it's used
non-trivially in a few places in the build runner and compiler tests:
this will be a future enhancement.
Resolves: #16354
Resolves: #17135
Zig Version
0.11.0-dev.3886+0c1bfe271
Steps to Reproduce and Observed Behavior
I'm encountering an issue where we get this error:
The
.cache/zig/.../build.zig.zonfile in the error message has these contents:My project has a package dependency tree which looks like this (I've excluded some deps for brevity):
When I build the
mach_glfwpackage, I do not encounter this error. Only when building themach_gpu_dawnpackage does this error occur.The generated `dependencies.zig` in `zig-cache` is this
Notably, we can see in the above that the dependency is there:
I modified this code in
std.Buildto help debug this:zig/lib/std/Build.zig
Lines 1523 to 1541 in b9fc0d2
I printed the
decl.namefor each dependency, and we can see it ends up there:In short, it's looking for
glfw.vulkan_headersbutmach_glfw.glfw.vulkan_headersis what should be matching:nameis"vulkan_headers"b.dep_prefixis"glfw."mem.startsWith(u8, decl.name, b.dep_prefix)isfalsemem.endsWith(u8, decl.name, name)istruedecl.name.len == b.dep_prefix.len + name.lenisfalseBut this is where my knowledge falls apart; I can see the problem clearly, but I'm not sure what to do about it:
b.dep_prefixneed to be corrected to be"mach_glfw.glfw."? If so, how?b.dependencylogic need to be relaxed to allow forglfw.vulkan_headersto matchmach_glfw.glfw.vulkan_headers?dependencies.zigprefixes be corrected to containglfw.vulkan_headersinstead ofmach_glfw.glfw.vulkan_headers?Expected Behavior
dependency matching works as expected, the import succeeds
The text was updated successfully, but these errors were encountered: