lib/init-exe, lib/init-lib: ensure tests runs again

[perillo]

After commit ede5dcf (make the build runner and test runner talk to each other), the std.Build.addTest function no longer runs tests, but the build.zig files in init-exe and init-lib where not updated.

Rename main_tests to lib_tests in init-lib, for consistency with the exe_tests variable in init-exe.

Add a RunStep to exe_tests and lib_tests.

Remove an empty line in the addCliTests function in tests/tests.zig.

Closes #15009

[perillo]

When running zig test-cli with an empty local cache I got an unexpected error:

build/stage3/bin/zig build --zig-lib-dir /home/manlio/src/contrib/zig/github.com/perillo/zig/lib test-cli -Dskip-release -Dskip-non-native -fsummary
zig build test: error: run test: error: unable to spawn /home/manlio/.cache/zig/local/o/cbd7089efe9f240e992b83a8865f738c/test: NotDir
Build Summary: 1/3 steps succeeded; 1 failed (disable with -fno-summary)
test transitive failure
└─ run test failure
   └─ zig test Debug native success 1s MaxRSS:179M
error: the following build command failed with exit code 1:
/home/manlio/.cache/zig/local/o/00b8495321ff609cf773192ee62b764c/build /home/manlio/src/contrib/zig/github.com/perillo/zig/build/stage3/bin/zig /home/manlio/.cache/zig/local/tmp/4310c08cfc9ead4d /home/manlio/.cache/zig/local /home/manlio/.cache/zig/global test

zig build test: error: the following command exited with code 1 (expected exited with code 0):
cd /home/manlio/.cache/zig/local/tmp/4310c08cfc9ead4d && /home/manlio/src/contrib/zig/github.com/perillo/zig/build/stage3/bin/zig build test

[perillo]

On aarch64-macos, the test-cli failed when running the test for init-lib.

Was this the reason why the build.zig files in init-exe and init-lib where not updated?

[perillo]

The only reason I can think for the NotDir error is that the init-lib and addRemoveDirTree steps are executed concurrently and, while the test binary is being built and cached, the src directory is removed.

However the error also occurs when setting the -j1 option.

[perillo]

I have replaced the use of Build.CompileStep.run with Build.addRunArtifact`, also documenting why it is necessary.
I also renamed "Run library tests" to "Run unit tests" for consistency.

[perillo]

@Vexu when you have time, can you help me finding the reason for the NotDir error?

Thanks,

[perillo]

I forgot to replace, in init-exe, const run_cmd = exe.run() to const run_cmd = b.addRunArtifact(). I will apply the change after test-cli bug is fixed.

[matu3ba]

@perillo Try to reduce the problem to a minimal reducible example beforehand.

Then use strace to identify, if the call to fork itself fails. You might want to apply filters.

If the timing itself remains unconclusive, you could try the bigger guns like ftrace https://opensource.com/article/21/7/linux-kernel-ftrace.

If that still remains unconclusive, you could go deeper with the backend-instrumentation https://wiki.st.com/stm32mpu/wiki/Linux_tracing,_monitoring_and_debugging, but I doubt this will be necessary.

Ideally Zig would provide us with a backtrace instead of only the error without any location info etc.

[perillo]

@perillo Try to reduce the problem to a minimal reducible example beforehand.

Then use strace to identify, if the call to fork itself fails. You might want to apply filters.

If the timing itself remains unconclusive, you could try the bigger guns like ftrace https://opensource.com/article/21/7/linux-kernel-ftrace.

If that still remains unconclusive, you could go deeper with the backend-instrumentation https://wiki.st.com/stm32mpu/wiki/Linux_tracing,_monitoring_and_debugging, but I doubt this will be necessary.

Ideally Zig would provide us with a backtrace instead of only the error without any location info etc.

Thanks for the suggestion.
Creating a minimal example was easy: https://gist.github.com/perillo/fcafbe87e5364e68e5bf70b3f4280499.

The problem is that I see only unlink syscalls for removing files; I was expecting a unlinkat syscall for removing the src directory created by zig init-lib.

[perillo]

Here is an independent example:

build.zig

const std = @import("std");

pub fn build(b: *std.Build) void {
    const step = b.step("test-bug", "Test bug");
    const tmp_path = b.makeTempPath();

    const init_lib = b.addSystemCommand(&.{ b.zig_exe, "init-lib" });
    init_lib.cwd = tmp_path;
    init_lib.setName("zig init-lib");

    const run_test = b.addSystemCommand(&.{ b.zig_exe, "build", "test" });
    run_test.cwd = tmp_path;
    run_test.setName("zig build test");
    run_test.expectStdOutEqual("");
    run_test.step.dependOn(&init_lib.step);

    const cleanup = b.addRemoveDirTree(tmp_path);
    cleanup.step.dependOn(&run_test.step);

    step.dependOn(&cleanup.step);
}

$ zig build test-bug
info: Created build.zig
info: Created src/main.zig
info: Next, try `zig build --help` or `zig build test`
zig build test: error: run test: error: unable to spawn /home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache/tmp/4ef82da693a614fa/zig-cache/o/8b79ec684c28526ca0bb5087766c89f2/test: NotDir
Build Summary: 1/3 steps succeeded; 1 failed (disable with -fno-summary)
test transitive failure
└─ run test failure
   └─ zig test Debug native success 1s MaxRSS:176M
error: the following build command failed with exit code 1:
/home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache/tmp/4ef82da693a614fa/zig-cache/o/3fc433ac561724a230acd8502d78eccc/build /home/manlio/src/contrib/zig/github.com/perillo/zig/build/stage3/bin/zig /home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache/tmp/4ef82da693a614fa /home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache/tmp/4ef82da693a614fa/zig-cache /home/manlio/.cache/zig test

zig build test: error: the following command exited with code 1 (expected exited with code 0):
cd /home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache/tmp/4ef82da693a614fa && /home/manlio/src/contrib/zig/github.com/perillo/zig/build/stage3/bin/zig build test 
Build Summary: 1/4 steps succeeded; 1 failed (disable with -fno-summary)
test-bug transitive failure
└─ RemoveDir /home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache/tmp/4ef82da693a614fa transitive failure
   └─ zig build test failure
      └─ zig init-lib success 19ms MaxRSS:60M
error: the following build command failed with exit code 1:
/home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache/o/426c8df43d3110df2fe824c5f8e2df95/build /home/manlio/src/contrib/zig/github.com/perillo/zig/build/stage3/bin/zig /home/manlio/src/zig/src/test.localhost/zig-issues/15009 /home/manlio/src/zig/src/test.localhost/zig-issues/15009/zig-cache /home/manlio/.cache/zig test-bug

[perillo]

Closed for #15129

[andrewrk]

Superseded by #15245 which landed in 1728d92.