Better message when a test fails

[Parzival-3141]

Closes #14972
Took awhile to figure out but it uses Server.zig's messaging to deliver the error name.

const std = @import("std");

test "oops" {
    const stuff = try std.testing.allocator.alloc(u8, 100);
    // defer std.testing.allocator.free(stuff); // leak!
    if (stuff.len > 10) try foo();
}

fn foo() !void {
    try std.testing.expect(false);
}

Before:

C:\zigsrc\zig\build\stage4\bin\zig.exe build test -Doptimize=ReleaseFast
run test: error: 'test.oops' failed
run test: error: the following test command failed:
C:\Users\pocke\projects\zig\compiler-testing\zig-cache\o\d49f522c92b922567a5cf32d9e08d200\test.exe --listen=-

After:

C:\zigsrc\zig\build\stage4\bin\zig.exe build test -Doptimize=ReleaseFast
run test: error: 'test.oops' failed with error TestUnexpectedResult
run test: error: the following test command failed:
C:\Users\pocke\projects\zig\compiler-testing\zig-cache\o\37fca4499b288793d660959afb8ac53e\test.exe --listen=-

It also works with leak detection, and I changed the wording to make it read better:

C:\zigsrc\zig\build\stage3\bin\zig.exe build test
run test: error: 'test.oops' leaked and failed with error TestUnexpectedResult:
C:\zigsrc\zig\lib\std\testing.zig:509:14: 0x7ff6c9b71027 in expect (test.exe.obj)
    if (!ok) return error.TestUnexpectedResult;
             ^
C:\Users\pocke\projects\zig\compiler-testing\src\main.zig:10:5: 0x7ff6c9b71147 in foo (test.exe.obj)
    try std.testing.expect(false);
    ^
C:\Users\pocke\projects\zig\compiler-testing\src\main.zig:6:25: 0x7ff6c9b711de in test.oops (test.exe.obj)
    if (stuff.len > 10) try foo();
                        ^
[gpa] (err): memory address 0x27c23e30000 leaked:
C:\Users\pocke\projects\zig\compiler-testing\src\main.zig:4:50: 0x7ff6c9b71186 in test.oops (test.exe.obj)
    const stuff = try std.testing.allocator.alloc(u8, 100);
                                                 ^
C:\zigsrc\zig\lib\test_runner.zig:104:29: 0x7ff6c9b7a357 in mainServer (test.exe.obj)
                test_fn.func() catch |e| switch (e) {
                            ^
C:\zigsrc\zig\lib\test_runner.zig:36:26: 0x7ff6c9b71738 in main (test.exe.obj)
        return mainServer() catch @panic("internal test runner failure");
                         ^
C:\zigsrc\zig\lib\std\start.zig:377:41: 0x7ff6c9b714e7 in WinStartup (test.exe.obj)
    std.debug.maybeEnableSegfaultHandler();
                                        ^
???:?:?: 0x7ffcb1aa7603 in ??? (KERNEL32.DLL)
???:?:?: 0x7ffcb3a826a0 in ??? (ntdll.dll)
run test: error: the following test command failed:
C:\Users\pocke\projects\zig\compiler-testing\zig-cache\o\2e611c757895ff8dbca7073c5385e029\test.exe --listen=-

[matu3ba]

Does this handle all cases for return statements?

• 1. Exited with code other than 0 (or is this only handled for the common case of an error?)
• 2. Signaled
• 3. Stopped
• 4. Unknown

UPDATE: Crossed out the process spawn things. Return status other than 0 can be also due to no error code, ie due to a return 4.

[Parzival-3141]

Exited with code other than 0 (or is this only handled for the common case of an error?)

I assume you mean doing something like:

test "non-zero exit code" {
    std.process.exit(4);
}

AFAIK nothing in this PR handles exit codes, just errors. The current message is pretty good already, though including a stack trace would be nice.

run test: error: while executing test 'test.non-zero exit code', the following command exited with code 4 (expected exited with code 0):
C:\Users\pocke\projects\zig\compiler-testing\zig-cache\o\f62b94d02f3e872bc61dc3bf21a8ea73\test.exe --listen=-

[Parzival-3141]

It seems since I opened this PR the test name has been added to the test command failed output, which now seems a bit redundant:

 C:\zigsrc\zig\build\stage4\bin\zig.exe build test -Doptimize=ReleaseFast
run test: error: 'test.oops' failed with error TestUnexpectedResult
run test: error: while executing test 'test.oops', the following test command failed:
...

[andrewrk]

I'm sorry, I didn't review this in time, and now it has bitrotted. Furthermore, so many pull requests have stacked up that I can't keep up and I am therefore declaring Pull Request Bankruptcy and closing old PRs that now have conflicts with master branch.

If you want to reroll, you are by all means welcome to revisit this changeset with respect to the current state of master branch, and there's a decent chance your patch will be reviewed the second time around.

Either way, I'm closing this now, otherwise the PR queue will continue to grow indefinitely.