Skip to content

unify the RunStep checks logic with CheckFileStep code and also with std.testing #14945

Open
@andrewrk

Description

@andrewrk

Extracted from #14647.

/// Causes the RunStep to be considered to *not* have side-effects. The
/// process will be re-executed if any of the input dependencies are
/// modified. The exit code and standard I/O streams will be checked for
/// certain conditions, and the step will succeed or fail based on these
/// conditions.
/// Note that an explicit check for exit code 0 needs to be added to this
/// list if such a check is desireable.
check: std.ArrayList(Check),
/// This RunStep is running a zig unit test binary and will communicate
/// extra metadata over the IPC protocol.
zig_test,
pub const Check = union(enum) {
expect_stderr_exact: []const u8,
expect_stderr_match: []const u8,
expect_stdout_exact: []const u8,
expect_stdout_match: []const u8,
expect_term: std.process.Child.Term,
};

for (self.expected_matches) |expected_match| {
if (mem.indexOf(u8, contents, expected_match) == null) {
return step.fail(
\\
\\========= expected to find: ===================
\\{s}
\\========= but file does not contain it: =======
\\{s}
\\===============================================
, .{ expected_match, contents });
}
}
if (self.expected_exact) |expected_exact| {
if (!mem.eql(u8, expected_exact, contents)) {
return step.fail(
\\
\\========= expected: =====================
\\{s}
\\========= but found: ====================
\\{s}
\\========= from the following file: ======
\\{s}
, .{ expected_exact, contents, src_path });
}
}

switch (self.stdio) {
.check => |checks| for (checks.items) |check| switch (check) {
.expect_stderr_exact => |expected_bytes| {
assert(!result.stdio.stderr_null);
if (!mem.eql(u8, expected_bytes, result.stdio.stderr)) {
return step.fail(
\\
\\========= expected this stderr: =========
\\{s}
\\========= but found: ====================
\\{s}
\\========= from the following command: ===
\\{s}
, .{
expected_bytes,
result.stdio.stderr,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},
.expect_stderr_match => |match| {
assert(!result.stdio.stderr_null);
if (mem.indexOf(u8, result.stdio.stderr, match) == null) {
return step.fail(
\\
\\========= expected to find in stderr: =========
\\{s}
\\========= but stderr does not contain it: =====
\\{s}
\\========= from the following command: =========
\\{s}
, .{
match,
result.stdio.stderr,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},
.expect_stdout_exact => |expected_bytes| {
assert(!result.stdio.stdout_null);
if (!mem.eql(u8, expected_bytes, result.stdio.stdout)) {
return step.fail(
\\
\\========= expected this stdout: =========
\\{s}
\\========= but found: ====================
\\{s}
\\========= from the following command: ===
\\{s}
, .{
expected_bytes,
result.stdio.stdout,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},
.expect_stdout_match => |match| {
assert(!result.stdio.stdout_null);
if (mem.indexOf(u8, result.stdio.stdout, match) == null) {
return step.fail(
\\
\\========= expected to find in stdout: =========
\\{s}
\\========= but stdout does not contain it: =====
\\{s}
\\========= from the following command: =========
\\{s}
, .{
match,
result.stdio.stdout,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},

zig/lib/std/testing.zig

Lines 609 to 631 in b3af5d0

pub fn expectEqualStrings(expected: []const u8, actual: []const u8) !void {
if (std.mem.indexOfDiff(u8, actual, expected)) |diff_index| {
print("\n====== expected this output: =========\n", .{});
printWithVisibleNewlines(expected);
print("\n======== instead found this: =========\n", .{});
printWithVisibleNewlines(actual);
print("\n======================================\n", .{});
var diff_line_number: usize = 1;
for (expected[0..diff_index]) |value| {
if (value == '\n') diff_line_number += 1;
}
print("First difference occurs on line {d}:\n", .{diff_line_number});
print("expected:\n", .{});
printIndicatorLine(expected, diff_index);
print("found:\n", .{});
printIndicatorLine(actual, diff_index);
return error.TestExpectedEqual;
}
}

All this logic should be unified.

Plus there should be more kinds of checks:

  • endswith
  • does not match
  • regular expressions (this point can be extracted into a separate issue)

Metadata

Metadata

Assignees

No one assigned

    Labels

    breakingImplementing this issue could cause existing code to no longer compile or have different behavior.enhancementSolving this issue will likely involve adding new logic or components to the codebase.zig build systemstd.Build, the build runner, `zig build` subcommand, package management

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions