Skip to content

unify the RunStep checks logic with CheckFileStep code and also with std.testing #14945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tracked by #14647
andrewrk opened this issue Mar 16, 2023 · 0 comments
Open
Tracked by #14647
Labels
breaking Implementing this issue could cause existing code to no longer compile or have different behavior. enhancement Solving this issue will likely involve adding new logic or components to the codebase. zig build system std.Build, the build runner, `zig build` subcommand, package management
Milestone

Comments

@andrewrk
Copy link
Member

andrewrk commented Mar 16, 2023

Extracted from #14647.

/// Causes the RunStep to be considered to *not* have side-effects. The
/// process will be re-executed if any of the input dependencies are
/// modified. The exit code and standard I/O streams will be checked for
/// certain conditions, and the step will succeed or fail based on these
/// conditions.
/// Note that an explicit check for exit code 0 needs to be added to this
/// list if such a check is desireable.
check: std.ArrayList(Check),
/// This RunStep is running a zig unit test binary and will communicate
/// extra metadata over the IPC protocol.
zig_test,
pub const Check = union(enum) {
expect_stderr_exact: []const u8,
expect_stderr_match: []const u8,
expect_stdout_exact: []const u8,
expect_stdout_match: []const u8,
expect_term: std.process.Child.Term,
};

for (self.expected_matches) |expected_match| {
if (mem.indexOf(u8, contents, expected_match) == null) {
return step.fail(
\\
\\========= expected to find: ===================
\\{s}
\\========= but file does not contain it: =======
\\{s}
\\===============================================
, .{ expected_match, contents });
}
}
if (self.expected_exact) |expected_exact| {
if (!mem.eql(u8, expected_exact, contents)) {
return step.fail(
\\
\\========= expected: =====================
\\{s}
\\========= but found: ====================
\\{s}
\\========= from the following file: ======
\\{s}
, .{ expected_exact, contents, src_path });
}
}

switch (self.stdio) {
.check => |checks| for (checks.items) |check| switch (check) {
.expect_stderr_exact => |expected_bytes| {
assert(!result.stdio.stderr_null);
if (!mem.eql(u8, expected_bytes, result.stdio.stderr)) {
return step.fail(
\\
\\========= expected this stderr: =========
\\{s}
\\========= but found: ====================
\\{s}
\\========= from the following command: ===
\\{s}
, .{
expected_bytes,
result.stdio.stderr,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},
.expect_stderr_match => |match| {
assert(!result.stdio.stderr_null);
if (mem.indexOf(u8, result.stdio.stderr, match) == null) {
return step.fail(
\\
\\========= expected to find in stderr: =========
\\{s}
\\========= but stderr does not contain it: =====
\\{s}
\\========= from the following command: =========
\\{s}
, .{
match,
result.stdio.stderr,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},
.expect_stdout_exact => |expected_bytes| {
assert(!result.stdio.stdout_null);
if (!mem.eql(u8, expected_bytes, result.stdio.stdout)) {
return step.fail(
\\
\\========= expected this stdout: =========
\\{s}
\\========= but found: ====================
\\{s}
\\========= from the following command: ===
\\{s}
, .{
expected_bytes,
result.stdio.stdout,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},
.expect_stdout_match => |match| {
assert(!result.stdio.stdout_null);
if (mem.indexOf(u8, result.stdio.stdout, match) == null) {
return step.fail(
\\
\\========= expected to find in stdout: =========
\\{s}
\\========= but stdout does not contain it: =====
\\{s}
\\========= from the following command: =========
\\{s}
, .{
match,
result.stdio.stdout,
try Step.allocPrintCmd(arena, self.cwd, final_argv),
});
}
},

zig/lib/std/testing.zig

Lines 609 to 631 in b3af5d0

pub fn expectEqualStrings(expected: []const u8, actual: []const u8) !void {
if (std.mem.indexOfDiff(u8, actual, expected)) |diff_index| {
print("\n====== expected this output: =========\n", .{});
printWithVisibleNewlines(expected);
print("\n======== instead found this: =========\n", .{});
printWithVisibleNewlines(actual);
print("\n======================================\n", .{});
var diff_line_number: usize = 1;
for (expected[0..diff_index]) |value| {
if (value == '\n') diff_line_number += 1;
}
print("First difference occurs on line {d}:\n", .{diff_line_number});
print("expected:\n", .{});
printIndicatorLine(expected, diff_index);
print("found:\n", .{});
printIndicatorLine(actual, diff_index);
return error.TestExpectedEqual;
}
}

All this logic should be unified.

Plus there should be more kinds of checks:

  • endswith
  • does not match
  • regular expressions (this point can be extracted into a separate issue)
@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. zig build system std.Build, the build runner, `zig build` subcommand, package management labels Mar 16, 2023
@andrewrk andrewrk added this to the 0.12.0 milestone Mar 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking Implementing this issue could cause existing code to no longer compile or have different behavior. enhancement Solving this issue will likely involve adding new logic or components to the codebase. zig build system std.Build, the build runner, `zig build` subcommand, package management
Projects
None yet
Development

No branches or pull requests

1 participant