Skip to content

std.testing.fuzzInput: ability to choose whether timing out is considered a fail or a pass #20815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andrewrk opened this issue Jul 26, 2024 · 0 comments
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. fuzzing
Milestone

Comments

@andrewrk
Copy link
Member

Extracted from #20773.

For example, timing out when fuzzing an interpreter is to be expected.

std.testing.fuzzInput should gain an option that allows making timeouts count as a pass.

Right now, timeouts are not implemented; the fuzzer will not detect an endless loop. This issue is to implement timeouts counting as a failure by default, along with an option to make them instead count as a pass.

Timeouts are harmful for fuzzer performance, however, so even a "pass" should indicate to the fuzzer to avoid this input because it will waste time.

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. fuzzing labels Jul 26, 2024
@andrewrk andrewrk added this to the 0.14.0 milestone Jul 26, 2024
@andrewrk andrewrk modified the milestones: 0.14.0, 0.15.0 Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. fuzzing
Projects
None yet
Development

No branches or pull requests

1 participant