Have `@testset` run conditionally on ARGS?

[antoine-levitt]

I was looking into running only a subset of tests. There is now functionality in Pkg for doing that by passing a list of tests to run to runtests.jl (JuliaLang/Pkg.jl#1226). The suggested workflow is that runtests.jl does the subsetting manually with something like

using HelloWorld
using Test

@testset "HelloWorld.jl" begin
    if isempty(ARGS) || "all" in ARGS
        all_tests = true
    else
        all_tests = false
    end
    if all_tests || "foo" in ARGS
        @testset "foo" begin
            @test HelloWorld.foo(1) == 2
        end
    end
    if all_tests || "bar" in ARGS
        @testset "bar" begin
            @test HelloWorld.bar(2) == 6
        end
    end
    if all_tests || "baz" in ARGS
        @testset "baz" begin
            @test HelloWorld.baz(3) == 12
        end
    end
end

That's a lot of boilerplate. Of course I can redefine my own @testset-like macro to check ARGS, but can't that be done by @testset itself? What I'm suggesting is that @testset checks if ARGS is non-empty and, if it is, only run the test if the string is in ARGS. It is breaking, though...

[fredrikekre]

Fixed by JuliaLang/Pkg.jl#1226 (merged two days ago).

[fredrikekre]

Eh, okay, this is a request to add filtering to the Test stdlib I guess, could also be trivially implemented in a package.

[rfourquet]

Maybe #33672 would fix this issue, albeit with a different mechanism?