[dev.fuzz] proposal: add flag to control parallelism

[dsnet]

@jayconrod @katiehockman

Currently go test -fuzz=MyFunc seems to spin up a number of workers equal to the number of cores on the machine. The default is reasonable, but it would be nice to provide a flag to control the number of workers. I'd like to set it to 1/2 or even 1/4 of the cores I have, so I can run fuzzing in the background as I work, and not have my computer become unresponsive.

The older go-fuzz tool has a -procs flag to accomplish this.

[josharian]

go test already has a -cpu flag. Maybe we could overload it?

[jayconrod]

So far, we've been overloading the -test.parallel flag for this. It also controls the number of subtests that called T.Parallel that may run at the same time. That seems reasonable to me since the tests don't run at the same time as fuzzing workers.

We've been talking about adding a separate flag for this (-test.fuzzworkers?) but I worry that it adds too much complexity to the CLI. We have -p, -test.cpu, and -test.parallel already, which are confusingly similar.

[dsnet]

Overloading either test.parallel or test.cpu sounds reasonable to me, though it's not obvious to me which one it should be. Documenting which one takes effect would be helpful.

[katiehockman]

Continuing to use test.parallel seems reasonable to me. The go command defines test.parallel for tests as:

The value of this flag is the maximum number of tests to run simultaneously.

In the case of fuzzing, the "maximum number of tests to run simultaneously" is the number of workers that are running the fuzz function. This sounds equivalent enough to me that I think we can document this effectively here.

[katiehockman]

http://golang.org/s/draft-fuzzing-design#go-command has also been updated to describe this.

[dr2chase]

Would it be wrong to just use GOMAXPROCS? I've been running into this for parallel builds of parallel compiles mucking up the order in which compiler errors were encountered, and NumCPU() is annoying and wrong on a laptop where I'm trying to do anything else. We have GOMAXPROCS, it defaults to NumCPU() so the default is unchanged, why not use that?

Arguments against GOMAXPROCS that I know of are (1) less easy on Windows (is his true?) and (2) what if I wanted fine-grained control of build and compile parallelism (in the case of Go Fuzz, tests in parallel vs parallelism in tests).

(2) makes more sense for testing/fuzzing than it does for building -- we've got two compelling use cases for GOMAXPROCS=1 while building (find compiler bugs in a reliable order; build things on low-memory machines), and very little use for Goldilocks-tuned build/compile parallelism hacking. I can easily imagine wanting to smoke out parallelism bugs in testing, but "please run those tests one at a time, thank you very much".

[jayconrod]

-test.parallel and -test.cpu default to runtime.GOMAXPROCS(), so I support GOMAXPROCS would work already.

[AlekSi]

I guess that one can be closed?

[katiehockman]

I'm going to go ahead and close this issue. What to do with -cpu is being discussed in #46629, so we can move the conversation there