cmd/go: provide cache support for code rewriting tools

[josharian]

This issue could easily have been a comment on #19109, but opening a new issue seemed better for everyone.

Some tools such as go-fuzz do a source-to-source transformation followed by a build. This doesn't play nicely with cmd/go's cache. The problem is where to the translated files. The natural option is to write them all to a tmp dir, but that busts cmd/go's cache, making 'go fuzz' compilation slow. You could instead write them to the user's GOPATH somewhere, but that's mildly hostile, and you end up needing to fight import path rewrites. You could write them into the vendor directory, except that that doesn't work when a package has already been vendored, plus it is definitely hostile (since many people check in their vendor directory).

Note that the one s2s transformation supported in the main Go toolchain, test coverage, gets explicit support from cmd/go, including caching instrumented source code in the pkg dir. (At least if I read the -x output correctly.)

I don't know what the right interface looks like here. It could be to tell cmd/go "use this transformation tool", but it's tricky to get cmd/go to call the transformation tool with the right set of information (see e.g. #15677). It could be to ask cmd/go for a work dir and then ask it to treat those work dir entries as replacements for the GOPATH/module packages. Or...?

cc @bcmills @dvyukov

[dvyukov]

This doesn't play nicely with cmd/go's cache.

This also does not play nicely with modules/vendor/internal/cgo/some_other_new_feature/etc.

The most reasonable interface from a tools perspective would be that it's invoked by go command and given a single package to transform (either in-place or to a new given location). But this also needs to work with cgo and allow parsing the package with go/types, including importing other packages.

But reflecting on go-fuzz experience, it's still an open question if s2s is the right approach. It may be a wrong level for such tools. The current go coverage instrumentation is simpler. But for go-fuzz we have an infinite long tail of corner cases where it miscompiles, crashes, produces invalid code, etc. These corner cases are related to specific constant values used in specific contexts, specific nested expression constructs, specific uses of types, imports, shadowing, etc. Here are just few examples:
dvyukov/go-fuzz@f15634d
https://github.com/dvyukov/go-fuzz/blob/31e2128c9be91445df05c3bbfb34e0dc3f1e4809/go-fuzz-build/cover.go#L299-L319
On SSA level (e.g. how we do it in llvm) all of that complexity simply does not exist.
And we are talking about just 1 tool that tries to do moderately complex instrumentation.

[mvdan]

I think this could fall under tooling, so I'll CC @ianthehat for his thoughts.

The most reasonable interface from a tools perspective would be that it's invoked by go command and given a single package to transform (either in-place or to a new given location). But this also needs to work with cgo and allow parsing the package with go/types, including importing other packages.

Reminds me of go/packages, see ParseFile and Overlay in here:
https://godoc.org/golang.org/x/tools/go/packages#Config

Not saying that that package should solve this problem, but the source-to-source usecase seems quite similar to what go/packages does.

[josharian]

Another spot-fix idea: Just as we use //line comments to tell the compiler that the following line is really from somewhere else, we could use a //file comment at the top of a file to tell the compiler and cmd/go that the file is really from somewhere else. It seems like that might help with some of the compilation caching failures.

it's still an open question if s2s is the right approach. It may be a wrong level for such tools.

I was pondering that myself yesterday, for obvious reasons. But cmd/compile doesn't have a stable IR like llvm, and I don't think we are anywhere near ready to introduce one. Source is the only lingua franca, which makes it almost the only game in town.

Here's another (half-baked) idea. Write a single s2s preprocessor (compiler) that rewrites source code into a regular, minimal SSA form, making it simpler to deal with in all other tools. All vars would have explicit types when declared and systematic names (vNNN or phiNNN). Extra information (original var name?) could perhaps be conveyed in structured comments. All dot imports would be rewritten to be regular imports, all imports would be given explicit unique names for easier syntactic resolution. And so on. This is a bit vague, but I think it'd move us a lot closer from N*M to N+M.

[dvyukov]

I was pondering that myself yesterday, for obvious reasons. But cmd/compile doesn't have a stable IR like llvm, and I don't think we are anywhere near ready to introduce one. Source is the only lingua franca, which makes it almost the only game in town.

Well, there is a bunch of SSA passes in the compiler that somehow exist ;)

[josharian]

Well, there is a bunch of SSA passes in the compiler that somehow exist ;)

Right, but the set of ops, their semantics, etc. change from release to release. There is no serialization format, much less a stable one. And ssa.Values refer (via Aux) to other compiler structures like gc.Node, obj.LSym, which opens a giant can of worms.

[dvyukov]

Yet these passes achieve their goal. I mean it may not be a generic solution for allowing users to write arbitrary s2s transformations, but it still may allow to solve the problem at hand (fuzzing coverage). Just like race detector is an SSA pass.

[josharian]

Yes, we could declare fuzzing special enough to build into the toolchain. That's #19109. But we obviously can't do that for all tools.

Over in #19109, Russ asked repeatedly for go-fuzz to be prototyped out of tree to match how it would ultimately be integrated. I was experimenting with that. (Before anyone gets excited, it is enough work that I'm unlikely to follow through on it unless I find sponsorship, and I'm not planning to search for that at the moment.) Many of the stumbling blocks struck me as addressable (using go/packages, fixing corner cases, etc.) or merely requiring extra design (corpus location and migration), but this caching problem did not, which means that it is of independent interest for similar tools, and thus worth discussing independently.

Incidentally, cc @alandonovan re: my s2s SSA form pipe dream idea above, since that is very much his wheelhouse.

[dvyukov]

All of these problems (go/packages, corner cases, caching) simply don't exist if we go with compiler instrumentation. I think "go-fuzz prototype" includes only the interface user-facing part. So I would start with the interface part, the rest is implementation details.

[josharian]

But as someone who works on the compiler, I don't want more special things in it, I want less. They increase the testing surface area, make other work harder, etc.

And as someone who writes tools, I want the toolchain to be more flexible and welcoming to tools, not to wonder "how does test coverage do it?" and discover the answer is that it is special.

I want go-fuzz to be built-in to the toolchain. (Although I have to say, writing a good fuzz function is often surprisingly subtle.) But I also think it is worth trying to face some of the challenges it raises head on, in a general way. If it turns out there are no good answers available, sure, let's make it special. But let's at least try first.

[dvyukov]

Fair point.
But it still leaves the question open if it's really the right level for fuzzing coverage. I mean it will sure solve the problem of not making the compiler any more complex. But do we want to also move all of optimization passes, escape analysis and codegen out of the compiler to make it simpler and to allow external users to write their own replacements? Or do we want to write and maintain 2 mostly identical compilers/ssa-engines, one without stable API that will be used for compilation and another with stable API that will be used for coverage? Note that the second one needs be as elaborate as the first one, because, say, if we don't want to trace trivial inlined functions, then the compiler used for coverage will need to be able to do inlinability analysis and inlining yet preserving all original source:line info for the output. Or, if we want to transform y := x + 3; if y < 10 to if x < 7 for comparison tracing, we will need to be able to do pretty complex analysis and transformations, potentially across functions too. But if the second impl is as good as the first one, why do we want the first one at all? Or our strategy will be "no, all smart cool stuff is only for us in the compiler; and you are limited only to inserting full statements in a dumb way"?

[FiloSottile]

Last time we discussed fuzzing with @rsc, I think he argued it should not get special compiler support, but just use s2s transformations and live outside of the go tool, at least initially. As I understood it it was a sustainability argument, since as the project grow we can’t add and maintain support to the compiler for everything, but we can add reliable support for s2s.

I agree with that view, and I think it’s consistent with the direction set by go generate. We should fix this.

(I even have a pipe dream of extracting cgo and maybe coverage into external, forkable tools, where they can be adapted to more niche use cases, instead of having to live with the tradeoffs we selected for the go tool.)

[dvyukov]

@FiloSottile what types of tools/transformations have you considered as a use case?

I even have a pipe dream of extracting cgo and maybe coverage into external, forkable tools

Well, if the current source coverage is not trivially expressible in this framework, then it's not working and we failed. I would convert source coverage just as a proof of concept because any such framework badly needs realistic use cases, so it would happen just as a side effect.