proposal: Configurable Post Processors

[tinyzimmer]

What do you want to change?

This idea comes from various open and closed issues. Most recently #1485 (which I jumped in at the last minute to help push over the finish line).

The idea is simple - the implemetation probably not so much. So, I was hoping to open an Issue to put forth the proposal and flesh out the idea before venturing into implementation.

---

Proposal: An extension to the configuration that allows for running arbitrary post-processors against generated code.

This could take one of two forms in my head:

• Simple stdin/stdout approach : Where essentially the input could be passed as stdin to some arbitrary command string and the output replaced with whatever it produces on its stdout.

This method would probably be pretty easy to implement (relatively speaking), but would come with limitations that I'll expand on in the second approach.

• A plugin model built around a public interface in the code. Ideally with support for third-party plugins (perhaps via an RPC interface such as used in Hashicorp Vault), but that wouldn't be as crucial out of the gate. The interface could look something like (pseudo-code)

type PostProcessor interface {
    Render(Context) (io.ReadCloser, error)
}

Where the Context contains AST information and other metadata for the file being rendered. A built-in reference implementation could be a simple search/replace and/or type override. A custom plugin could have satisfied the NullableEnum issue until support was included upstream. This makes things much more flexible for the post-processor and runs into far fewer limitations.

If you are open to either of these proposals, I'm happy to help see them to reality. I feel like this would end up having considerable overlap with some of the existing functionality regarding the child directives in the gen config. But perhaps there is a way for it to fit into that paradigm as well, and then some of those directives could be implemented as going through a chain of "Post Processors".

---

Some additional context regarding this idea. One thing I've done for an internal project (though this may become open source) is a post-processor (that has to be invoked after sqlc) that generates most of a GraphQL schema from the generated code using
https://github.com/graphql-go/graphql.

One of the pain points in that generator, is it has to effectively rebuild its own Go AST by parsing the sqlc output. If a plugin interface like described above existed, the implementation would be much cleaner.

What database engines need to be changed?

PostgreSQL, MySQL

What programming language backends need to be changed?

Go, Python, Kotlin

[kyleconroy]

Support for codegen plugins (both WASM and process-based) was merged recently (#1684, #1578). As we build out documentation and examples for those, I have no plans to add additional plugins or post-processors.

The existing plugin system is designed after protoc in which plugins are additive; they can't change the output of other plugins. While that may sound limiting, it's too difficult to version the actual Go AST. sqlc reserves the right to change internal implementation details.