x/tools/go/packages: Packages.{Syntax,TypeInfo} fields are empty unless NeedTypes

[adonovan]

This program, which uses go/packages:

package main

import (
	"fmt"
	"log"
	"os"

	"golang.org/x/tools/go/packages"
)

func main() {
	mode := packages.NeedSyntax | packages.NeedTypesInfo | packages.NeedTypesSizes
	f(mode)
	f(mode | packages.NeedTypes)
}

func f(mode packages.LoadMode) {
	cfg := &packages.Config{Mode: mode}
	pkgs, err := packages.Load(cfg, "fmt")
	if err != nil {
		log.Fatal(err)
	}
	if packages.PrintErrors(pkgs) > 0 {
		os.Exit(1)
	}
	fmt.Printf("Mode: %s\n", mode)
	fmt.Printf("TypesInfo = %p\n", pkgs[0].TypesInfo)
	fmt.Printf("Types = %p\n", pkgs[0].Types)
	fmt.Printf("Syntax = %v\n", pkgs[0].Syntax)
	fmt.Println()
}

produces this output:

Mode: LoadMode(NeedSyntax|NeedTypesInfo|NeedTypesSizes)
TypesInfo = 0x0
Types = 0x0
Syntax = []

Mode: LoadMode(NeedTypes|NeedSyntax|NeedTypesInfo|NeedTypesSizes)
TypesInfo = 0x1400023b770
Types = 0x1400032e000
Syntax = [0x1400064c000 0x140001f0000 0x1400064c090 0x140001f0090 0x140000da090]

Why does absence of the NeedTypes mode flag cause the Syntax and TypesInfo fields, both explicitly "needed", to be missing? I think that's a bug.

This is not the first such bug like this that we've seen, and in my experience the fine-grained mode bits seem to demand a great deal of experimentation before one finds something that works. Also, they create a powerset of paint shades of Package structures of varying degrees of population, which means a function that receives a Package doesn't really know what it's going to get. In practice almost all clients want one of four things:

• just the metadata for one package +/- its deps;
• the syntax of one package;
• syntax and complete type information for one package (with deps from export data);
• complete typed syntax for everything.

If memory serves a lot of the hair splitting was because of the surprising costs of some fields (e.g. package name?) in Bazel/Blaze. I'm not convinced it was worthwhile, and I wish we could un-deprecate the old compound modes and improve their documentation (perhaps with some better-named aliases). A few compound modes would cover the vast majority of practical cases, with the option of fine-grained bits for the brave and foolhardy.

[dmitshur]

CC @matloob.

[gopherbot]

Change https://go.dev/cl/588141 mentions this issue: go/packages: document fields that are part of JSON schema