Skip to content

x/tools/go/types/objectpath: add Encoder type, to amortize Scope.Names #58668

Closed
@adonovan

Description

@adonovan

The objectpath.For function calls Scope.Names, which allocates and sorts a slice of all package members. If For is called for every member of a package (or for every declaration within a package, which is much more), then it incurs an amount of allocation that is quadratic in the number of package members. Some packages (e.g. for CPU instructions sets) are very large, and objectpath loops become so slow they appear to be stuck.

I propose we add a new Encoder type with a For method so that new(Encoder).For gives the old behavior, but re-using the encoder across multiple calls for objects in the same package amortizes the expensive steps like Scope.Names.

See https://go.dev/cl/470679 for the expedient workaround in gopls (merged).
See https://go.dev/cl/470678 for the change to the public API.

Activity

added
ToolsThis label describes issues relating to any tools in the x/tools repository.
on Feb 23, 2023
added this to the Unreleased milestone on Feb 23, 2023
gopherbot

gopherbot commented on Feb 23, 2023

@gopherbot
Contributor

Change https://go.dev/cl/470679 mentions this issue: go/types/objectpath: add encoder type, to amortize Scope.Names

gopherbot

gopherbot commented on Feb 23, 2023

@gopherbot
Contributor

Change https://go.dev/cl/470678 mentions this issue: go/types/objectpath: add Encoder type, to amortize Scope.Names

findleyr

findleyr commented on Feb 28, 2023

@findleyr
Member

To add additional context: we simply wouldn't be able to use the objectpath package in gopls without this optimization.

Furthermore, I think Encoder is a good name.

moved this to Incoming in Proposalson Mar 1, 2023
rsc

rsc commented on Mar 8, 2023

@rsc
Contributor

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

moved this from Incoming to Active in Proposalson Mar 8, 2023
rsc

rsc commented on Mar 15, 2023

@rsc
Contributor

The API is

package objectpath // import "golang.org/x/tools/go/types/objectpath"

func For(obj types.Object) (Path, error) {
	return new(Encoder).For(obj)
}

// An Encoder amortizes the cost of encoding the paths of multiple objects.
// The zero value of an Encoder is ready to use.
type Encoder struct {
	...
}

func (enc *Encoder) For(obj types.Object) (Path, error) 

Are there any concerns about this API?

rsc

rsc commented on Mar 29, 2023

@rsc
Contributor

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

moved this from Active to Likely Accept in Proposalson Mar 29, 2023

23 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @dominikh@rsc@timothy-king@adonovan@gopherbot

        Issue actions

          x/tools/go/types/objectpath: add Encoder type, to amortize Scope.Names · Issue #58668 · golang/go