x/tools/gopls: bad state after creating new directory and package

[atombender]

What did you do?

Created a file and typed a bunch of code. After saving, gopls correctly added an import to net/http.

What did you expect to see?

No errors.

What did you see instead?

However, it still displays a marker for the error undeclared name: http, and on the wrong line.

This kind of thing happens fairly commonly (and I just restart gopls), but in this case it was a brand new file, so the section of log starts at the initial opening of the file, so hopefully this has all relevant log messages to diagnose the problem.

Log

gopls.log.gz

Build info

golang.org/x/tools/gopls v0.3.4
    golang.org/x/tools/gopls@(devel)
    github.com/BurntSushi/[email protected] h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/[email protected] h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/[email protected] h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/[email protected] h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
    golang.org/x/[email protected] => ../
    golang.org/x/[email protected] h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/[email protected] h1:sXmLre5bzIR6ypkjXCDI3jHPssRhc8KD/Ome589sc3U=
    mvdan.cc/xurls/[email protected] h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info

go version go1.14 darwin/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/alex/Library/Caches/go-build"
GOENV="/Users/alex/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="github.com/t11e/*"
GONOSUMDB="github.com/t11e/*"
GOOS="darwin"
GOPATH="/Users/alex/.go"
GOPRIVATE="github.com/t11e/*"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/opt/go/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/qs/wpmg19r12_9_nz7pvvs2_82r0000gn/T/go-build683361864=/tmp/go-build -gno-record-gcc-switches -fno-common"

[stamblerre]

Thank you for the report and the log!

A few quick thoughts / questions:

• Is the client.go file the only file in the package?
• I noticed in the log is that none of the textDocument/codeAction requests have a non-empty response (code actions are used for import organization), but the "net/http" import does get added. If you can remember - did you add net/http by accepting a completion or do you have any other VS Code extensions installed that might be adding this import?
• The main issue seems to begin after the didChange to add import "net/http". We seem to try to reload the package command-line-arguments, which is a placeholder package path for an ad-hoc package (a package that is outside of GOPATH or a Go module).

[Info  - 8:37:28 PM] 2020/03/20 20:37:28 go/packages.Load
	snapshot = 1975
	query = [command-line-arguments]
	packages = 1
[Trace - 20:37:28.546 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2020/03/20 20:37:28 go/packages.Load\n\tsnapshot = 1975\n\tpackage = command-line-arguments\n\tfiles = []"}


[Info  - 8:37:28 PM] 2020/03/20 20:37:28 go/packages.Load
	snapshot = 1975
	package = command-line-arguments
	files = []
2020/03/20 20:37:28 diagnose: could not generate diagnostics for go.mod file: command-line-arguments has no metadata

It seems you are using modules - are you opening VS Code at the root of your module or a subdirectory? It seems like we're treating your package as an ad-hoc package.

Something we can definitely do is make sure that we don't try to reload packages with the path command-line-arguments as that is always wrong.

[atombender]

Is the client.go file the only file in the package?

Yes.

... did you add net/http by accepting a completion or do you have any other VS Code extensions installed that might be adding this import?

I typed *http.Client and hit "save" to let the formatter kick in. No other extensions.

It seems you are using modules - are you opening VS Code at the root of your module or a subdirectory?

Root, though it's a workspace with multiple folders. In case it's relevant, the two other folders are a Go project (with go.mod) and a non-Go project (no go.mod).

[stamblerre]

I typed *http.Client and hit "save" to let the formatter kick in. No other extensions.

Hm, based on the log that doesn't seem to be the case. Would you mind sharing a log where you follow that same workflow just so that I can confirm if things are working properly?

Root, though it's a workspace with multiple folders. In case it's relevant, the two other folders are a Go project (with go.mod) and a non-Go project (no go.mod).

Thanks for the confirmation. It sounds like gopls may not be loading your workspace correctly. Would you mind sharing the logs starting from the very beginning of the gopls instance when you open that workspace?

[atombender]

I am actually able to reproduce the bug consistently, doing exactly what I described:

1. Create a new folder client.
2. Create an empty file called client.go.
3. Type:

package client

type Client struct {
  hc *http.Client
}

4. Hit Save.

Here's a gopls log from doing the above, and nothing else: gopls.log.gz.

[stamblerre]

Thanks for attaching the log! It's helped me notice a few deficiencies in the way we log for multi-project workspaces, so I will mail a change to address some of them soon.

Another question about your workspace - it seems like only one go.mod workspace was detected by gopls, but it sounds like you expect there to be 2 - is that correct?

Also, are you able to reproduce this if you only open that project instead of your multi-project workspace?

[atombender]

but it sounds like you expect there to be 2 - is that correct?

Yes.

Also, are you able to reproduce this if you only open that project instead of your multi-project workspace?

No.

[atombender]

I am also able to reproduce the bug if I have only one project in the workspace. So it's not related to the other project in the workspace that also has a go.mod file.

[gopherbot]

Change https://golang.org/cl/225117 mentions this issue: internal/lsp: never reload command-line-arguments directly

[gopherbot]

Change https://golang.org/cl/225137 mentions this issue: internal/lsp: print view-specific environment

[stamblerre]

Thanks! I've mailed the above changes to address some of the immediate issues I noticed from the logs, so I will ask you to try again after those are merged.

If you'd like to try one more thing in the meantime, could you create a repro log with a single-project workspace? It might make it a bit easier for me to parse your log. If not, totally fine to wait until after those changes are merged. Thanks for your help on this!

[stamblerre]

These changes are now merged, and you can try them out by updating to gopls at master (GO111MODULE=ON go get golang.org/x/tools/gopls@master golang.org/x/tools@master). Please let me know if these fixes addressed the problem, and if not, please attach new logs. Thanks for your patience!

[atombender]

Thanks! Unfortunately, no difference. Here's a log.

I will get you a log from a single-project workspace.

[atombender]

Uh, I don't know what happened. After reloading the VSCode window (Developer: Reload window) a few times, I am no longer able to reproduce in the same workspace. I definitely upgraded before. Will do some more experimenting.

[atombender]

Reloaded the window, and am once again able to reproduce it, even in a workspace that contains just the one Go project: gopls.log.gz. I'm no longer not able to reproduce it.

[atombender]

By the way, I'm installing from source: git reset --hard origin/master && go install .. Installing via go get doesn't work for me:

$ GO111MODULE=on go get golang.org/x/tools/gopls@master
go: downloading golang.org/x/tools/gopls v0.1.8-0.20200324201824-1fc30e1f4ccc
go: golang.org/x/tools/gopls master => v0.1.8-0.20200324201824-1fc30e1f4ccc
go: downloading golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d
# golang.org/x/tools/gopls/internal/hooks
.go/pkg/mod/golang.org/x/tools/[email protected]/internal/hooks/hooks.go:19:9: options.URLRegexp undefined (type *source.Options has no field or method URLRegexp)

Latest commit:

$ git log -1 --no-abbrev
commit 1fc30e1f4ccc2986b7ad007d40afcf96be78a429 (HEAD, origin/master, origin/HEAD)
Author: Rob Findley <[email protected]>
Date:   Mon Mar 23 17:26:05 2020 -0400

    internal/lsp/regtest: remove redundant T and ctx params from regtest funcs

[stamblerre]

Thanks for the log! I am now able to reproduce this issue myself. I'll aim to have a fix soon.

By the way, some of the error messages in your logs (no matching versions for query "latest") would likely stop if you added your project to your GOPRIVATE environment variable. Also, the command to get gopls from master is GO111MODULE=ON go get golang.org/x/tools/gopls@master golang.org/x/tools@master.

[atombender]

Super, thank you.

your GOPRIVATE environment variable

Good catch, I didn't realize that!

GO111MODULE=ON go get golang.org/x/tools/gopls@master golang.org/x/tools@master

Oh, I didn't realize it took multiple arguments. For global installs, it maybe resolves them together as if they were in the same go.mod/go.sum?

[gopherbot]

Change https://golang.org/cl/225317 mentions this issue: internal/lsp: handle creation of a new workspace-level directory