cmd/go: tries to resolve github.com as a module

[myitcv]

What version of Go are you using (go version)?

$ go version
go version devel +c4ca60313e Sun Jul 28 18:09:57 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOENV="/home/myitcv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/tmp/tmp.4LfxYnl2lH"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/myitcv/gos"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/tmp.Byzg7Tw4cq/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build167955296=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Simply reproduced via the following (with a clean GOPATH):

$ go get github.com/myitcv/[email protected]
go: finding github.com/myitcv/gobin v0.1.11
go: finding github.com/myitcv/gobin v0.1.11
go: finding github.com v0.1.11
go: finding github.com/myitcv v0.1.11
go get github.com/myitcv/[email protected]: github.com/myitcv/[email protected]: invalid version: unknown revision v0.1.11

v0.1.11 does not exist; that's expected.

What did you expect to see?

No attempt to resolve github.com or github.com/myitcv

What did you see instead?

As above.

cc @bcmills @jayconrod

[jayconrod]

Is the issue that we're trying to resolve github.com and github.com/myitcv specifically? It would be reasonable to add a special case for github.com, since we know any module will have at least two path components after the domain there. However, I don't think we have any special cases here, and I'd be hesitant to start adding them. It would be nice to keep the proxy protocol very simple.

Or is the issue that, having discovered github.com/myitcv/gobin is a module (without v0.1.11), we stop searching for other modules with prefixes of that path? This part may be working as intended. Anecdotally, I've heard of packages moved from a root module to a nested module (I think this was in github.com/hashicorp/vault). There are overlapping versions across those modules, and it would not be safe to stop the search early.

[bcmills]

Note that avoiding those particular paths is just an optimization, and the requests are made in parallel so the optimization would have relatively little effect on overall latency.

(To my knowledge, there is nothing currently stopping GitHub from using <meta name="go-import" […]> tags to publish their own module at the root of github.com, or to allow users to publish their own modules at github.com/$USER.)

[bcmills]

There is an interesting implication for program encapsulation, in that a top-level github.com module could potentially provide packages that allow access to the internal paths of other modules hosted on github.com.

However, the presence of such a module would be relatively easy to detect: if it can be fetched through proxy.golang.org then it will be recorded in the public index (at index.golang.org), and if it cannot be fetched through proxy.golang.org then it will also not be recorded in the checksum database (and go1.13 will error out at that step).

(Besides, projects hosted on github.com are already trusting GitHub to serve unadulterated contents for their modules. Trusting GitHub not to serve a malicious top-level module doesn't seem like much of a stretch.)

[myitcv]

Ah, it could be my misunderstanding. I thought I remembered cmd/go stopping at github.com/X/Y and not going any higher... hence why I raised the issue. But my memory might well be failing me!

[hyangah]

@myitcv I believe this is the recent change in go1.13. For users like me who don't know the underlying go command implementation the message "go: finding github.com v0.1.11" will be quite surprising. Shouldn't the message be hidden by default?

[bcmills]

@hyangah, that's more the purview of #26152. (See #26152 (comment) for my current thoughts on that.)

[bcmills]

Since as far as we're aware the impact on overall latency is negligible, we don't plan to change this — I don't think it would even be worth the code complexity as a contributed change. Closing.