Can't build after a private module has moved

[kstenerud]

What version of Go are you using (go version)?

$ go version
go version go1.13.5 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/karl/.cache/go-build"
GOENV="/home/karl/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/karl/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/karl/work/newhotness.io/agent/cli/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build742242685=/tmp/go-build -gno-record-gcc-switches"

Issue Description

I'm trying to build a project that includes a private module whose location has changed.

In my source files, I changed import "github.com/oldwaycorp/mysupermodule" to import "github.com/newhotnessio/mysupermodule", then attempted to build:

$ go build
build github.com/newhotnessio/newhotness.io/agent/cli: cannot load github.com/newhotnessio/mysupermodule: git ls-remote -q https://github.com/newhotnessio/mysupermodule in /home/karl/go/pkg/mod/cache/vcs/cb524ab9ca473eeb1a039701126a9218ddbb7854fdc95ac3de2e831573ad13e4: exit status 128:
	fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

According to https://golang.org/doc/faq#git_https I should add this to .gitconfig:

[url "ssh://[email protected]/"]
	insteadOf = https://github.com/

Try building again:

$ go build
go: finding github.com/newhotnessio/mysupermodule latest
verifying github.com/newhotnessio/[email protected]: github.com/newhotnessio/[email protected]: reading https://sum.golang.org/lookup/github.com/newhotnessio/[email protected]: 410 Gone

Searching around online, I find this: https://medium.com/mabar/today-i-learned-fix-go-get-private-repository-return-error-reading-sum-golang-org-lookup-93058a058dd8

Apparently, I need to use GOPRIVATE:

$ GOPRIVATE="github.com/newhotnessio/mysupermodule" go build
go: finding github.com/newhotnessio/mysupermodule latest
go: github.com/newhotnessio/newhotness.io/agent/cli imports
	github.com/newhotnessio/mysupermodule: github.com/newhotnessio/[email protected]: parsing go.mod:
	module declares its path as: github.com/oldwaycorp/mysupermodule
	        but was required as: github.com/newhotnessio/mysupermodule

I'm not sure where it's getting oldwaycorp from. grep -r oldwaycorp returns no results.

After some digging around in various blogs and reddit, I tried adding this to go.mod:

replace github.com/newhotnessio/oldwaycorp/mysupermodule => github.com/newhotnessio/newhotnessio/mysupermodule v0.0.0

Then attempted a build:

$ GOPRIVATE="github.com/newhotnessio/mysupermodule" go build
go: finding github.com/newhotnessio/mysupermodule latest
go: github.com/newhotnessio/newhotness.io/agent/cli imports
	github.com/newhotnessio/mysupermodule: github.com/newhotnessio/[email protected]: parsing go.mod:
	module declares its path as: github.com/oldwaycorp/mysupermodule
	        but was required as: github.com/newhotnessio/mysupermodule

Nope, that didn't work. More googling finds #35723

Looks like I'm also supposed to use the OLD import path, not the new one. I change my source files back to use the old import path ad try again:

$ GOPRIVATE="github.com/newhotnessio/mysupermodule" go build
build github.com/newhotnessio/newhotness.io/agent/cli: cannot load github.com/oldwaycorp/mysupermodule: git ls-remote -q https://github.com/oldwaycorp/mysupermodule in /home/karl/go/pkg/mod/cache/vcs/a93c8f67f51ce43e15b90b0308fb1c456d20b47395571ae1d7a8435ebfc53ef6: exit status 128:
	ERROR: Repository not found.
	fatal: Could not read from remote repository.
	
	Please make sure you have the correct access rights
	and the repository exists.

Now it's trying to fetch from the old repository, which of course it can't because it's supposed to be fetching from the new repository. Isn't the replace directive supposed to redirect to the new repository?

Aside from the terrible error messaging in all of this, I'm still left with an unbuildable codebase. How should I handle a private repo that has moved?

[bcmills]

When you change the import path, you must also change the path declared in the go.mod file.

Replace

module github.com/oldwaycorp/mysupermodule

with

module github.com/newhotnessio/mysupermodule

and it should be fine without a replace directive.

[kstenerud]

You mean in the module I'm trying to import?

The module is a mirror of a module internal to another department. We used to have access to it, but now have to use the mirror. They export to this repo whenever they're stable, so I have no control over its go.mod file.

I just want to import and use it.

[bcmills]

Ok. In that case, you do need something like

replace github.com/oldwaycorp/mysupermodule => github.com/newhotnessio/mysupermodule v0.0.0-20191128115051-441c255ee06a

and to import via the old import path.

Are you sure the paths in your replace directive were entered correctly? (What does go list -m github.com/oldwaycorp/mysupermodule report?)

[kstenerud]

OK, I set this in my go.mod:

replace github.com/oldwaycorp/mysupermodule => github.com/newhotnessio/mysupermodule v0.0.0

And my imports now look like this:

import "github.com/oldwaycorp/mysupermodule"

Doing a go list:

$ go list -m github.com/oldwaycorp/mysupermodule
go list -m: module github.com/oldwaycorp/mysupermodule: not a known dependency

Doing a go build:

$ go build
go: downloading github.com/newhotnessio/mysupermodule v0.0.0
build github.com/newhotnessio/newhotness.io/agent/cli: cannot load github.com/oldwaycorp/mysupermodule: reading github.com/newhotnessio/mysupermodule/go.mod at revision v0.0.0: unknown revision v0.0.0

[bcmills]

You need a real version (or a real commit hash) on the right-hand side of that replace directive. v0.0.0 won't do unless you have an actual v0.0.0 tag.

[kstenerud]

There are no tags in that repository. What would I put in?

[bcmills]

A commit hash, or latest might work (I'm not entirely sure).

[kstenerud]

OK, putting in latest brings me to:

go: finding github.com/newhotnessio/mysupermodule latest
verifying github.com/newhotnessio/[email protected]: github.com/newhotnessio/[email protected]: reading https://sum.golang.org/lookup/github.com/newhotnessio/[email protected]: 410 Gone

So it's not able to see the repository for some reason?

[bcmills]

You still need to set GOPRIVATE for github.com/newhotnessio/mysupermodule if it is not publicly accessible.

[kstenerud]

Ah right, that worked :)

[bcmills]

All set then? (Can we close this issue?)

FYI @tbpg @jadekler @jayconrod: this workflow might be something to bear in mind for documentation examples.

[kstenerud]

Yup that works, thanks. Although for future I think the error messages need tweaking. It was impossible on my own to figure out what was wrong.