cmd/go: incorrect pseudo-versions for merge commits

[jefferai]

What version of Go are you using (go version)?

$ go version
go version devel +4b3f04c63b Thu Jan 10 18:15:48 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes. Tested on 1.11.4 and tried with 1.12-beta2.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jeff/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jeff/go"
GOPROXY=""
GORACE=""
GOROOT="/home/jeff/src/go"
GOTMPDIR=""
GOTOOLDIR="/home/jeff/src/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jeff/hc/vault/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build737217089=/tmp/go-build -gno-record-gcc-switches"

What did you do?

When trying to convert a large program from govendor to go modules, I'm seeing behavior where many of the dependencies are being overwritten after I set them; a version using a git commit will be overwritten by the latest tag, even if that latest tag is extremely old.

Invoking any go command causes this overwriting.

This only seems to affect specified revisions that are merge commits (as in, the product of a PR merge). When the merge commit is used the version is v0.0.0-<timestamp>-<revision>. When I use the non-merge commit, it shows up with v0.9.2-<timestamp>-<revision>.

What did you expect to see?

Once I set a version, it is left alone.

What did you see instead?

(You can follow along in the gomodtry2 branch in github.com/hashicorp/vault if you like)

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.1

$ go get github.com/SermoDigital/jose@803625baeddc

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go mod why github.com/SermoDigital/jose
go: finding github.com/influxdata/platform/models latest
go: finding github.com/influxdata/platform latest
# github.com/SermoDigital/jose
github.com/hashicorp/vault/audit
github.com/SermoDigital/jose/jws
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.1

$ go get github.com/SermoDigital/jose@a0450ddff675

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.2-0.20180104202408-a0450ddff675

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/hashicorp/vault/audit
github.com/SermoDigital/jose/jws
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.2-0.20180104202408-a0450ddff675

$ go get github.com/SermoDigital/jose@803625baeddc

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go mod why github.com/SermoDigital/jose
go: finding github.com/influxdata/platform/models latest
go: finding github.com/influxdata/platform latest
# github.com/SermoDigital/jose
github.com/hashicorp/vault/audit
github.com/SermoDigital/jose/jws
github.com/SermoDigital/jose

 $ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.1

[jefferai]

The inconsistent part of the issue subject is that I created a dead simple test program:

package main

import "github.com/SermoDigital/jose/jwt"

func main() {
        var _ jwt.JWT
}

and I do not see the behavior here:

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.9.1

$ go get github.com/SermoDigital/jose@803625baeddc

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/jefferai/modbug
github.com/SermoDigital/jose/jwt
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go get github.com/SermoDigital/jose@a0450ddff675

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/jefferai/modbug
github.com/SermoDigital/jose/jwt
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.9.2-0.20180104202408-a0450ddff675

$ go get github.com/SermoDigital/jose@803625baeddc

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/jefferai/modbug
github.com/SermoDigital/jose/jwt
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

Other than "the other go.mod file has many more deps and is more complicated" I can't figure out why I would see different behavior given the same set of commands and revisions for the same dep.

[ChrsMark]

+1

Having exactly the same issue. This can be really painful in my case since CIs are dependent on the fact that dependencies are located in versioned directories, so extracting them to another format break things.

[bcmills]

You are running into two separate issues here.

One is #27171: for some reason, your module cache maps commit 803625baeddc to version v0.0.0-20180104203859-803625baeddc. With a clean module cache, I'm seeing that map to version v0.9.2-0.20180104203859-803625baeddc instead.

The other issue is that you have a dependency on github.com/influxdata/platform. The latest version of github.com/influxdata/platform has a go.mod file that requires github.com/SermoDigital/jose v0.9.1. When you try to go get a version older than that, the go command attempts to downgrade all other modules to compatible versions, and because github.com/influxdata/platform does not have version tags, the only downgrade that go get can perform is to remove it entirely.

Once the dependency on github.com/influxdata/platform has been removed, you keep the dependency on the older-versioned github.com/SermoDigital/jose until the next time something tries to resolve the import from github.com/influxdata/platform. At that point, you pick up the latest version of github.com/influxdata/platform again, and that upgrades github.com/SermoDigital/jose back to v0.9.1.

vault$ rm go.sum

vault$ go mod tidy
[…]

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.1

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/jose@v0.9.1
github.com/influxdata/platform@v0.0.0-20181208014635-df0b08454316 github.com/SermoDigital/jose@v0.9.1

vault$ go get github.com/SermoDigital/jose@803625baeddc
go: finding github.com/SermoDigital/jose 803625baeddc
go: downloading github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc
go: extracting github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/jose@v0.9.2-0.20180104203859-803625baeddc
github.com/influxdata/platform@v0.0.0-20181208014635-df0b08454316 github.com/SermoDigital/jose@v0.9.1

vault$ go mod tidy

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc

vault$ go get github.com/SermoDigital/jose@6686625
go: finding github.com/SermoDigital/jose 6686625
go: downloading github.com/SermoDigital/jose v0.0.0-20151001034355-66866257aa55
go: extracting github.com/SermoDigital/jose v0.0.0-20151001034355-66866257aa55

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.0.0-20151001034355-66866257aa55

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/jose@v0.0.0-20151001034355-66866257aa55

vault$ go list -m github.com/influxdata/...
github.com/influxdata/flux v0.7.2
github.com/influxdata/influxdb v1.7.1
github.com/influxdata/influxql v0.0.0-20180925231337-1cbfca8e56b6
github.com/influxdata/line-protocol v0.0.0-20180522152040-32c6aa80de5e
github.com/influxdata/tdigest v0.0.0-20181121200506-bf2b5ad3c0a9
github.com/influxdata/usage-client v0.0.0-20160829180054-6d3895376368

vault$ go mod tidy
go: finding github.com/influxdata/platform/models latest
go: finding github.com/influxdata/platform latest
go: downloading github.com/influxdata/platform v0.0.0-20190117200541-d500d3cf5589
go: extracting github.com/influxdata/platform v0.0.0-20190117200541-d500d3cf5589
go: finding github.com/circonus-labs/circonusllhist v0.1.3
go: finding github.com/influxdata/flux v0.13.0
go: finding github.com/circonus-labs/circonus-gometrics v2.2.5+incompatible
go: finding github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82
go: finding github.com/benbjohnson/tmpl v1.0.0
go: finding github.com/yudai/pp v2.0.1+incompatible
go: finding github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88
go: finding github.com/golang/gddo v0.0.0-20181116215533-9bd4a3295021
go: finding github.com/apache/arrow/go/arrow v0.0.0-20181217213538-e9ed591db9cb
go: finding github.com/goreleaser/goreleaser v0.94.0
go: finding github.com/yudai/gojsondiff v1.0.0
go: finding github.com/dustin/go-humanize v1.0.0
go: finding github.com/miekg/dns v1.1.1
go: finding gopkg.in/src-d/go-git.v4 v4.8.1
go: finding github.com/aokoli/goutils v1.0.1
go: finding github.com/Masterminds/sprig v2.16.0+incompatible
go: finding github.com/aws/aws-sdk-go v1.15.64
go: finding github.com/apache/arrow/go/arrow v0.0.0-20181031164735-a56c009257a7
go: finding github.com/huandu/xstrings v1.0.0
go: finding github.com/stretchr/testify v1.2.0
go: finding github.com/xanzy/ssh-agent v0.2.0
go: finding github.com/emirpasic/gods v1.9.0
go: finding github.com/gliderlabs/ssh v0.1.1
go: finding golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a
go: finding golang.org/x/sys v0.0.0-20181030150119-7e31e0c00fa0
go: finding github.com/kevinburke/ssh_config v0.0.0-20180830205328-81db2a75821e
go: finding gopkg.in/src-d/go-billy.v4 v4.2.1
go: finding gopkg.in/src-d/go-git-fixtures.v3 v3.1.1
go: finding golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9
go: finding golang.org/x/crypto v0.0.0-20180505025534-4ec37c66abab
go: finding github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
go: finding github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
go: finding github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7
go: finding github.com/src-d/gcfg v1.4.0
go: finding github.com/pelletier/go-buffruneio v0.2.0
go: finding gopkg.in/warnings.v0 v0.1.2
go: finding github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: finding golang.org/x/sys v0.0.0-20180903190138-2b024373dcd9
go: finding golang.org/x/tools v0.0.0-20181221154417-3ad2d988d5e2
go: downloading github.com/aws/aws-sdk-go v1.15.64
go: downloading golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a
go: downloading golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9
go: extracting golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9
go: extracting golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a
go: extracting github.com/aws/aws-sdk-go v1.15.64
go: downloading github.com/miekg/dns v1.1.1
go: downloading github.com/circonus-labs/circonus-gometrics v2.2.5+incompatible
go: extracting github.com/circonus-labs/circonus-gometrics v2.2.5+incompatible
go: downloading github.com/circonus-labs/circonusllhist v0.1.3
go: extracting github.com/circonus-labs/circonusllhist v0.1.3
go: extracting github.com/miekg/dns v1.1.1

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.1

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/jose@v0.9.1
github.com/influxdata/platform@v0.0.0-20190117200541-d500d3cf5589 github.com/SermoDigital/jose@v0.9.1

[bcmills]

Are you able to reproduce this issue using a clean module cache (go clean -modcache)?

If so, could you post the output of git --version? There may be a bad interaction between go and git that is interfering with our ability to infer the correct pseudo-version of merge commits.

[jefferai]

I'll give it a try with a wiped module cache. However, one problem with your analysis:

The latest version of github.com/influxdata/platform has a go.mod file that requires github.com/SermoDigital/jose v0.9.1. When you try to go get a version older than that, the go command attempts to downgrade all other modules to compatible versions ....... At that point, you pick up the latest version of github.com/influxdata/platform again, and that upgrades github.com/SermoDigital/jose back to v0.9.1

The issue here is that go mod is assuming that 0.9.1 is in fact newer than v0.0.0-20180104203859-803625baeddc -- but in fact it's way older. I would think looking at the commit date for the given tag and comparing it against the existing commit date would cause it to realize that -- but maybe it's due to the bug you mentioned.

[jefferai]

I can confirm that with a cleaned modcache this works. I have no idea how it got unclean, but...there you go. I'll close the bug -- thanks for the response!