cmd/go: go mod init fails to retrieve company.github.com ssl only repositories

[MelleKoning]

What version of Go are you using (go version)?

go version go1.12 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GOARCH=amd64
set GOBIN=C:\git\go\bin
set GOCACHE=C:\Users\xxx\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=c:\git\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\git\notgo\engineering\veritasapi\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\xxx\AppData\Local\Temp\go-build238261902=/tmp/go-build -gno-record-gcc-switches

What did you do?

go mod init mymodule

this starts reading the current used glide.yaml to process project dependencies.

There are some imports in this go project that refer to a company located git-host which is accessible via git using private ssl certificates

for exampe in code:

import "oucompany.github.com/engineering/myrepo/package"

whereby in the glide.yaml we might have:

- package: github.ourcompany.local/engineering/go-utils
  repo: git@github.ourcompany.local:engineering/go-utils.git
  vcs: git
  subpackages:
  - env

What did you expect to see?

A properly initialized go.mod

What did you see instead?

errors/ exception thrown like:

$ go mod init veritasapi
go: creating new go.mod: module veritasapi
go: copying requirements from glide.lock
go: converting glide.lock: stat github.mycompany.local/engineering/protos@7462cb9dbaba285e3c931bc1edc0b62ebe8d005e: git fetch -f https://github.mycompany.local/engineering/protos.git refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in c:\git\go\pkg\mod\cache\vcs\5626dfafc13719b661e5e62c892d3563fa39030a15a5ca1fdcad3e06591ea342: exit status 128:
        fatal: unable to access 'https://github.mycompany.local/engineering/protos.git/': schannel: next InitializeSecurityContext failed: Unknown error (0x80092013) - The revocation function was unable to check revocation because the revocation server was offline.

Also tried to change the .gitconfig setting:

[url "ssh://git@github.ourcompany.local:engineering/"]
        insteadOf = https://github.ourcompany.local/engineering/

but this does not show any noticable difference.

Also ensured the repository is available/readable on console:

>git ls-remote git@github.ourcompany.local:engineering/mygitrepo

outputs latest commit hashes

Question: how come

go mod init

is trying to access the git repository using https instead of via ssl / git@etc..?

[agnivade]

Question: how come go mod init is trying to access the git repository using https instead of via ssl / git@etc..?

See https://golang.org/doc/faq#git_https.

/cc @bcmills

[MelleKoning]

Question: how come go mod init is trying to access the git repository using https instead of via ssl / git@etc..?

See https://golang.org/doc/faq#git_https.

/cc @bcmills

Hi @agnivade thanks for that reference. As from the docs:

Git can also be configured to use SSH in place of HTTPS for URLs matching a given prefix. For example, to use SSH for all GitHub access, add these lines to your ~/.gitconfig:

[url "ssh://git@github.com/"]
insteadOf = https://github.com/

This has been tried but it seems the 'insteadOf' workaround to explicitly use SSH is not functioning as mentioned in the original issue remark.

[ update ]

turns out that only providing the domain name actually makes a difference. so:

[url "ssh://git@github.ourcompany.local"]
insteadOf = https://github.ourcompany.local

(without trailing slash or /engineering)

This still shows the following error though:

$ go mod init modulename
go: creating new go.mod: module modulename
go: copying requirements from glide.lock
go: converting glide.lock: stat github.ourcompany.local/engineering/protos@7462cb9dbaba285e3c931bc1edc0b62ebe8d005e: git fetch -f https://github.ourcompany.local/engineering/protos.git refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in c:\git\go\pkg\mod\cache\vcs\5626dfafc13719b661e5e62c892d3563fa39030a15a5ca1fdcad3e06591ea342: **exit status 128:
        fatal: unable to access 'https://github.ourcompany.local/engineering/protos.git/': SSL certificate problem: unable to get local issuer certificate**

Even when providing this part in the gitconfig, to give a hint on the used private SSL certificate:

[http "https://github.ourcompany.local"]
sslCaInfo = /c/Users/me/.ssh/github-ourcompany.pem
schannelUseSSLCAInfo = true

what would be the proper settings to point to the private ssl certificate or gitconfig?

[bcmills]

accessible via git using private ssl certificates

Do you mean that the go command needs to trust SSL certificates from a private CA, or that the go command needs to present a client certificate to the server? (If the latter, is this the same issue as #30119?)

[bcmills]

Question: how come

go mod init

is trying to access the git repository using https instead of via ssl / git@etc..?

Presumably that's what the go-import tags served by https://github.ourcompany.local/engineering/protos?go-get=1 told it to use. (See also #30304.)

You should be able to verify that by running

curl https://github.ourcompany.local/engineering/protos?go-get=1

and checking for a <meta name="go-import" […]> tag.

[bcmills]

Did the go mod init veritasapi command actually fail, or did it emit some errors and write a go.mod file anyway? (It should have done the latter; if not, please file a separate issue and mention #26603 in it.)

[MelleKoning]

Did the go mod init veritasapi command actually fail, or did it emit some errors and write a go.mod file anyway? (It should have done the latter; if not, please file a separate issue and mention #26603 in it.)

It wrote a go.mod file anyway, because of other dependencies that it could find from public repositories like from public github.com.

on checking with curl, the https version returns:

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

This is fine, because of the gitconfig 'insteadOf' setting moves to the SSH git@ way of connecting.

Thus, using the SSH method returns no error:

curl git@github.ourcompany.local/engineering/protos?go-get=1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0

Still, using go mod init does not seem to be able to make the SSH connection. Not entirely sure how to setup the certificate configured in github.ourcompany.local within gitconfig, or if more certificate-access would be needed.

[bcmills]

on checking with curl, the https version returns:

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

This is fine, because of the gitconfig 'insteadOf' setting moves to the SSH git@ way of connecting.

No, that's not fine. There are two steps to fetching a module with a give path: first we resolve that path to a repo or module server path (via HTTPS), then we fetch the module from that server. If we can't resolve the path, then we have no way of knowing that we need to use git in the first place, let alone that we should use it with the ssh protocol.

That means that your local machine needs to be configured to recognize the issuer of the private certificate. On Windows, I believe that you can use the MMC Certificates snap-in or the Certmgr tool for that, but that configuration is independent of the Go toolchain itself.

If you can configure your certificates so that curl […]?go-get=1 works, then cmd/go should also work. Please get curl working and then let us know where you land.

[MelleKoning]

@bcmills how come

glide update

does work and retrieves the company.github.com code from remote while

go mod init myname

does not?
I did make sure I first executed a glide cc (to clean glide cache).
Is there a certain difference in retrieving code that exposes via that curl command? To clarify - I just do not know the underlying git commands the different go tools are using to get dependencies and what 'go mod init' would do differently than 'glide update', if anything.