cmd/go: GOPROXY should not override GONOPROXY/GOPRIVATE

[FiloSottile]

What version of Go are you using (go version)?

$ go version
go version go1.19.2 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/filippo/Library/Caches/go-build"
GOENV="/Users/filippo/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/filippo/pkg/mod"
GONOPROXY="github.com/FiloSottile/*,filippo.io/*"
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/filippo"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org"
GOROOT="/opt/homebrew/Cellar/go/1.19.2/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.19.2/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19.2"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/filippo/src/filippo.io/mostly-harmless/filippo.fly.dev/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/var/folders/_j/hq4ytn1n4b94fhrpvvb9tktr0000gn/T/go-build3821845917=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ GOPRIVATE="c2sp.org" go mod download -json -x c2sp.org/CCTV/age@main

What did you expect to see?

A direct connection to c2sp.org.

What did you see instead?

# get https://proxy.golang.org/c2sp.org/%21c%21c%21t%21v/age/@v/main.info
# get https://proxy.golang.org/c2sp.org/%21c%21c%21t%21v/age/@v/main.info: 404 Not Found (5.486s)
{
	"Path": "c2sp.org/CCTV/age",
	"Version": "main",
	"Error": "c2sp.org/CCTV/age@main: invalid version: reading https://proxy.golang.org/c2sp.org/%21c%21c%21t%21v/age/@v/main.info: 404 Not Found\n\tserver response: not found: c2sp.org/CCTV/age@main: unrecognized import path \"c2sp.org/CCTV/age\": https://c2sp.org/CCTV/age?go-get=1 and //c2sp.org/CCTV?go-get=1 disagree about go-import for c2sp.org/CCTV"
}

Ignore the error, the point is that with GOPRIVATE set this request should not have reached the proxy.

[heschi]

cc @bcmills @matloob

[seankhliao]

has your GOPROXY been explicitly set? it's non default and I believe that overrides GOPRIVATE (to allow you to use private proxies).

[heschi]

Oh, good catch. Though I think you meant GONOPROXY.

[FiloSottile]

Oh yeah, good catch. I do that to mitigate VCS vulnerabilities. However, I would expect using GONOPROXY without direct in GOPROXY to be an error at most, not to silently ignore GONOPROXY.

https://go.dev/ref/mod#private-module-privacy explicitly says (emphasis mine)

The GOPRIVATE or GONOPROXY environment variables may be set to lists of glob patterns matching module prefixes that are private and should not be requested from any proxy.

and then it goes on to explain not to set GONOPROXY if you're using a private proxy.

[seankhliao]

I think heschi is correct in saying it's your GONOPROXY setting that's overriding it.

https://go.dev/ref/mod#private-modules :

GOPRIVATE — list of glob patterns of module path prefixes that should be considered private. Acts as a default value for GONOPROXY and GONOSUMDB.

with an explicit GONOPROXY="github.com/FiloSottile/*,filippo.io/*", the default value from GOPRIVATE isn't used.

[bcmills]

@FiloSottile

https://go.dev/ref/mod#private-module-privacy explicitly says (emphasis mine)

You left out the very next sentence, which is key to the behaviors you observe here:

GOPRIVATE simply acts as a default for GONOPROXY and GONOSUMDB, so it’s not necessary to set GONOPROXY unless GONOSUMDB should have a different value.

That also matches https://go.dev/ref/mod#environment-variables:

Variable	Description
GOPRIVATE	Comma-separated list of glob patterns (in the syntax of Go's path.Match) of module path prefixes that should be considered private. GOPRIVATE is a default value for GONOPROXY and GONOSUMDB. See Privacy. GOPRIVATE also determines whether a module is considered private for GOVCS.

---

Because you have set GONOPROXY explicitly in the environment, that overrides the default set by GOPRIVATE.
Because c2sp.org does not match a pattern in GONOPROXY, it is fetched using your GOPROXY setting.

(The GOPRIVATE setting still affects GONOSUMDB and the interpretation of private: in GOVCS.)

[bcmills]

As far as I can tell this is working as documented. It's unfortunate that the environment variables are this complex, but on the other hand I don't think most users are setting GONOPROXY at all.

@FiloSottile, do you want to file a proposal for a behavior change here? Is there something we could do to clarify the documentation? (I'd be happy to review a CL to clean up the docs.)