cmd/link: reproducible builds don't work on macOS

[mandrigin]

What version of Go are you using (go version)?

$ go version go1.15 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/mandrigin/Library/Caches/go-build"
GOENV="/Users/mandrigin/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/mandrigin/golang/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/mandrigin/golang"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.15/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.15/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/mandrigin/Desktop/repro-builds-issue/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/y5/rmr800b53s53zt42d2tfmpdc0000gn/T/go-build570372631=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I've created a repository with a small app where I can reproduce this issue easily. It uses one module with native files (go-duktape): https://github.com/mandrigin/golang-reproducible-builds-issue-osx/

The app builds the same source twice into 2 different files
go build -asmflags -trimpath -ldflags=-buildid= -o t1 .
go build -asmflags -trimpath -ldflags=-buildid= -o t2 .

Then it takes sha sums of both files and prints them. Also it prints a diff between these files.

What did you expect to see?

Reproducible builds on both Alpine (in Docker) and macOS.

What did you see instead?

Reproducible builds on Alpine work. (hashes match, no diff between binary files).

Reproducible builds on macOS are broken.
Diff on macOS contains entries like this one

< 005b1850: 6e2f 542f 676f 2d6c 696e 6b2d 3732 3938  n/T/go-link-7298
< 005b1860: 3738 3633 362f 3030 3030 3237 2e6f 005f  78636/000027.o._
---
> 005b1850: 6e2f 542f 676f 2d6c 696e 6b2d 3532 3038  n/T/go-link-5208
> 005b1860: 3330 3433 312f 3030 3030 3237 2e6f 005f  30431/000027.o._

Clang Version

Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[ianlancetaylor]

CC @thanm @neild

[cherrymui]

It looks like temporary file names are built into the binary. I'll take a look.

[cherrymui]

Yeah, it seems the case for any cgo program on darwin, since at least Go 1.11 (Go 1.10 doesn't work on my machine).

[cherrymui]

I think it is the darwin linker adds them. I don't know how to make it not to...

[thanm]

Possibly DWARF related? What happens when "-w" is added to linker options?

[cherrymui]

-w doesn't help.

If I understand the otool dump correctly, the file paths are in __LINKEDIT segment.

[ianlancetaylor]

Are the paths coming from the command line? Would it help to chdir to the directory, and just pass plain .o files to the linker?

[cherrymui]

Good point. The command line does include the paths. But if I cd into the temp dir and run the command with only the file name without the directory, the binary still contains the paths...

[ianlancetaylor]

If you chdir, does it record the paths for each object, or is it recording the current directory somewhere?

[cherrymui]

The paths for each object.

[ianlancetaylor]

Well that's quite annoying.

[gopherbot]

Change https://golang.org/cl/250944 mentions this issue: cmd/link: strip STAB (symbolic debugging) symbols on darwin

[gopherbot]

Change https://golang.org/cl/253157 mentions this issue: cmd/link: add a test for reproducible build