cmd/link: error if pass a very long args which contains \n character

[lyyyuna]

What version of Go are you using (go version)?

$ go version
go version go1.15.3 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

darwin

What did you do?

I build with the following command,

go build  -v -ldflags "-X 'qiniu.com/kodotest.regionNb=\'a very long string.....which contains \n characters.....\''"

get go link error.

Then I change the above command with a shorter one,

go build  -v -ldflags "-X 'qiniu.com/kodotest.regionNb=\'a short string.....which contains \n characters.....\''"

no go link error this time.

What did you expect to see?

No go link error if I pass a very long args with \n character inside.

My analysis

go1.14 and go1.13 have no such issue.

I notice that go has introduced response file to pass long args in Windows: https://go-review.googlesource.com/c/go/+/110395/7/src/cmd/go/internal/work/exec.go#2483

and enabled it for other OS in go1.15:

go/src/cmd/go/internal/work/exec.go

Line 3131 in f7e2646

func useResponseFile(path string, argLen int) bool {

response file use \n to separate args will cause this issue.

[cherrymui]

get go link error.

What error?

Could you share the exact command you run that succeeded with Go 1.14 but failed with Go 1.15? Thanks.

[lyyyuna]

@cherrymui
just show link's usage message, and the return exit code is 2.

In our case, we need to put a INI config file's content(which contains a lot of '\n') in the -ldflags.
The INI config content is very long which exceeds 32KB, so the -ldflags is written into a response file instead to pass to cmd/link as args.

It's easy to reproduce it with long -ldflags, I think go should not simply use '\n' as a separator in the response file.

[cherrymui]

Can you paste in exact commands you run, and the exact output? Possibly to use go build -x to give extra information.

The command you mentioned above,

go build  -v -ldflags "-X 'qiniu.com/kodotest.regionNb=\'a short string.....which contains \n characters.....\''"

short or long, when I paste it to a shell, never works on my machine with any version of Go.

[lyyyuna]

Here are two example config files (config-wrong.ini, config.ini):
https://gist.github.com/lyyyuna/ca5e6c7720260180080689373f5e5d62

config-wrong.ini is longer than config.ini with only one extra line: a=a
machine: darwin

cmd/link error with go1.15 / config-wrong.ini

go build  -ldflags="-X 'qiniu.com/qtest.sample=`cat config-wrong.ini`'" .

get

# xxxxxxxxx
usage: link [options] main.o
  -B note
        add an ELF NT_GNU_BUILD_ID note when using ELF
  -E entry
        set entry symbol name

build success with go1.15 / config.ini

go build -ldflags="-X 'qiniu.com/qtest.sample=`cat config.ini`'" .

I am not sure if link arg is same on all darwin's machines, so you can add extra lines in config-wrong.ini to reproduce it.

[jeremyfaller]

On darwin, in bash, I was able to reproduce this with:

uname -a Darwin jfaller-macbookpro2.roam.corp.google.com 19.5.0 Darwin Kernel Version 19.5.0: Thu Apr 30 18:25:59 PDT 2020; root:xnu-6153.121.1~7/RELEASE_X86_64 x86_6

package main

import "fmt"

var str string

func main() {
        fmt.Println(str)
}

The following fails:

go build -ldflags="-X 'main.str=`openssl rand -base64 22342`'" ./

But, this succeeds:

go build -ldflags="-X 'main.str=`openssl rand -base64 22341`'" ./

And for the people following along at home: getconf ARG_MAX = 262144.

[gopherbot]

Change https://golang.org/cl/272870 mentions this issue: cmd/go: encode backslash and newline in response files

[dmitshur]

@jeremyfaller I understand this issue was meant to be moved to NeedsFix state, is that right? When an issue doesn't have any of NeedsInvestigation, NeedsDecision, or NeedsFix labels, it appears as one that hasn't been triaged.