all: test failures with `ETXTBSY`

[bcmills]

#!watchflakes
post <- `text file busy` && date < "2023-03-01"

This issue is to collect test failures with ETXTBSY. In practice, nearly all such failures are symptoms of #22315, but we may be able to work around them more specificially.

(Factored out from #55311 (comment).)

[gopherbot]

Found new dashboard test flakes for:

#!watchflakes
post <- `text file busy`

2023-01-17 19:55 linux-amd64-longtest-race go@f9cba430 cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:41143
https://vcs-test.golang.org rerouted to https://127.0.0.1:40539
go test proxy running at GOPROXY=http://127.0.0.1:42349/mod
2023/01/17 20:54:36 http: TLS handshake error from 127.0.0.1:48104: EOF
--- FAIL: TestScript (0.16s)
    --- FAIL: TestScript/list_goroot_symlink (0.17s)
        script_test.go:134: 2023-01-17T20:56:14Z
        script_test.go:136: $WORK=/workdir/tmp/cmd-go-test-3489566031/tmpdir3979779748/list_goroot_symlink2744314060
        script_test.go:154: 
            # Regression test for https://go.dev/issue/57754: 'go list' failed if ../src
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.160s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /workdir/tmp/cmd-go-test-3489566031/tmpdir3979779748/list_goroot_symlink2744314060/lib/goroot/bin/go env GOROOT: fork/exec /workdir/tmp/cmd-go-test-3489566031/tmpdir3979779748/list_goroot_symlink2744314060/lib/goroot/bin/go: text file busy

2023-01-17 19:55 linux-arm64-longtest go@248950f1 cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:40473
https://vcs-test.golang.org rerouted to https://127.0.0.1:45455
go test proxy running at GOPROXY=http://127.0.0.1:41419/mod
2023/01/17 20:19:51 http: TLS handshake error from 127.0.0.1:42222: read tcp 127.0.0.1:45455->127.0.0.1:42222: read: connection reset by peer
2023/01/17 20:19:51 http: TLS handshake error from 127.0.0.1:42206: read tcp 127.0.0.1:45455->127.0.0.1:42206: read: connection reset by peer
2023/01/17 20:19:51 http: TLS handshake error from 127.0.0.1:42212: EOF
2023/01/17 20:19:51 http: TLS handshake error from 127.0.0.1:42246: EOF
2023/01/17 20:19:51 http: TLS handshake error from 127.0.0.1:42262: EOF
2023/01/17 20:19:51 http: TLS handshake error from 127.0.0.1:42296: read tcp 127.0.0.1:45455->127.0.0.1:42296: read: connection reset by peer
2023/01/17 20:19:51 http: TLS handshake error from 127.0.0.1:42282: EOF
--- FAIL: TestScript (0.07s)
    --- FAIL: TestScript/list_goroot_symlink (0.04s)
        script_test.go:134: 2023-01-17T20:19:53Z
        script_test.go:136: $WORK=/tmp/workdir/tmp/cmd-go-test-949652263/tmpdir2422550999/list_goroot_symlink3453521521
        script_test.go:154: 
            # Regression test for https://go.dev/issue/57754: 'go list' failed if ../src
            # relative to the location of the go executable was a symlink to the real src
            # directory. (cmd/go expects that ../src is GOROOT/src, but it appears that the
            # Debian build of the Go toolchain is attempting to split GOROOT into binary and
            # source artifacts in different parent directories.) (0.000s)
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.036s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /tmp/workdir/tmp/cmd-go-test-949652263/tmpdir2422550999/list_goroot_symlink3453521521/lib/goroot/bin/go env GOROOT: fork/exec /tmp/workdir/tmp/cmd-go-test-949652263/tmpdir2422550999/list_goroot_symlink3453521521/lib/goroot/bin/go: text file busy

— watchflakes

[gopherbot]

Change https://go.dev/cl/463748 mentions this issue: cmd/go/internal/script: retry ETXTBSY errors in scripts

[gopherbot]

Found new dashboard test flakes for:

#!watchflakes
post <- `text file busy`

2023-01-25 18:32 linux-arm64-longtest go@a896219b cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:35487
https://vcs-test.golang.org rerouted to https://127.0.0.1:35111
go test proxy running at GOPROXY=http://127.0.0.1:42229/mod
2023/01/25 18:54:32 http: TLS handshake error from 127.0.0.1:34762: EOF
2023/01/25 18:54:40 http: TLS handshake error from 127.0.0.1:44554: read tcp 127.0.0.1:35111->127.0.0.1:44554: read: connection reset by peer
2023/01/25 18:54:40 http: TLS handshake error from 127.0.0.1:44572: EOF
2023/01/25 18:54:41 http: TLS handshake error from 127.0.0.1:44640: read tcp 127.0.0.1:35111->127.0.0.1:44640: read: connection reset by peer
--- FAIL: TestScript (0.06s)
    --- FAIL: TestScript/list_goroot_symlink (0.03s)
        script_test.go:134: 2023-01-25T18:54:43Z
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.030s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /tmp/workdir/tmp/cmd-go-test-77560195/tmpdir1748134965/list_goroot_symlink3339312632/lib/goroot/bin/go env GOROOT: fork/exec /tmp/workdir/tmp/cmd-go-test-77560195/tmpdir1748134965/list_goroot_symlink3339312632/lib/goroot/bin/go: text file busy

2023-01-25 22:05 linux-amd64-longtest go@2c785608 cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:32847
https://vcs-test.golang.org rerouted to https://127.0.0.1:44713
go test proxy running at GOPROXY=http://127.0.0.1:36469/mod
2023/01/25 22:34:23 http: TLS handshake error from 127.0.0.1:46980: read tcp 127.0.0.1:44713->127.0.0.1:46980: read: connection reset by peer
2023/01/25 22:34:23 http: TLS handshake error from 127.0.0.1:47024: EOF
2023/01/25 22:34:24 http: TLS handshake error from 127.0.0.1:47082: read tcp 127.0.0.1:44713->127.0.0.1:47082: read: connection reset by peer
2023/01/25 22:34:24 http: TLS handshake error from 127.0.0.1:47048: read tcp 127.0.0.1:44713->127.0.0.1:47048: read: connection reset by peer
--- FAIL: TestScript (0.07s)
    --- FAIL: TestScript/list_goroot_symlink (0.05s)
        script_test.go:134: 2023-01-25T22:34:29Z
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.044s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /workdir/tmp/cmd-go-test-3337829568/tmpdir2180002773/list_goroot_symlink4269922523/lib/goroot/bin/go env GOROOT: fork/exec /workdir/tmp/cmd-go-test-3337829568/tmpdir2180002773/list_goroot_symlink4269922523/lib/goroot/bin/go: text file busy

2023-01-27 17:13 linux-386-longtest go@c8c646d3 cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:37545
https://vcs-test.golang.org rerouted to https://127.0.0.1:38205
go test proxy running at GOPROXY=http://127.0.0.1:42975/mod
2023/01/27 18:26:40 http: TLS handshake error from 127.0.0.1:58518: EOF
2023/01/27 18:26:58 http: TLS handshake error from 127.0.0.1:40112: EOF
2023/01/27 18:26:58 http: TLS handshake error from 127.0.0.1:40088: EOF
2023/01/27 18:26:58 http: TLS handshake error from 127.0.0.1:40102: EOF
2023/01/27 18:26:58 http: TLS handshake error from 127.0.0.1:40144: EOF
2023/01/27 18:26:59 http: TLS handshake error from 127.0.0.1:40172: read tcp 127.0.0.1:38205->127.0.0.1:40172: read: connection reset by peer
2023/01/27 18:26:59 http: TLS handshake error from 127.0.0.1:40196: EOF
...
--- FAIL: TestScript (0.07s)
    --- FAIL: TestScript/list_goroot_symlink (0.04s)
        script_test.go:134: 2023-01-27T18:27:04Z
        script_test.go:136: $WORK=/workdir/tmp/cmd-go-test-1355527519/tmpdir2052141148/list_goroot_symlink3211899521
        script_test.go:154: 
            # Regression test for https://go.dev/issue/57754: 'go list' failed if ../src
            # relative to the location of the go executable was a symlink to the real src
            # directory. (cmd/go expects that ../src is GOROOT/src, but it appears that the
            # Debian build of the Go toolchain is attempting to split GOROOT into binary and
            # source artifacts in different parent directories.) (0.000s)
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.039s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /workdir/tmp/cmd-go-test-1355527519/tmpdir2052141148/list_goroot_symlink3211899521/lib/goroot/bin/go env GOROOT: fork/exec /workdir/tmp/cmd-go-test-1355527519/tmpdir2052141148/list_goroot_symlink3211899521/lib/goroot/bin/go: text file busy

2023-01-30 18:10 linux-386-longtest go@dbfdc446 cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:39301
https://vcs-test.golang.org rerouted to https://127.0.0.1:46019
go test proxy running at GOPROXY=http://127.0.0.1:39435/mod
2023/01/30 18:46:25 http: TLS handshake error from 127.0.0.1:43336: EOF
2023/01/30 18:46:54 http: TLS handshake error from 127.0.0.1:39486: EOF
2023/01/30 18:46:54 http: TLS handshake error from 127.0.0.1:39484: EOF
2023/01/30 18:46:54 http: TLS handshake error from 127.0.0.1:39498: EOF
2023/01/30 18:46:54 http: TLS handshake error from 127.0.0.1:39518: EOF
2023/01/30 18:46:54 http: TLS handshake error from 127.0.0.1:39546: EOF
2023/01/30 18:46:54 http: TLS handshake error from 127.0.0.1:39584: EOF
...
--- FAIL: TestScript (0.07s)
    --- FAIL: TestScript/list_goroot_symlink (0.03s)
        script_test.go:134: 2023-01-30T18:46:57Z
        script_test.go:136: $WORK=/workdir/tmp/cmd-go-test-3178230797/tmpdir729658518/list_goroot_symlink2424326606
        script_test.go:154: 
            # Regression test for https://go.dev/issue/57754: 'go list' failed if ../src
            # relative to the location of the go executable was a symlink to the real src
            # directory. (cmd/go expects that ../src is GOROOT/src, but it appears that the
            # Debian build of the Go toolchain is attempting to split GOROOT into binary and
            # source artifacts in different parent directories.) (0.000s)
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.027s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /workdir/tmp/cmd-go-test-3178230797/tmpdir729658518/list_goroot_symlink2424326606/lib/goroot/bin/go env GOROOT: fork/exec /workdir/tmp/cmd-go-test-3178230797/tmpdir729658518/list_goroot_symlink2424326606/lib/goroot/bin/go: text file busy

2023-01-31 13:55 linux-amd64-longtest go@da0c375c cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:42393
https://vcs-test.golang.org rerouted to https://127.0.0.1:33297
go test proxy running at GOPROXY=http://127.0.0.1:35529/mod
2023/01/31 14:27:14 http: TLS handshake error from 127.0.0.1:34874: read tcp 127.0.0.1:33297->127.0.0.1:34874: read: connection reset by peer
2023/01/31 14:28:40 http: TLS handshake error from 127.0.0.1:57276: EOF
2023/01/31 14:28:40 http: TLS handshake error from 127.0.0.1:57334: EOF
--- FAIL: TestScript (0.07s)
    --- FAIL: TestScript/list_goroot_symlink (0.06s)
        script_test.go:134: 2023-01-31T14:29:34Z
        script_test.go:136: $WORK=/workdir/tmp/cmd-go-test-2681287106/tmpdir532174821/list_goroot_symlink1053752192
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.060s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /workdir/tmp/cmd-go-test-2681287106/tmpdir532174821/list_goroot_symlink1053752192/lib/goroot/bin/go env GOROOT: fork/exec /workdir/tmp/cmd-go-test-2681287106/tmpdir532174821/list_goroot_symlink1053752192/lib/goroot/bin/go: text file busy

2023-01-31 16:53 linux-arm64-longtest go@5c6d7322 cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:35041
https://vcs-test.golang.org rerouted to https://127.0.0.1:34211
go test proxy running at GOPROXY=http://127.0.0.1:39067/mod
2023/01/31 17:12:25 http: TLS handshake error from 127.0.0.1:54124: read tcp 127.0.0.1:34211->127.0.0.1:54124: read: connection reset by peer
2023/01/31 17:12:25 http: TLS handshake error from 127.0.0.1:54126: EOF
2023/01/31 17:12:25 http: TLS handshake error from 127.0.0.1:54122: EOF
2023/01/31 17:12:25 http: TLS handshake error from 127.0.0.1:54182: read tcp 127.0.0.1:34211->127.0.0.1:54182: read: connection reset by peer
2023/01/31 17:12:25 http: TLS handshake error from 127.0.0.1:54186: read tcp 127.0.0.1:34211->127.0.0.1:54186: read: connection reset by peer
2023/01/31 17:12:25 http: TLS handshake error from 127.0.0.1:54200: EOF
--- FAIL: TestScript (0.06s)
    --- FAIL: TestScript/list_goroot_symlink (0.02s)
        script_test.go:134: 2023-01-31T17:12:35Z
        script_test.go:136: $WORK=/tmp/workdir/tmp/cmd-go-test-2918864871/tmpdir3244305613/list_goroot_symlink2495268199
        script_test.go:154: 
            # Regression test for https://go.dev/issue/57754: 'go list' failed if ../src
            # relative to the location of the go executable was a symlink to the real src
            # directory. (cmd/go expects that ../src is GOROOT/src, but it appears that the
            # Debian build of the Go toolchain is attempting to split GOROOT into binary and
            # source artifacts in different parent directories.) (0.000s)
            # Ensure that the relative path to $WORK/lib/goroot/src from $PWD is a different
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.022s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /tmp/workdir/tmp/cmd-go-test-2918864871/tmpdir3244305613/list_goroot_symlink2495268199/lib/goroot/bin/go env GOROOT: fork/exec /tmp/workdir/tmp/cmd-go-test-2918864871/tmpdir3244305613/list_goroot_symlink2495268199/lib/goroot/bin/go: text file busy

2023-01-31 20:36 linux-arm64-longtest go@ac7efcb0 cmd/go.TestScript (log)

vcs-test.golang.org rerouted to http://127.0.0.1:37759
https://vcs-test.golang.org rerouted to https://127.0.0.1:39019
go test proxy running at GOPROXY=http://127.0.0.1:36163/mod
2023/01/31 20:56:35 http: TLS handshake error from 127.0.0.1:37416: EOF
2023/01/31 20:56:35 http: TLS handshake error from 127.0.0.1:37474: EOF
2023/01/31 20:56:35 http: TLS handshake error from 127.0.0.1:37512: EOF
2023/01/31 20:56:35 http: TLS handshake error from 127.0.0.1:37492: EOF
2023/01/31 20:56:40 http: TLS handshake error from 127.0.0.1:55104: read tcp 127.0.0.1:39019->127.0.0.1:55104: read: connection reset by peer
--- FAIL: TestScript (0.06s)
    --- FAIL: TestScript/list_goroot_symlink (0.03s)
...
            #
            # To do so, we copy the actual cmd/go executable — which is implemented as the
            # cmd/go test binary instead of the original $GOROOT/bin/go, which may be
            # arbitrarily stale — into the bin subdirectory of the fake GOROOT, causing
            # os.Executable to report a path in that directory. (0.029s)
            > mkdir $WORK/lib/goroot/bin
            > cp $TESTGO_EXE $WORK/lib/goroot/bin/go$GOEXE
            > env GOROOT=''  # Clear to force cmd/go to find GOROOT itself.
            > exec $WORK/lib/goroot/bin/go env GOROOT
        script_test.go:154: FAIL: list_goroot_symlink.txt:39: exec /tmp/workdir/tmp/cmd-go-test-3552578516/tmpdir1016144016/list_goroot_symlink3265508911/lib/goroot/bin/go env GOROOT: fork/exec /tmp/workdir/tmp/cmd-go-test-3552578516/tmpdir1016144016/list_goroot_symlink3265508911/lib/goroot/bin/go: text file busy

— watchflakes

[bcmills]

@gopherbot, please backport to Go 1.20. The list_goroot_symlink test sometimes fails with this symptom on the release branch, and the fix is test-only.

[gopherbot]

Backport issue(s) opened: #58431 (for 1.20).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[gopherbot]

Change https://go.dev/cl/466856 mentions this issue: [release-branch.go1.20] cmd/go/internal/script: retry ETXTBSY errors in scripts