memory corruption while using appending to a slice across multiple goroutines

[bionoren]

What version of Go are you using (go version)?

$ go version
1.13.8 and 1.14.0

Does this issue reproduce with the latest release?

Yes - 1.14.0

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/bion/Library/Caches/go-build"
GOENV="/Users/bion/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/bion/modo/coin"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mb/50wl13bn0t386wx67ns686p80000gn/T/go-build304849267=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

https://play.golang.org/p/Jwo8Ub1mU8L

I think I've simplified this as far as I can. It's tricky to get it to crash at this point (but it does occasionally happen!), but it reliably panics in ways I don't expect. This was originally a benchmark, which is reproduced here: https://gist.github.com/bionoren/c8e45a802dbe0c0be6f1924b56a375f7 and I ran it with

go test -run=xxx -bench=BenchmarkCollection_Transition -benchmem -count=10 .

What did you expect to see?

Benchmark passes, or at least panics in my code somewhere.

What did you see instead?

Sometimes it passes, sometimes it panics - usually due to null pointer exceptions, but I've seen several reasons and stack traces, often down in the json or ioutil packages. On rare occasion, it encounters a fatal error. I've seen a variety of these while trying to trim my example down:

runtime: nameOff 0x400 base 0x11cbd70 not in ranges:
	types 0x17a6520 etypes 0x1bcc82e
fatal error: runtime: name offset base pointer out of range

runtime: pointer 0xc00379ac60 to unused region of span span.base()=0xc001794000 span.limit=0xc001795e00 span.state=1
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)

runtime: nelems=8 nalloc=5 previous allocCount=4 nfreed=65535
fatal error: sweep increased allocation count

fatal error: unexpected signal during runtime execution
[signal SIGBUS: bus error code=0x2 addr=0x16dc358 pc=0x100c850]

panic: runtime error: invalid memory address or nil pointer dereference
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x0]

I realize this fails

go vet

, and I realize that's causing me to use append() on a slice across multiple go-routines, but I expected that to result in a slice of unusual contents, not what appears to be stack corruption resulting in a crash.

[bcmills]

Have you run the program under the race detector? The Go memory model does not allow concurrent goroutines to append to the same slice.

[bcmills]

(https://research.swtch.com/gorace is a good read if you haven't seen it already.)

[bionoren]

I had not seen it. The go memory model article was particularly interesting.

Thank you! I knew concurrent appends to the same slice was a bad idea, but I hadn't realized it was officially not allowed.