cmd/compile: internal compiler error: Type.Elem UNSAFEPTR

[kirillDanshin]

What version of Go are you using (go version)?

$ go version
go version devel +4091cf972a Sun Mar 31 23:35:35 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

reproducible only on tip

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/travis/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/travis/gopath"
GOPROXY=""
GORACE=""
GOROOT="/home/travis/.gimme/versions/go"
GOTMPDIR=""
GOTOOLDIR="/home/travis/.gimme/versions/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build848806544=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have a pretty old package with some hacks for optimization.
I use some of its features in my web framework.
In travis, I check if the framework pass the tests on tip.
One of the steps in travis script is the go get -v.
When go get tries to precompile that old package, it crashes.

So, one of the files looks like this:

package runtimer

import (
	"unsafe" // #nosec
)

func PtrToString(ptr unsafe.Pointer) string {
	return *(*string)(ptr)
}

func PtrToStringPtr(ptr unsafe.Pointer) *string {
	return (*string)(ptr)
}

func PtrPtrToStringPtr(ptr *unsafe.Pointer) *string {
	return (*string)(*ptr) // <- compiler complains about this line
}

Then I've commented out the last func.

Then the file looks like this:

package runtimer

import (
	"unsafe" // #nosec
)

func PtrToString(ptr unsafe.Pointer) string {
	return *(*string)(ptr)
}

func PtrToStringPtr(ptr unsafe.Pointer) *string {
	return (*string)(ptr) // <- and then it complains about this line
}
/*
func PtrPtrToStringPtr(ptr *unsafe.Pointer) *string {
	return (*string)(*ptr)
}
*/

What did you expect to see?

unsafe.Pointer works as it work two years ago

What did you see instead?

# github.com/gramework/runtimer
../runtimer/utils.go:16:2: internal compiler error: Type.Elem UNSAFEPTR
goroutine 34 [running]:
runtime/debug.Stack(0x1039ae0, 0xc00000e018, 0x0)
	/home/travis/.gimme/versions/go/src/runtime/debug/stack.go:24 +0x9d
cmd/compile/internal/gc.Fatalf(0xe8ef7d, 0xc, 0xc0007c7550, 0x1, 0x1)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/gc/subr.go:190 +0x292
cmd/compile/internal/types.(*Type).Elem(0xc000061e60, 0xc000538460)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/types/type.go:801 +0xff
cmd/compile/internal/ssa.(*Func).computeZeroMap(0xc000804000, 0xe12f01)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/ssa/writebarrier.go:391 +0x101
cmd/compile/internal/ssa.writebarrier(0xc000804000)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/ssa/writebarrier.go:80 +0x6b
cmd/compile/internal/ssa.Compile(0xc000804000)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/ssa/compile.go:90 +0x476
cmd/compile/internal/gc.buildssa(0xc0003c9b80, 0x0, 0x0)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/gc/ssa.go:288 +0xbf3
cmd/compile/internal/gc.compileSSA(0xc0003c9b80, 0x0)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/gc/pgen.go:297 +0x4d
cmd/compile/internal/gc.compileFunctions.func2(0xc000404de0, 0xc000407200, 0x0)
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/gc/pgen.go:362 +0x49
created by cmd/compile/internal/gc.compileFunctions
	/home/travis/.gimme/versions/go/src/cmd/compile/internal/gc/pgen.go:360 +0x128

[cuonglm]

git bisect points to ca36af2

[cuonglm]

@randall77 A quick fix seems to check if v Type is unsafe pointer:

diff --git a/src/cmd/compile/internal/ssa/writebarrier.go b/src/cmd/compile/internal/ssa/writebarrier.go
index 3c64da20a7..7eedd96885 100644
--- a/src/cmd/compile/internal/ssa/writebarrier.go
+++ b/src/cmd/compile/internal/ssa/writebarrier.go
@@ -526,6 +526,9 @@ func IsReadOnlyGlobalAddr(v *Value) bool {
 
 // IsNewObject reports whether v is a pointer to a freshly allocated & zeroed object at memory state mem.
 func IsNewObject(v *Value, mem *Value) bool {
+       if v.Type.IsUnsafePtr() {
+               return false
+       }
        if v.Op != OpLoad {
                return false
        }

If you're ok with it, I will send a CL.

[cherrymui]

@Gnouc this looks ok to me. Thanks!

[gopherbot]

Change https://golang.org/cl/170159 mentions this issue: cmd/compile: fix write barrier removal with unsafe pointer

[randall77]

I'm confused by this as well. I can believe the crash, but the provided code doesn't trigger that crash. We need a standalone reproducer (or any reproducer, really, from which we could extract a standalone one).

[kirillDanshin]

@randall77 https://travis-ci.org/gramework/gramework/jobs/513914890

[cuonglm]

@randall77 https://go-review.googlesource.com/c/go/+/170159/3/test/fixedbugs/issue31174.go

@cherrymui found it, the error message is a bit confused.

[randall77]

Thanks, that example crashes for me.

But that example uses linkname to access runtime internals. I'm not inclined to fix the Go toolchain to permit this use.

[cuonglm]

@randall77

But that example uses linkname to access runtime internals. I'm not inclined to fix the Go toolchain to permit this use.

Yes.

But I think it's worth to make the change, even not for fixing this issue. IsNewObject indicates it reports whether Value is a pointer to something, but without checking its Type is a pointer first.

If not, then the function doc should be updated.

[randall77]

That's because the compiler has control over all calls to runtime.newobject (except for linkname shenanigans, of course), and ensures that the type of the result of that call is always correct - a pointer to a base type.

[cherrymui]

Yes, if it only crashes with linkname into the runtime, I'm ok with not fixing it.

[kirillDanshin]

Honestly, I don't think that changing the compiler behavior in such a way that folks can't use basically one of the unsafe features is a good idea for a release. Previously, I reported #31121, a bug that silently broke about a thousand of my services by changing Map behavior, which changed ToLower behavior. This bug broke a package, which I need to do #17043, which as stated in #17043 (comment) will not be supported in Go (altho it is supported as a sync.Map, but sync.Map doesn't fit my needs due to its performance and algorithm). For me it's weird to see that go1.12 just changed really common function's behavior and 1.13 is going to break several technics that was allowed before, and I'm trying to understand where it's going when the current decision process kinda conflicts with ideas from @ianlancetaylor talk (watch it, if you haven't yet, it's really awesome talk).
I understand when some feature will never be available out-of-box, but I don't see any reasons to not add a three line fix to keep old compiler behavior, that was broken by another change. At least, if for some reason this issue will not be fixed, please, document that breaking change, and if possible, suggest a workaround so developers can migrate to the new Go version.