Skip to content

cmd/compile: internal compiler error instantiating generic type #48185

New issue
New issue
Closed
Closed
cmd/compile: internal compiler error instantiating generic type#48185
Labels
FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.
Milestone
 
Go1.18
@dsnet

Description

@dsnet
dsnet
opened on Sep 3, 2021

I apologize for all the bug reports, but I'm hitting another internal compiler error.

$ go.tip version
go version devel go1.18-d9244f8b64 Fri Sep 3 21:31:30 2021 +0000 linux/amd64

$ go.tip test
# github.com/go-json-experiment/json_test [github.com/go-json-experiment/json.test]
<autogenerated>:1: internal compiler error: Dictionary should have already been generated: github.com/go-json-experiment/json..dict.(*arshalers[json.UnmarshalOptions,json.Decoder]).lookup

goroutine 1 [running]:
runtime/debug.Stack()
	/usr/local/go.tip/src/runtime/debug/stack.go:24 +0x65
cmd/compile/internal/base.FatalfAt({0x9df580, 0xc0}, {0xd120ee, 0x34}, {0xc000589320, 0x2, 0x2})
	/usr/local/go.tip/src/cmd/compile/internal/base/print.go:227 +0x154
cmd/compile/internal/base.Fatalf(...)
	/usr/local/go.tip/src/cmd/compile/internal/base/print.go:196
cmd/compile/internal/reflectdata.getDictionary(0xc000732eb0, {0xc0006b1c00, 0x2, 0x2})
	/usr/local/go.tip/src/cmd/compile/internal/reflectdata/reflect.go:2043 +0x245
cmd/compile/internal/reflectdata.methodWrapper(0xc000730070, 0xc000732f00, 0xf5)
	/usr/local/go.tip/src/cmd/compile/internal/reflectdata/reflect.go:1909 +0x10ea
cmd/compile/internal/reflectdata.methods(0xc000730070)
	/usr/local/go.tip/src/cmd/compile/internal/reflectdata/reflect.go:347 +0x39f
cmd/compile/internal/reflectdata.uncommonSize(...)
	/usr/local/go.tip/src/cmd/compile/internal/reflectdata/reflect.go:79
cmd/compile/internal/reflectdata.dcommontype(0xc0006dbf80, 0xc000730070)
	/usr/local/go.tip/src/cmd/compile/internal/reflectdata/reflect.go:716 +0x1cc
cmd/compile/internal/reflectdata.writeType(0xc0006b29a0)
	/usr/local/go.tip/src/cmd/compile/internal/reflectdata/reflect.go:1140 +0x327
cmd/compile/internal/reflectdata.WriteRuntimeTypes()
	/usr/local/go.tip/src/cmd/compile/internal/reflectdata/reflect.go:1254 +0x3d8
cmd/compile/internal/gc.dumpdata()
	/usr/local/go.tip/src/cmd/compile/internal/gc/obj.go:118 +0x6e
cmd/compile/internal/gc.Main(0xd1dcb0)
	/usr/local/go.tip/src/cmd/compile/internal/gc/main.go:327 +0x10de
main.main()
	/usr/local/go.tip/src/cmd/compile/main.go:55 +0xdd

FAIL	github.com/go-json-experiment/json [build failed]

Reproduction:

$ git clone https://github.com/go-json-experiment/json.git
$ cd json
$ git checkout origin/generics-bug4
$ go test

Based on @mdempsky's comment in #48016 (comment), I tried building this with GOEXPERIMENT=unified and that also panics, but due to a different problem:

$ GOEXPERIMENT=unified go.tip test
# github.com/go-json-experiment/json_test [github.com/go-json-experiment/json.test]
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_SemacquireMutex(0xc0000cf068, 0x98, 0xa2)
	/usr/local/go.tip/src/runtime/sema.go:71 +0x25
sync.(*Mutex).lockSlow(0xc0004ade4c)
	/usr/local/go.tip/src/sync/mutex.go:138 +0x165
sync.(*Mutex).Lock(...)
	/usr/local/go.tip/src/sync/mutex.go:81
sync.(*Once).doSlow(0xb70e27, 0xc0000cf0e0)
	/usr/local/go.tip/src/sync/once.go:64 +0x53
sync.(*Once).Do(...)
	/usr/local/go.tip/src/sync/once.go:59
cmd/compile/internal/types2.(*Named).load(0xc0004addd0)
	/usr/local/go.tip/src/cmd/compile/internal/types2/named.go:56 +0xdb
cmd/compile/internal/types2.(*Named).TParams(...)
	/usr/local/go.tip/src/cmd/compile/internal/types2/named.go:124
cmd/compile/internal/types2.(*typeWriter).typ(0xc0000cf398, {0xe50750, 0xc0004addd0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/typestring.go:237 +0x3bb
cmd/compile/internal/types2.(*typeWriter).typeList(0xc0000cf398, {0xc0003daec0, 0x2, 0x138b1e0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/typestring.go:272 +0xd2
cmd/compile/internal/types2.typeHash({0xe50750, 0xc0004ade60}, {0xc0003daec0, 0x10, 0x2})
	/usr/local/go.tip/src/cmd/compile/internal/types2/subst.go:269 +0xef
cmd/compile/internal/types2.(*Checker).instance(0xc0000f4640, {0x0, 0x0, 0x0}, {0xe50750, 0xc0004ade60}, {0xc0003daec0, 0xc0005a9e90, 0xc0005d7bc0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/instantiate.go:128 +0x31f
cmd/compile/internal/types2.Instantiate(0xc0005d7bc0, {0xe50750, 0xc0004ade60}, {0xc0003daec0, 0x2, 0x2}, 0x0)
	/usr/local/go.tip/src/cmd/compile/internal/types2/instantiate.go:53 +0xb1
cmd/compile/internal/noder.(*reader2).doTyp(0xc0005d7bc0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:231 +0x150
cmd/compile/internal/noder.(*pkgReader2).typIdx(0xc0000e0900, {0xc0005d7b60, 0xb}, 0xc0005cc4e0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:206 +0x175
cmd/compile/internal/noder.(*reader2).typ(0xc0005d7b60)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:178 +0x38
cmd/compile/internal/noder.(*reader2).doTyp(0xc0005d7b60)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:248 +0x7d
cmd/compile/internal/noder.(*pkgReader2).typIdx(0xc0000e0900, {0xc0005d7680, 0x9}, 0xc0005cc4e0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:206 +0x175
cmd/compile/internal/noder.(*reader2).typ(0xc0005d7680)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:178 +0x38
cmd/compile/internal/noder.(*reader2).structType(0xc0005d7680)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:268 +0xf2
cmd/compile/internal/noder.(*reader2).doTyp(0xc0005d7680)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:254 +0x39c
cmd/compile/internal/noder.(*pkgReader2).typIdx(0xc0000e0900, {0xb0, 0xe0}, 0xc0005cc4e0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:206 +0x175
cmd/compile/internal/noder.(*reader2).typ(0xc0005cc540)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:178 +0x38
cmd/compile/internal/noder.(*pkgReader2).objIdx.func1.1(0xd1dca0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:406 +0x3f
cmd/compile/internal/types2.(*Named).load.func1()
	/usr/local/go.tip/src/cmd/compile/internal/types2/named.go:66 +0x2a
sync.(*Once).doSlow(0xb70e27, 0xc0000c80a0)
	/usr/local/go.tip/src/sync/once.go:68 +0xc2
sync.(*Once).Do(...)
	/usr/local/go.tip/src/sync/once.go:59
cmd/compile/internal/types2.(*Named).load(0xc0004addd0)
	/usr/local/go.tip/src/cmd/compile/internal/types2/named.go:56 +0xdb
cmd/compile/internal/types2.(*Named).TParams(...)
	/usr/local/go.tip/src/cmd/compile/internal/types2/named.go:124
cmd/compile/internal/types2.(*typeWriter).typ(0xc0000cff90, {0xe50750, 0xc0004addd0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/typestring.go:237 +0x3bb
cmd/compile/internal/types2.(*typeWriter).typeList(0xc0000cff90, {0xc0003dae80, 0x2, 0x138b1e0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/typestring.go:272 +0xd2
cmd/compile/internal/types2.typeHash({0xe50750, 0xc0004ade60}, {0xc0003dae80, 0x1, 0x2})
	/usr/local/go.tip/src/cmd/compile/internal/types2/subst.go:269 +0xef
cmd/compile/internal/types2.(*Checker).instance(0xc0000f4640, {0x0, 0x0, 0x0}, {0xe50750, 0xc0004ade60}, {0xc0003dae80, 0xc0005a9e90, 0xc0005d70e0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/instantiate.go:128 +0x31f
cmd/compile/internal/types2.Instantiate(0xc0005d70e0, {0xe50750, 0xc0004ade60}, {0xc0003dae80, 0x2, 0x2}, 0x0)
	/usr/local/go.tip/src/cmd/compile/internal/types2/instantiate.go:53 +0xb1
cmd/compile/internal/noder.(*reader2).doTyp(0xc0005d70e0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:231 +0x150
cmd/compile/internal/noder.(*pkgReader2).typIdx(0xc0000e0900, {0xc0005d7080, 0xb}, 0xc0005d6c60)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:206 +0x175
cmd/compile/internal/noder.(*reader2).typ(0xc0005d7080)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:178 +0x38
cmd/compile/internal/noder.(*reader2).doTyp(0xc0005d7080)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:248 +0x7d
cmd/compile/internal/noder.(*pkgReader2).typIdx(0xc0000e0900, {0xc0005d6cc0, 0x81}, 0xc0005d6c60)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:206 +0x175
cmd/compile/internal/noder.(*reader2).typ(0xc0005d6cc0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:178 +0x38
cmd/compile/internal/noder.(*reader2).param(0xc0005d6cc0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:333 +0x69
cmd/compile/internal/noder.(*reader2).params(0xc0005d6cc0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:323 +0x74
cmd/compile/internal/noder.(*reader2).signature(0xc0005d6cc0, 0x0)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:313 +0x47
cmd/compile/internal/noder.(*pkgReader2).objIdx.func1()
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:392 +0x4f5
cmd/compile/internal/types2.resolve.func1()
	/usr/local/go.tip/src/cmd/compile/internal/types2/scope.go:255 +0x3c
sync.(*Once).doSlow(0x6, 0x7f9623e772cb)
	/usr/local/go.tip/src/sync/once.go:68 +0xc2
sync.(*Once).Do(...)
	/usr/local/go.tip/src/sync/once.go:59
cmd/compile/internal/types2.resolve({0x7f9623e3d55e, 0xc0004fac00}, {0xe61cf8, 0xc0005d24e0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/scope.go:254 +0x85
cmd/compile/internal/types2.(*Scope).Lookup(...)
	/usr/local/go.tip/src/cmd/compile/internal/types2/scope.go:72
cmd/compile/internal/noder.(*reader2).obj(0xc0004ff620)
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:346 +0xd0
cmd/compile/internal/noder.readPackage2(0xc0000f4640, 0xc0003d7080, {{0xc0000b24b1, 0x22}, {0x5cc, 0x5cd, 0x607, 0x687, 0x79f, 0xc78, ...}, ...})
	/usr/local/go.tip/src/cmd/compile/internal/noder/reader2.go:46 +0x314
cmd/compile/internal/noder.unified.func1({0x7f9623e35ccd, 0x5ccd}, 0xc0005001e0, 0x22, 0x40a3bd)
	/usr/local/go.tip/src/cmd/compile/internal/noder/unified.go:86 +0x108
cmd/compile/internal/noder.readImportFile({0xc0000b24b1, 0xc7f9e0}, 0xc0000e0180, 0xc0003c8901, 0xc0003d7080)
	/usr/local/go.tip/src/cmd/compile/internal/noder/import.go:282 +0x493
cmd/compile/internal/noder.(*gcimports).ImportFrom(0xc77f60, {0xc0000b24b1, 0xc00009f4a0}, {0x0, 0xc0000f18f0}, 0x20)
	/usr/local/go.tip/src/cmd/compile/internal/noder/import.go:64 +0x3a
cmd/compile/internal/types2.(*Checker).importPackage(0xc0000f4640, {0xc000376ea0, 0x0, 0x0}, {0xc0000b24b1, 0x22}, {0x7ffed7a85ba1, 0x9})
	/usr/local/go.tip/src/cmd/compile/internal/types2/resolver.go:144 +0x19e
cmd/compile/internal/types2.(*Checker).collectObjects(0xc0000f4640)
	/usr/local/go.tip/src/cmd/compile/internal/types2/resolver.go:252 +0x10b0
cmd/compile/internal/types2.(*Checker).checkFiles(0xc0000f4640, {0xc0000b45e0, 0x0, 0x0})
	/usr/local/go.tip/src/cmd/compile/internal/types2/check.go:266 +0x1ec
cmd/compile/internal/types2.(*Checker).Files(...)
	/usr/local/go.tip/src/cmd/compile/internal/types2/check.go:245
cmd/compile/internal/noder.checkFiles({0xc0000b4450, 0x1, 0xc0000d1548})
	/usr/local/go.tip/src/cmd/compile/internal/noder/irgen.go:65 +0x485
cmd/compile/internal/noder.writePkgStub({0xc0000b4450, 0x1, 0x1})
	/usr/local/go.tip/src/cmd/compile/internal/noder/unified.go:163 +0x46
cmd/compile/internal/noder.unified({0xc0000b4450, 0x0, 0x2})
	/usr/local/go.tip/src/cmd/compile/internal/noder/unified.go:90 +0x12a
cmd/compile/internal/noder.LoadPackage({0xc0000ba0e0, 0x1, 0x0})
	/usr/local/go.tip/src/cmd/compile/internal/noder/noder.go:84 +0x35d
cmd/compile/internal/gc.Main(0xd1dcb0)
	/usr/local/go.tip/src/cmd/compile/internal/gc/main.go:190 +0xaf3
main.main()
	/usr/local/go.tip/src/cmd/compile/main.go:55 +0xdd
FAIL	github.com/go-json-experiment/json [build failed]

The relevant change that seems to cause an ICE is the code that instantiates a generic type:

Unmarshalers: jsonv2.UnmarshalFuncV2(func(opts jsonv2.UnmarshalOptions, dec *jsonv2.Decoder, val *interface{}) (err error) {
	if dec.PeekKind() == '0' {
		*val = jsonv2.RawValue(nil)
	}
	return jsonv2.SkipFunc
}),

\cc @danscales @mdempsky @griesemer @ianlancetaylor

Activity

cherrymui
added
NeedsFixThe path to resolution is known, but the work has not been done.
on Sep 3, 2021
cherrymui
added this to the Go1.18 milestone on Sep 3, 2021
cuonglm

cuonglm commented on Sep 4, 2021

@cuonglm
cuonglm
on Sep 4, 2021 · Hidden as outdated
dsnet

dsnet commented on Sep 4, 2021

@dsnet
dsnet
on Sep 4, 2021 · Hidden as outdated
Author
cuonglm

cuonglm commented on Sep 4, 2021

@cuonglm
cuonglm
on Sep 4, 2021 · Hidden as outdated
anacrolix

anacrolix commented on Sep 4, 2021

@anacrolix
anacrolix
on Sep 4, 2021
Contributor

I get the same error with anacrolix/stm@80e033a, running go test ./rate.

# github.com/anacrolix/stm/rate [github.com/anacrolix/stm/rate.test]
<autogenerated>:1: internal compiler error: Dictionary should have already been generated: github.com/anacrolix/stm..dict.(*Var[bool]).Set

goroutine 1 [running]:
runtime/debug.Stack()
	/Users/anacrolix/src/go.master/src/runtime/debug/stack.go:24 +0x65
cmd/compile/internal/base.FatalfAt({0x27faeca, 0x0}, {0x190f2b3, 0x34}, {0xc0004caec0, 0x2, 0x2})
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/base/print.go:227 +0x154
cmd/compile/internal/base.Fatalf(...)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/base/print.go:196
cmd/compile/internal/reflectdata.getDictionary(0xc000961630, {0xc000a1a4b8, 0x1, 0x1})
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/reflectdata/reflect.go:2043 +0x245
cmd/compile/internal/reflectdata.methodWrapper(0xc0008a79d0, 0xc000961680, 0x40)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/reflectdata/reflect.go:1909 +0x10ea
cmd/compile/internal/reflectdata.methods(0xc0008a79d0)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/reflectdata/reflect.go:347 +0x39f
cmd/compile/internal/reflectdata.writeITab(0xc000b27e00, 0xc0008a79d0, 0xc00086a070)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/reflectdata/reflect.go:1290 +0x26a
cmd/compile/internal/reflectdata.ITabLsym(0xc00086a070, 0xc0008776c0)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/reflectdata/reflect.go:847 +0x19c
cmd/compile/internal/noder.(*irgen).finalizeSyms(0xc0000bae00)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/noder/stencil.go:1632 +0x7a5
cmd/compile/internal/noder.(*irgen).stencil(0xc0000bae00)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/noder/stencil.go:234 +0x46d
cmd/compile/internal/noder.(*irgen).generate(0xc0000bae00, {0xc000096b20, 0x2, 0xc0})
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/noder/irgen.go:304 +0x359
cmd/compile/internal/noder.check2({0xc000096b20, 0x2, 0x2})
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/noder/irgen.go:92 +0x175
cmd/compile/internal/noder.LoadPackage({0xc0000ba0e0, 0x2, 0x0})
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/noder/noder.go:90 +0x335
cmd/compile/internal/gc.Main(0x191aec0)
	/Users/anacrolix/src/go.master/src/cmd/compile/internal/gc/main.go:190 +0xaf3
main.main()
	/Users/anacrolix/src/go.master/src/cmd/compile/main.go:55 +0xdd

FAIL	github.com/anacrolix/stm/rate [build failed]
FAIL
cuonglm

cuonglm commented on Sep 5, 2021

@cuonglm
cuonglm
on Sep 5, 2021 · edited by cuonglm
Member

Simpler reproducer for unified IR:

$ cat p.go
package p

type MarshalOptions struct {
	Marshalers *Marshalers
}

type Encoder struct {}

type Marshalers = arshalers[MarshalOptions, Encoder]

type arshalers[Options, Coder any] struct{}

func MarshalFuncV1[T any](fn func(T) ([]byte, error)) *Marshalers {
	return &Marshalers{}
}
$ cat p_test.go
package p

I think this is problem of types2 lazy loading API. In (*Named).load, we call t.resolve(t) which may refer back to the same named type, so calling (*Named).load again, causing the deadlock. In above code, it's the arshalers type. I can't find a way to trigger the bug purely in types2, though.

cc @mdempsky @griesemer

danscales

danscales commented on Sep 7, 2021

@danscales
danscales
on Sep 7, 2021
Contributor

I have a fix for the -G=3 version of the issue. I will not close automatically, so the bug can remain open for the unified IR version of the issue. Or we can create separate issues.

gopherbot

gopherbot commented on Sep 7, 2021

@gopherbot
gopherbot
on Sep 7, 2021
Contributor

Change https://golang.org/cl/347909 mentions this issue: cmd/compile: make sure imported instantiated types have their methods created

danscales

danscales commented on Sep 7, 2021

@danscales
danscales
on Sep 7, 2021
Contributor

Thanks for all the issue reports, @dsnet and @anacrolix . They are all useful, so don't worry about filing them. Of course, somewhat minimized cases are always appreciated (so it is easier to add a test case).

gopherbot

gopherbot commented on Sep 7, 2021

@gopherbot
gopherbot
on Sep 7, 2021
Contributor

Change https://golang.org/cl/347534 mentions this issue: cmd/compile: fix deadlock in (*Named).load

gopherbot
added a commit that references this issue on Sep 7, 2021

cmd/compile: fix deadlock in (*Named).load

2a48452
gopherbot
closed this as completedin 8078355on Sep 7, 2021
ALTree
mentioned this on Jan 12, 2022
golang
locked and limited conversation to collaborators on Sep 7, 2022
gopherbot
added
FrozenDueToAge
on Sep 7, 2022
julieqiu
added this to Go Security and removed this from Go Securityon Sep 8, 2022
salewski
added a commit that references this issue on Apr 29, 2024

cmd/compile: make sure imported instantiated types have their methods…

8078355
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @anacrolix@danscales@cuonglm@dsnet@gopherbot

        Issue actions
          cmd/compile: internal compiler error instantiating generic type · Issue #48185 · golang/go