cmd/compile: out of memory building github.com/cosmos72/gomacro with go1.11beta2 on 386/linux, arm/linux

[jcajka]

What version of Go are you using (go version)?

go version go1.11beta2 linux/386

Does this issue reproduce with the latest release?

Yes, master is affected.
go version devel +48c79734ff Fri Jul 20 20:08:15 2018 +0000 linux/386
not observed with 1.10

What operating system and processor architecture are you using (go env)?

386/linux

What did you do?

go get -u github.com/cosmos72/gomacro

What did you expect to see?

Successful get/build

What did you see instead?

# github.com/cosmos72/gomacro/imports/syscall
runtime: out of memory: cannot allocate 57344-byte block (2901835776 in use)
fatal error: out of memory

runtime stack:
runtime.throw(0x882f7ac, 0xd)
	/usr/lib/golang/src/runtime/panic.go:608 +0x6a
runtime.(*mcache).refill(0xb7ed5000, 0x67)
	/usr/lib/golang/src/runtime/mcache.go:124 +0xf2
runtime.(*mcache).nextFree.func1()
	/usr/lib/golang/src/runtime/malloc.go:749 +0x28
runtime.systemstack(0xa890460)
	/usr/lib/golang/src/runtime/asm_386.s:396 +0x53
runtime.mstart()
	/usr/lib/golang/src/runtime/proc.go:1229

goroutine 52 [running]:
runtime.systemstack_switch()
	/usr/lib/golang/src/runtime/asm_386.s:357 fp=0x1a2884c4 sp=0x1a2884c0 pc=0x8095700
runtime.(*mcache).nextFree(0xb7ed5000, 0x67, 0x0, 0x0, 0x897ab00)
	/usr/lib/golang/src/runtime/malloc.go:748 +0xa5 fp=0x1a2884ec sp=0x1a2884c4 pc=0x80513f5
runtime.mallocgc(0x2500, 0x0, 0x1500, 0xbf3e7a80)
	/usr/lib/golang/src/runtime/malloc.go:903 +0x608 fp=0x1a288540 sp=0x1a2884ec pc=0x8051b98
runtime.growslice(0x87d84e0, 0xbf3e7a80, 0x6a0, 0x6a0, 0x6a1, 0xbf3e7a80, 0x540, 0x6a0)
	/usr/lib/golang/src/runtime/slice.go:197 +0x1bb fp=0x1a288580 sp=0x1a288540 pc=0x8081b9b
cmd/compile/internal/ssa.(*regAllocState).regalloc(0xdca4000, 0xb1c00b0)
	/usr/lib/golang/src/cmd/compile/internal/ssa/regalloc.go:1619 +0x4160 fp=0x1a288d1c sp=0x1a288580 pc=0x822f160
cmd/compile/internal/ssa.regalloc(0xb1c00b0)
	/usr/lib/golang/src/cmd/compile/internal/ssa/regalloc.go:145 +0x4f fp=0x1a288d2c sp=0x1a288d1c pc=0x8227d6f
cmd/compile/internal/ssa.Compile(0xb1c00b0)
	/usr/lib/golang/src/cmd/compile/internal/ssa/compile.go:70 +0x26c fp=0x1a28bf04 sp=0x1a288d2c pc=0x81efbec
cmd/compile/internal/gc.buildssa(0xa40a0c0, 0x3, 0x0)
	/usr/lib/golang/src/cmd/compile/internal/gc/ssa.go:211 +0x8ca fp=0x1a28bf98 sp=0x1a28bf04 pc=0x86f0b6a
cmd/compile/internal/gc.compileSSA(0xa40a0c0, 0x3)
	/usr/lib/golang/src/cmd/compile/internal/gc/pgen.go:259 +0x2d fp=0x1a28bfcc sp=0x1a28bf98 pc=0x86c5a8d
cmd/compile/internal/gc.compileFunctions.func2(0xdbb7e00, 0xdbb8610, 0x3)
	/usr/lib/golang/src/cmd/compile/internal/gc/pgen.go:323 +0x3b fp=0x1a28bfe0 sp=0x1a28bfcc pc=0x87633ab
runtime.goexit()
	/usr/lib/golang/src/runtime/asm_386.s:1324 +0x1 fp=0x1a28bfe4 sp=0x1a28bfe0 pc=0x8096f51
created by cmd/compile/internal/gc.compileFunctions
	/usr/lib/golang/src/cmd/compile/internal/gc/pgen.go:321 +0xee

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xdbb8618)
	/usr/lib/golang/src/runtime/sema.go:56 +0x2e
sync.(*WaitGroup).Wait(0xdbb8610)
	/usr/lib/golang/src/sync/waitgroup.go:130 +0x7c
cmd/compile/internal/gc.compileFunctions()
	/usr/lib/golang/src/cmd/compile/internal/gc/pgen.go:333 +0x189
cmd/compile/internal/gc.Main(0x884b214)
	/usr/lib/golang/src/cmd/compile/internal/gc/main.go:647 +0x22f3
main.main()
	/usr/lib/golang/src/cmd/compile/main.go:51 +0x7e

First observed via https://bugzilla.redhat.com/show_bug.cgi?id=1607270 (also noted arm as affected)
I assume that go build is exceeding the address space size at the build time on 32-bit targets.
Reducing the parallelism of the build via -p flag has no effect.

[bcmills]

See also #26186.

It's a bit strange that you're hitting the error at that point, though: 2901835776+57344 bytes is only ~2.7 GiB, and my impression is that 32-bit Linux allows 3GiB of address space. I wonder where the other ~0.3 GiB went?

(0.3 GiB of headroom is too close for comfort either way, though.)

CC: @randall77 @griesemer @josharian @cherrymui @dr2chase

[bcmills]

Possibly the same underlying cause as #26523.

[cherrymui]

Probably this is not specific to 386 or ARM. Even on AMD64:

$ cd $GOPATH/src/github.com/cosmos72/gomacro/imports/syscall
$ /usr/bin/time go tool compile -p github.com/cosmos72/gomacro/imports/syscall ./a_package.go ./syscall_linux_amd64.go 
45.02user 1.20system 0:43.96elapsed 105%CPU (0avgtext+0avgdata 3766164maxresident)k
0inputs+26928outputs (0major+984217minor)pagefaults 0swaps
$ /usr/bin/time go1.10 tool compile -p github.com/cosmos72/gomacro/imports/syscall ./a_package.go ./syscall_linux_amd64.go 
11.64user 0.33system 0:11.21elapsed 106%CPU (0avgtext+0avgdata 1072588maxresident)k
0inputs+7216outputs (0major+290042minor)pagefaults 0swaps

The compiler at tip is now taking much longer time and using much more memory.

The ~2.7 GiB is probably just the physical RAM size on that machine?

[randall77]

Definitely looks like a regression. I suspect we've reverted to initializing big maps with code instead of with a loop over key/value slices. Investigating...

[randall77]

My map initialization thought was a red herring. Both 1.10 and tip initialize with code because the value is not a compile-time constant (it's the result of a reflect.Value call).
Most of the time & space is in regalloc. Go 1.10 used ~1GB and takes ~7sec on my machine. Tip uses ~3.7GB and takes ~33 seconds.

The most obvious difference between the assembly outputs is that reflect.Value is inlined at tip. The assembly code is ~2x as big. This looks like O(n^2) behavior in regalloc, 2x code is 4x time and memory, which is about what I'm seeing. Regalloc is known to have this problem (#25120).

[dr2chase]

Should we explicitly //go:noinline reflect.Value?

[randall77]

That's the simple fix (doing that reverts to 1.10 behavior).
This is more generally a case of inlining heuristics going wrong. For big functions I think we want to be more strict about not doing inlining that increases code size. I'm going to see if I can do something simple that also fixes this issue and is safe enough at this point in the cycle.
There's the larger #25120 issue which we can't fix for 1.11. In addition there's another issue here where we evaluate all ~2K values, then assign them to the map. It would be better (stacksize-wise, and maybe regalloc-wise) to evaluate and then assign the values one at a time. I'll open a 1.12 issue for that.