runtime: missed deferred calls [1.16 backport]

[mdempsky]

This program should run successfully: https://play.golang.org/p/XIVsN45Oyzz

It does with gccgo and with cmd/compile using -gcflags=-N, but not when compiled normally. It also still fails at CL 286712, PS 3.

The problem is the "defer step(12)" and "defer step(13)" calls are being missed. Instead, execution jumps directly to the "step(14)" defer.

Sorry the reproducer is still so large. I'm having trouble minimizing it any further. It seems particularly sensitive to changes that I wouldn't expect to have any effect.

/cc @danscales @randall77

[mdempsky]

FWIW, since this is a valid program that's misbehaving silently (modulo the test program's own consistency checks), I think this is probably a higher priority issue than #43920 or #43942, which are also valid programs but failing loudly.

[mdempsky]

Somewhat more minimized test case: https://play.golang.org/p/60U6DFD6Ew- https://play.golang.org/p/Zpgkg3tZ8BO https://play.golang.org/p/jP3CpfpxSku https://play.golang.org/p/mX8ZnRSzZEh https://play.golang.org/p/AjhP4QHH1QU

After the panic(4), the ok = true assignment should execute. But instead execution jumps directly to the expect(4, recover())+log.Fatal defer.

It seems like it's particularly sensitive to stack frame layout. In particular, the issue was I needed to keep making matching changes to both of the main function literals within main. It seems like some of the state left on the stack by the first function is getting reused without proper reinitialization by the second function?

[mdempsky]

Having stared at these cases a bunch, I'm starting to think they're likely duplicates. In each of the reproducers, there's some code of the shape:

defer func() {
	defer func() {
		expect(3, recover())
	}()
	defer panic(3)
	panic(2)
}()
defer func() {
	expect(1, recover())
}()
panic(1)

The differences are all in the code around the above. I suspect the above code is just corrupting the defer stack, and then the surrounding code just influences how that corruption ultimately manifests. Most commonly it's simply "bad defer entry in panic" (#43920), but when the stars align (or stacks align?) it manifests instead as issues like #43941 and #43942.

[zigo101]

A small variant:

package main

func main() {
	defer func() {
		println(recover().(int))
	}()
	func() {
		func() (_ [2]int) { type _ int; return }()
		func() {
			defer func() {
				defer func() {
					recover()
				}()
				defer panic(3)
				panic(2)
			}()
			defer func() {
				recover()
			}()
			panic(1)
		}()
		defer func() {}()
	}()
	
	var x = 123
	func() {
		defer print(x) // not executed!
		func() {
			defer func() {}()
			panic(4)
		}()
	}()
}

[batara666]

this need to be fix

[ianlancetaylor]

@go101 I think that your example is a different problem. Please open a different issue for it. Thanks.

[ianlancetaylor]

@danscales The test case in the initial comment appears to work with Go 1.14, Go 1.17, and tip. It fails with Go 1.15 and Go 1.16. Go 1.15 is no longer supported. Do you think it would be feasible to backport the fix (whatever it is) to Go 1.16?