runtime: Go 1.14/Windows asynchronous preemption mechanism likely incompatible with debugging

[aarzilli]

If I understand correctly asynchronous preemption is implemented in Windows by:

1. Calling SuspendThread on the target thread
2. Waiting for SuspendThread to take effect by calling GetThreadContext
3. Injecting a function call on the target thread by manipulating the context and calling SetThreadContext
4. Calling ResumeThread on the target thread

This procedure being implemented by preemptM in src/runtime/os_windows.go.

However, from what I can observe, it seems that GetThreadContext will also return when the target thread get suspended after hitting a software breakpoint (placed by a debugger). This has two effects: first the breakpoint will be missed (because preemptM will manipulate the thread context to insert a function call, masking the breakpoint), secondly the return address for the injected call will be in the middle of an instruction (because the software breakpoint was partially overwriting an instruction).

A debugger that knows about this can work around it by checking if a thread is stopped on the entry point of asyncPreempt, and fix its return address. For this to work however preemptM needs to fully finish its context manipulation, if it ends up being stopped between the point where GetThreadContext returns and SetThreadContext is called the same problem will happen.

I'm not sure that there is a way to use software breakpoints with Go 1.14 with async preemption, as it currently is. Did I misunderstand anything?

[ALTree]

cc @aclements

[aclements]

Can you explain a bit more about how software breakpoints work under Windows (or point me to the relevant APIs)? I thought this would all be fine because SuspendThread acts like a semaphore, but I keep finding more and more little surprises with SuspendThread. :P

One possible, slightly awful work around may be for the debugging to poke a 1 into runtime.debug.asyncpreemptoff when it attaches. (Or if it's starting the process itself, add asyncpreemptoff=1 to GODEBUG.) Obviously not ideal because it may change program behavior.

[aarzilli]

Debuggers call WaitForDebugEvent, when a breakpoint happens (of any kind) the thread that executed the breakpoint is stopped and WaitForDebugEvent returns. Apparently this is also enough for any GetThreadContext call to return. A software breakpoint is just an INT 3 instruction overwriting the first byte of some other instruction, since executing it also updates the value of RIP the context returned by GetThreadContext will have RIP one byte inside whatever instruction was overwritten with the breakpoint.

One only-works-with-delve solution would be to add this between the SuspendThread call and the GetThreadContext call in preemptM:

if debuggerAttached {
    INT 3
}

with debuggerAttached being some global variable that the debugger is supposed to set after attaching. This should prevent (if I'm not wrong) preemptM from seeing the target thread's context before the debugger has a chance to and prevent preemptM from seeing the thread in a weird state.

The only other change needed in the debugger would be to ignore this breakpoint inside preemptM.

Of course this won't solve the problem for any debugger other than delve (also: I haven't tested this, so it's possible that it doesn't work at all).