Clang miscompiling code when using -fms-hotpatch and optimizations

[MolecularMatters]

When using the -fms-hotpatch option (which is a requirement for Live++), the tailcall/sibling call optimizations will lead to clang miscompiling function calls.

Example code:

extern "C" {
    void* mi_new(size_t size);
}

void *mi_new_test(size_t count)
{
    return mi_new(count);
}

void *builtin_malloc_test(size_t count)
{
    return __builtin_malloc(count);
}

Using -O1, the code will compile to the following:

"?mi_new_test@@YAPEAX_K@Z":             # @"?mi_new_test@@YAPEAX_K@Z"
        jmp     mi_new                          # TAILCALL
"?builtin_malloc_test@@YAPEAX_K@Z":     # @"?builtin_malloc_test@@YAPEAX_K@Z"
        jmp     malloc                          # TAILCALL

However, adding -fms-hotpatch, the code will miscompile to this:

"?mi_new_test@@YAPEAX_K@Z":             # @"?mi_new_test@@YAPEAX_K@Z"
        xchg    ax, ax

"?builtin_malloc_test@@YAPEAX_K@Z":     # @"?builtin_malloc_test@@YAPEAX_K@Z"
        xchg    ax, ax

As a workaround, specifying -fno-optimize-sibling-calls will produce the correct (but suboptimal) following code:

"?mi_new_test@@YAPEAX_K@Z":             # @"?mi_new_test@@YAPEAX_K@Z"
        sub     rsp, 40
        call    mi_new
        nop
        add     rsp, 40
        ret
"?builtin_malloc_test@@YAPEAX_K@Z":     # @"?builtin_malloc_test@@YAPEAX_K@Z"
        sub     rsp, 40
        call    malloc
        nop
        add     rsp, 40
        ret

Please refer to this godbolt compiler explorer repro: https://gcc.godbolt.org/z/r86nYbqvo

[sylvain-audi]

Linking this issue with #59039, it looks like it's the same pattern.

[shafik]

Duplicate of: #76879