Too strong validity checks for [[clang::musttail]]

[davidbolvansky]

int base(int);

int ok1() { return base(5); }
int ok2(int x, int y /* unused */) { return base(x); }

ok1():                                # @ok1()
        mov     edi, 5
        jmp     base(int)@PLT                    # TAILCALL
ok2(int, int):                               # @ok2(int, int)
        jmp     base(int)@PLT                    # TAILCALL

So clearly LLVM can apply tail call optimization for these cases. But Clang does not think so and emits errors for these cases:

int f1() { [[clang::musttail]] return base(5); }
int f2(int x, int y /* unused */) { [[clang::musttail]] return base(5); }

error: cannot perform a tail call to function 'base' because its signature is incompatible with the calling function
int f1() { [[clang::musttail]] return base(5); }
^
error: cannot perform a tail call to function 'base' because its signature is incompatible with the calling function
int f2(int x, int y /* unused */) { [[clang::musttail]] return base(5); }

https://godbolt.org/z/7h6h1YjWT

[davidbolvansky]

cc @haberman
cc @efriedma-quic

Maybe Clang should just drop signature check?

[efriedma-quic]

In general, we need to restrict the cases where we musttail: C calling convention rules limit the stack space available for tail calls, so we can't tail-call in the general case using the normal C calling convention. To ensure we have enough stack space, and that we can satisfy any other unusual conditions, we check that the signature strictly matches. This is why both clang and LLVM IR restrict musttail to those cases.

I guess there are a few things we could do:

1. Expose tailcc in clang, and relax the [[musttail]] rules for calls to such functions.
2. Try to come up with some rule that covers your specific case: allow tail-calling a function where the called function has fewer arguments, but the signature otherwise matches, or something like that. That probably works for most targets. There might be edge cases here I'm not thinking about.
3. Add some sort of [[clang::nonportable_musttail]] that makes the check to do some target-specific computation.

[haberman]

Yes, in general the rules around musttail are intended to provide assurances that the tail call can be optimized on all targets, not just one.

When I implemented musttail for Clang, I followed the existing rules that were previously established for LLVM IR. The rules would need to be loosed at both the Clang and LLVM levels (or we could expose tailcc as suggested above).

Note that even the existing musttail rules are not stringent enough to support all backends: #50758

Loosening the rules would probably create more cases where we see these kinds of crashes. In other words, the existing attribute is already not fully portable.

[davidbolvansky]

musttail are intended to provide assurances that the tail call can be optimized on all targets

but there is no such ‘promise’ in https://clang.llvm.org/docs/AttributeReference.html and users may not need this additional assurance, which sadly, restricts this feature.

Maybe we could have musttail and portable_musttail or introduce optional argument to musttail like ´__nonportable__’.

I linked above link to CTRE project where they would like to use this feature but they cannot (and they consider current behaviour rather strange, and I agree).

[davidbolvansky]

In other words, the existing attribute is already not fully portable.

This is right - best effort feature, but no strict promises, so some obvious cases could be diagnosed by Clang and anything else by backend IMHO.

[jacobsa]

Here is another example where I suspect the check is too stringent in a similar way as the original example:

struct Foo {
  __attribute__((noinline)) int foo(int) { return 0; }
};

struct Bar {
  Foo foo;

  int bar(int);
};

int Bar::bar(const int x) {
  return foo.foo(x);
}

A tail call works just fine (compiler explorer) on the targets I've checked. For example, x86-64:

Bar::bar(int):                          # @Bar::bar(int)
        jmp     Foo::foo(int)                   # TAILCALL

But adding [[clang::musttail]] results in an error. As far as I know all of the discussion above applies to this too, so just noting it to use as a potential test case whenever an outcome is decided.

Personally I would like the semantics "give me an error if this can't be a tail call on this target", not the semantics "give me an error if this can't be a tail call on all targets clang supports". I suspect most people would be looking for the former, since most people care about nearly none of the supported targets by volume.

[davidbolvansky]

Personally I would like the semantics "give me an error if this can't be a tail call on this target", not the semantics "give me an error if this can't be a tail call on all targets clang supports".

Strong +1, I would love such behaviour as well! So probably we need to come up with new attribute which would implement such behaviour.

[davidbolvansky]

Proposal for [[clang::nonportable_musttail]]
https://reviews.llvm.org/D147714

[Alcaro]

I'm just some random guy, so feel free to ignore me, but I would like to see some more tests for that attribute.

In particular, I want some tests that it properly errors out if asked for something impossible, and doesn't (for example) ICE, as in haberman's example https://bugs.llvm.org/show_bug.cgi?id=51416.

int a(int a1, int a2);
int b(int a1)
{
[[clang::nonportable_musttail]]
return a(1, a1); // legal, all args are in regs
}

int c(int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9, int a10);
int d(int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9)
{
[[clang::nonportable_musttail]]
return c(a1, a2, a3, a4, a5, a6, a7, a8, a9, 1); // expected-error {{can't push an extra argument onto a stack frame that's not ours}}
}

int e(int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9, int a10)
{
[[clang::nonportable_musttail]]
return c(1, a1, 3, a4, a5, a6, a7, a8, a9, a10); // legal, only register arguments are changed
}

int f(int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9, int a10)
{
[[clang::nonportable_musttail]]
return c(a1, a2, a3, a4, a5, a6, a7, 1, 2, 3); // legal, but Clang does currently not perform that optimization
}

int g(int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9, int a10)
{
[[clang::nonportable_musttail]]
return a(1, a2); // legal, x64 sysv abi is caller pop (if there are stack args at all)
}

int h(int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9, int a10, int a11)
{
[[clang::nonportable_musttail]]
return c(a1, a2, a3, a4, a5, a6, a7, a8, a9, a10); // legal, same reason as g
}

short i();
int j()
{
[[clang::nonportable_musttail]]
return i(); // expected-error {{upper bits of a int16 return value are garbage on x64 sysv abi, this return must sign extend}}
}

int k();
short l()
{
[[clang::nonportable_musttail]]
return k(); // legal, but Clang does currently not perform that optimization
}

The legal-but-unimplemented ones can be marked expected-error for now with a fixme comment, but I would like to see them in the tests.

[davidbolvansky]

Sounds like tests for LLVM backends.