[clang] Always pass fp128 arguments indirectly on Windows

Clang currently passes and returns `__float128` in vector registers on
MinGW targets. However, the Windows x86-64 calling convention [1] states
the following:

    __m128 types, arrays, and strings are never passed by immediate
    value. Instead, a pointer is passed to memory allocated by the
    caller. Structs and unions of size 8, 16, 32, or 64 bits, and __m64
    types, are passed as if they were integers of the same size. Structs
    or unions of other sizes are passed as a pointer to memory allocated
    by the caller. For these aggregate types passed as a pointer,
    including __m128, the caller-allocated temporary memory must be
    16-byte aligned.

Based on the above it sounds like `__float128` should be passed
indirectly; this is what MinGW GCC already does, so change Clang to
match. Passing by value causes problems with varargs. E.g. the below
completes successfully when built with GCC but has a runtime crash when
built with Clang:

    void va_f128(int count, ...) {
        va_list args;
        va_start(args, count);
        __float128 val = va_arg(args, __float128);
        va_end(args);
    }

    int main() {
        va_f128(0, 0.0);
    }

This patch fixes the above. It also resolves crashes when calling
GCC-built f128 libcalls.

Regarding return values, the documentation states:

    A scalar return value that can fit into 64 bits, including the __m64
    type, is returned through RAX. Non-scalar types including floats,
    doubles, and vector types such as __m128, __m128i, __m128d are
    returned in XMM0.

This makes it sound like it should be acceptable to return `__float128`
in XMM0. However, GCC returns `__float128` on the stack, so do the same
here to be consistent.

Clang's MSVC targets do not support `__float128` or `_Float128`, but
these changes would also apply there if it is eventually enabled.

[1]: https://learn.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-170

Author: tgross35

clang/lib/CodeGen/Targets/X86.cpp

@@ -3390,6 +3390,7 @@ ABIArgInfo WinX86_64ABIInfo::classify(QualType Ty, unsigned &FreeSSERegs,
 
     case BuiltinType::Int128:
     case BuiltinType::UInt128:
+    case BuiltinType::Float128:
       // If it's a parameter type, the normal ABI rule is that arguments larger
       // than 8 bytes are passed indirectly. GCC follows it. We follow it too,
       // even though it isn't particularly efficient.
@@ -3399,7 +3400,7 @@ ABIArgInfo WinX86_64ABIInfo::classify(QualType Ty, unsigned &FreeSSERegs,
             /*ByVal=*/false);
 
       // Mingw64 GCC returns i128 in XMM0. Coerce to v2i64 to handle that.
-      // Clang matches them for compatibility.
+      // Clang matches them for compatibility. f128 uses the same ABI.
       return ABIArgInfo::getDirect(llvm::FixedVectorType::get(
           llvm::Type::getInt64Ty(getVMContext()), 2));
 

clang/test/CodeGen/win64-fp128.c

@@ -3,17 +3,16 @@
 // __float128 is unsupported on MSVC
 
 __float128 fp128_ret(void) { return 0; }
-// GNU64: define dso_local fp128 @fp128_ret()
+// GNU64: define dso_local <2 x i64> @fp128_ret()
 
 __float128 fp128_args(__float128 a, __float128 b) { return a * b; }
-// GNU64: define dso_local fp128 @fp128_args(fp128 noundef %a, fp128 noundef %b)
+// GNU64: define dso_local <2 x i64> @fp128_args(ptr noundef %0, ptr noundef %1)
 
 void fp128_vararg(int a, ...) {
   // GNU64-LABEL: define dso_local void @fp128_vararg
   __builtin_va_list ap;
   __builtin_va_start(ap, a);
   __float128 i = __builtin_va_arg(ap, __float128);
-  // movaps  xmm0, xmmword ptr [rax]
   // GNU64: load ptr, ptr
   // GNU64: load fp128, ptr
   __builtin_va_end(ap);