[DirectX] Fix crash in DXILOpBuilder for vector types

[bogner]

This function needs to return the "undefined" sigil for unknown types so that the actual error handling triggers instead of a crash.

[llvmbot]

@llvm/pr-subscribers-backend-directx

Author: Justin Bogner (bogner)

Changes

This function needs to return the "undefined" sigil for unknown types so that the actual error handling triggers instead of a crash.

---

Full diff: https://github.com/llvm/llvm-project/pull/107334.diff

2 Files Affected:

• (modified) llvm/lib/Target/DirectX/DXILOpBuilder.cpp (+1-2)
• (added) llvm/test/CodeGen/DirectX/sin_vector_error.ll (+11)

diff --git a/llvm/lib/Target/DirectX/DXILOpBuilder.cpp b/llvm/lib/Target/DirectX/DXILOpBuilder.cpp
index ab3ea61d05fc45..efe019a07acaa9 100644
--- a/llvm/lib/Target/DirectX/DXILOpBuilder.cpp
+++ b/llvm/lib/Target/DirectX/DXILOpBuilder.cpp
@@ -123,8 +123,7 @@ static OverloadKind getOverloadKind(Type *Ty) {
   case Type::StructTyID:
     return OverloadKind::ObjectType;
   default:
-    llvm_unreachable("invalid overload type");
-    return OverloadKind::VOID;
+    return OverloadKind::UNDEFINED;
   }
 }
 
diff --git a/llvm/test/CodeGen/DirectX/sin_vector_error.ll b/llvm/test/CodeGen/DirectX/sin_vector_error.ll
new file mode 100644
index 00000000000000..45b8d403390b91
--- /dev/null
+++ b/llvm/test/CodeGen/DirectX/sin_vector_error.ll
@@ -0,0 +1,11 @@
+; RUN: not opt -S -dxil-op-lower -mtriple=dxil-pc-shadermodel6.0-library %s 2>&1 | FileCheck %s
+; The sin intrinsic needs to be scalarized before op lowering
+
+; CHECK: error:
+; CHECK-SAME: in function sin_vector
+; CHECK-SAME: Cannot create Sin operation: Invalid overload type
+
+define <4 x float> @sin_vector(<4 x float> %a) {
+  %x = call <4 x float> @llvm.sin.v4f32(<4 x float> %a)
+  ret <4 x float> %x
+}