[Hexagon] Cannot handle call operand v8i1

[HazyFish]

Description

Hexagon backend cannot handle call operand v8i1, v16i1, v32i1 with latest LLVM.
The problem doesn't exist when using LLVM 14 / 15 release.

Minimal Reproduction

https://godbolt.org/z/obdbYonah

Code

define i32 @f() {
BB:
  %C = call i32 @f.1(<8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>)
  ret i32 %C
}

declare i32 @f.1(<8 x i1>)

Stack Trace

Call operand #0 has unhandled type v8i1
UNREACHABLE executed at /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/CallingConvLower.cpp:134!
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: ./llvm-project/build-debug/bin/llc -mtriple=hexagon ./crash-reports/dagisel-hexagon/1.ll
1.	Running pass 'Function Pass Manager' on module './crash-reports/dagisel-hexagon/1.ll'.
2.	Running pass 'Hexagon DAG->DAG Pattern Instruction Selection' on function '@f'
 #0 0x00000000047c438a llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/Support/Unix/Signals.inc:569:11
 #1 0x00000000047c453b PrintStackTraceSignalHandler(void*) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/Support/Unix/Signals.inc:636:1
 #2 0x00000000047c2b76 llvm::sys::RunSignalHandlers() /home/henry/aflplusplus-isel/llvm-project/llvm/lib/Support/Signals.cpp:104:5
 #3 0x00000000047c4c65 SignalHandler(int) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/Support/Unix/Signals.inc:407:1
 #4 0x00007f90918c9980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
 #5 0x00007f90907b9e87 raise /build/glibc-CVJwZb/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
 #6 0x00007f90907bb7f1 abort /build/glibc-CVJwZb/glibc-2.27/stdlib/abort.c:81:0
 #7 0x00000000046ec2f0 llvm::install_out_of_memory_new_handler() /home/henry/aflplusplus-isel/llvm-project/llvm/lib/Support/ErrorHandling.cpp:193:0
 #8 0x0000000003337626 llvm::CCState::AnalyzeCallOperands(llvm::SmallVectorImpl<llvm::ISD::OutputArg> const&, bool (*)(unsigned int, llvm::MVT, llvm::MVT, llvm::CCValAssign::LocInfo, llvm::ISD::ArgFlagsTy, llvm::CCState&)) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/CallingConvLower.cpp:136:3
 #9 0x0000000001df86d0 llvm::HexagonTargetLowering::LowerCall(llvm::TargetLowering::CallLoweringInfo&, llvm::SmallVectorImpl<llvm::SDValue>&) const /home/henry/aflplusplus-isel/llvm-project/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp:0:12
#10 0x000000000444cf53 llvm::TargetLowering::LowerCallTo(llvm::TargetLowering::CallLoweringInfo&) const /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:10086:15
#11 0x00000000044756e8 llvm::SelectionDAGBuilder::lowerInvokable(llvm::TargetLowering::CallLoweringInfo&, llvm::BasicBlock const*) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:7790:3
#12 0x00000000044547b3 llvm::SelectionDAGBuilder::LowerCallTo(llvm::CallBase const&, llvm::SDValue, bool, bool, llvm::BasicBlock const*) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:7920:14
#13 0x000000000443e00c llvm::SelectionDAGBuilder::visitCall(llvm::CallInst const&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:8473:1
#14 0x0000000004432e67 llvm::SelectionDAGBuilder::visit(unsigned int, llvm::User const&) /home/henry/aflplusplus-isel/llvm-project/llvm/include/llvm/IR/Instruction.def:209:1
#15 0x000000000443203f llvm::SelectionDAGBuilder::visit(llvm::Instruction const&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:1155:8
#16 0x0000000004527428 llvm::SelectionDAGISel::SelectBasicBlock(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Instruction, true, false, void>, false, true>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Instruction, true, false, void>, false, true>, bool&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:678:3
#17 0x0000000004526f6b llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1603:11
#18 0x0000000004524516 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:467:3
#19 0x0000000001dcd8be llvm::HexagonDAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/Target/Hexagon/HexagonISelDAGToDAG.h:44:23
#20 0x0000000003585535 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/CodeGen/MachineFunctionPass.cpp:91:8
#21 0x0000000003c6cde6 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1430:23
#22 0x0000000003c71c12 llvm::FPPassManager::runOnModule(llvm::Module&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1476:16
#23 0x0000000003c6d6b9 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1545:23
#24 0x0000000003c6d22d llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:535:16
#25 0x0000000003c71ef1 llvm::legacy::PassManager::run(llvm::Module&) /home/henry/aflplusplus-isel/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1672:3
#26 0x0000000000d1edec compileModule(char**, llvm::LLVMContext&) /home/henry/aflplusplus-isel/llvm-project/llvm/tools/llc/llc.cpp:737:41
#27 0x0000000000d1d192 main /home/henry/aflplusplus-isel/llvm-project/llvm/tools/llc/llc.cpp:418:13
#28 0x00007f909079cc87 __libc_start_main /build/glibc-CVJwZb/glibc-2.27/csu/../csu/libc-start.c:344:0
#29 0x0000000000d1c99a _start (./llvm-project/build-debug/bin/llc+0xd1c99a)

[HazyFish]

cc @androm3da

[llvmbot]

@llvm/issue-subscribers-backend-hexagon

[kparzysz-quic]

Passing/returning boolean vectors to functions has never been supported, there no ABI for that. If you want to pass those, you need to convert them to vectors of integers and back.

I can reproduce this crash with LLVM 13 and 14 (I have builds with assertions enabled). With LLVM 10 (without assertions) the code compiles, but the output doesn't pass the indicated value to f.1.

[androm3da]

Passing/returning boolean vectors to functions has never been supported, there no ABI for that. If you want to pass those, you need to convert them to vectors of integers and back.

Sorry if this is a dumb question @kparzysz-quic but could a compiler like clang produce the IR above? And if it did, it would be a clang defect for not recognizing the target's ABI?

[kparzysz-quic]

It's a legitimate question---I don't think that right now people have the ability to write C/C++ code that would translate to something like this, but clang already has some sort of extension allowing boolean vectors (which is only enabled for some GPU target, IIRC), so eventually it may be possible to do this.

The issue is that we'd need to amend the ABI, to either allow passing/returning boolean vectors, or specify the conversions. At the moment LLVM is (AFAIK) the only compiler targeting Hexagon with HVX, so we could get away with handling it internally to LLVM, but it would make it harder for other potential future compilers/code generators to be binary-compatible with LLVM.

My point was really that this is not a regression. Any code trying to do this has never really worked correctly even if it compiled without errors.

[androm3da]

The issue is that we'd need to amend the ABI, to either allow passing/returning boolean vectors, or specify the conversions.

Ok, I think that might be useful, I'll look into this along with the architecture documentation improvements we'll need.

My point was really that this is not a regression. Any code trying to do this has never really worked correctly even if it compiled without errors.

Ok, I understand now.

[HazyFish]

@kparzysz-quic I found another case related to Boolean vectors.

https://godbolt.org/z/cqGEhTM8G

Is this also something that's not supported by ABI, or is it something wrong with the backend?

[kparzysz-quic]

That's an actual bug. Thanks for reporting it.

[HazyFish]

Ok, I will open a new issue for that

That's an actual bug. Thanks for reporting it.

[HazyFish]

#59042