excessive compile-time and memory usage with `-fsanitize=undefined`

[firewave]

Some (mostly generated) files from https://github.com/mamedev/mame take several minutes to compile with -fsanitize=undefined - e.g. src/devices/cpu/m68000/m68000-sdf.cpp.

They also consume a lot of memory during compilation:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                         
3190941 user      20   0 2333788   2,1g 151380 R 100,0  14,1   1:00.92 clang++

I do not now what information to provide for this to proceed so I am waiting for feedback.

IIRC this also applies to -fsanitize=address.

[efriedma-quic]

Needs preprocessed source and command to reproduce.

You can kill -s SIGSEGV `pgrep clang` to force clang to crash, so it will dump preprocessed source/compile command.

[firewave]

This is with a different file which takes even longer (I terminated it after 12+ minutes).

Stack dump:
0.      Program arguments: clang++ -MMD -MP -MP -DPTR64=1 -DNDEBUG -DCRLF=2 -DLSB_FIRST -DFLAC__NO_DLL -DPUGIXML_HEADER_ONLY -DNATIVE_DRC=drcbe_x64 -DASMJIT_STATIC -DLUA_COMPAT_ALL -DLUA_COMPAT_5_1 -DLUA_COMPAT_5_2 -I../../../../../src/osd -I../../../../../src/emu -I../../../../../src/devices -I../../../../../src/lib -I../../../../../src/lib/util -I../../../../../3rdparty -I../../../../generated/emu -I../../../../generated/emu/layout -I../../../../../3rdparty/asio/include -I../../../../../3rdparty/expat/lib -I../../../../../3rdparty/flac/include -g -m64 -std=c++17 -pipe -g1 -fno-omit-frame-pointer -fno-optimize-sibling-calls -O2 -fno-strict-aliasing -Wno-unknown-pragmas -Wall -Wcast-align -Wformat-security -Wundef -Wwrite-strings -Wno-conversion -Wno-sign-compare -Wno-error=deprecated-declarations -fsanitize=undefined -fno-sanitize=function -fno-sanitize=alignment -fdiagnostics-show-note-include-stack -Wno-error=tautological-compare -Wno-cast-align -Wno-constant-logical-operand -Wno-extern-c-compat -Wno-ignored-qualifiers -Wno-pragma-pack -Wno-unknown-attributes -Wno-unknown-warning-option -Wno-unused-value -Wno-unused-const-variable -Wno-xor-used-as-pow -Wno-bitwise-instead-of-logical -m64 -std=c++17 -Woverloaded-virtual -Wvla -include ../../../../linux_clang/obj/x64/Release/emu.h -o ../../../../linux_clang/obj/x64/Release/src/devices/cpu/z80/nsc800.o -c ../../../../../src/devices/cpu/z80/nsc800.cpp
1.      <eof> parser at end of file
2.      Optimizer
3.      Running pass "function<eager-inv>(ee-instrument<>,lower-expect,simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-range-to-icmp;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts;speculate-blocks;simplify-cond-branch;no-speculate-unpredictables>,sroa<modify-cfg>,early-cse<>)" on module "../../../../../src/devices/cpu/z80/nsc800.cpp"
4.      Running pass "early-cse<>" on function "_ZN13nsc800_device5do_opEv"
 #0 0x00007885cf81a730 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /usr/src/debug/llvm/llvm-19.1.7.src/lib/Support/Unix/Signals.inc:723:22
 #1 0x00007885cf817bf5 llvm::sys::RunSignalHandlers() /usr/src/debug/llvm/llvm-19.1.7.src/lib/Support/Signals.cpp:105:20
 #2 0x00007885cf817bf5 llvm::sys::CleanupOnSignal(unsigned long) /usr/src/debug/llvm/llvm-19.1.7.src/lib/Support/Unix/Signals.inc:367:31
 #3 0x00007885cf701319 HandleCrash /usr/src/debug/llvm/llvm-19.1.7.src/lib/Support/CrashRecoveryContext.cpp:73:5
 #4 0x00007885cf701319 CrashRecoverySignalHandler /usr/src/debug/llvm/llvm-19.1.7.src/lib/Support/CrashRecoveryContext.cpp:390:62
 #5 0x00007885cea4bcd0 (/usr/lib/libc.so.6+0x3dcd0)
 #6 0x00007885cf958d22 bool llvm::DenseMapBase<llvm::DenseMap<llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>, llvm::DenseMapInfo<llvm::BasicBlock*, void>, llvm::detail::DenseMapPair<llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>>>, llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>, llvm::DenseMapInfo<llvm::BasicBlock*, void>, llvm::detail::DenseMapPair<llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>>>::LookupBucketFor<llvm::BasicBlock const*>(llvm::BasicBlock const* const&, llvm::detail::DenseMapPair<llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>> const*&) const /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/ADT/DenseMap.h:674:7
 #7 0x00007885cf958d22 llvm::DenseMapBase<llvm::DenseMap<llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>, llvm::DenseMapInfo<llvm::BasicBlock*, void>, llvm::detail::DenseMapPair<llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>>>, llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>, llvm::DenseMapInfo<llvm::BasicBlock*, void>, llvm::detail::DenseMapPair<llvm::BasicBlock*, std::unique_ptr<llvm::DomTreeNodeBase<llvm::BasicBlock>, std::default_delete<llvm::DomTreeNodeBase<llvm::BasicBlock>>>>>::find(llvm::BasicBlock const*) const /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/ADT/DenseMap.h:166:24
 #8 0x00007885cf958ed7 llvm::DominatorTreeBase<llvm::BasicBlock, false>::getNode(llvm::BasicBlock const*) const /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/Support/GenericDomTree.h:367:30
 #9 0x00007885cf9590dc llvm::DominatorTreeBase<llvm::BasicBlock, false>::properlyDominates(llvm::BasicBlock const*, llvm::BasicBlock const*) const /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/Support/GenericDomTree.h:957:27
#10 0x00007885cf95915c llvm::DominatorTree::dominates(llvm::BasicBlockEdge const&, llvm::BasicBlock const*) const /usr/src/debug/llvm/llvm-19.1.7.src/lib/IR/Dominators.cpp:206:3
#11 0x00007885d0a48b77 replaceDominatedUsesWith<llvm::BasicBlockEdge, llvm::replaceDominatedUsesWith(Value*, Value*, DominatorTree&, const BasicBlockEdge&)::<lambda(const llvm::BasicBlockEdge&, const llvm::Use&)> > /usr/src/debug/llvm/llvm-19.1.7.src/lib/Transforms/Utils/Local.cpp:3465:5
#12 0x00007885d0a48b77 llvm::replaceDominatedUsesWith(llvm::Value*, llvm::Value*, llvm::DominatorTree&, llvm::BasicBlockEdge const&) /usr/src/debug/llvm/llvm-19.1.7.src/lib/Transforms/Utils/Local.cpp:3497:36
#13 0x00007885d0f0fbbf handleBranchCondition /usr/src/debug/llvm/llvm-19.1.7.src/lib/Transforms/Scalar/EarlyCSE.cpp:1231:7
#14 0x00007885d0f122b4 processNode /usr/src/debug/llvm/llvm-19.1.7.src/lib/Transforms/Scalar/EarlyCSE.cpp:1370:41
#15 0x00007885d0f122b4 run /usr/src/debug/llvm/llvm-19.1.7.src/lib/Transforms/Scalar/EarlyCSE.cpp:1806:29
#16 0x00007885d0f149ee llvm::EarlyCSEPass::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /usr/src/debug/llvm/llvm-19.1.7.src/lib/Transforms/Scalar/EarlyCSE.cpp:1838:3
#17 0x00007885d43850c7 llvm::detail::PassModel<llvm::Function, llvm::EarlyCSEPass, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/IR/PassManagerInternal.h:91:3
#18 0x00007885cfa4ad1a llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/IR/PassManagerImpl.h:85:18
#19 0x00007885d2a9eee7 llvm::detail::PassModel<llvm::Function, llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/IR/PassManagerInternal.h:91:3
#20 0x00007885cfa4b22f llvm::ModuleToFunctionPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /usr/src/debug/llvm/llvm-19.1.7.src/lib/IR/PassManager.cpp:129:41
#21 0x00007885d2a9fe47 llvm::detail::PassModel<llvm::Module, llvm::ModuleToFunctionPassAdaptor, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/IR/PassManagerInternal.h:91:3
#22 0x00007885cfa45e39 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /usr/src/debug/llvm/llvm-19.1.7.src/include/llvm/IR/PassManagerImpl.h:85:18
#23 0x00007885d8f325ec llvm::SmallPtrSetImplBase::isSmall() const /usr/include/llvm/ADT/SmallPtrSet.h:197:0
#24 0x00007885d8f325ec llvm::SmallPtrSetImplBase::~SmallPtrSetImplBase() /usr/include/llvm/ADT/SmallPtrSet.h:84:0
#25 0x00007885d8f325ec llvm::SmallPtrSetImpl<llvm::AnalysisKey*>::~SmallPtrSetImpl() /usr/include/llvm/ADT/SmallPtrSet.h:323:0
#26 0x00007885d8f325ec llvm::SmallPtrSet<llvm::AnalysisKey*, 2u>::~SmallPtrSet() /usr/include/llvm/ADT/SmallPtrSet.h:479:0
#27 0x00007885d8f325ec llvm::PreservedAnalyses::~PreservedAnalyses() /usr/include/llvm/IR/Analysis.h:111:0
#28 0x00007885d8f325ec RunOptimizationPipeline /usr/src/debug/clang/clang-19.1.7.src/lib/CodeGen/BackendUtil.cpp:1114:0
#29 0x00007885d8f35dab RunCodegenPipeline /usr/src/debug/clang/clang-19.1.7.src/lib/CodeGen/BackendUtil.cpp:1124:0
#30 0x00007885d8f35dab EmitAssembly /usr/src/debug/clang/clang-19.1.7.src/lib/CodeGen/BackendUtil.cpp:1180:0
#31 0x00007885d8f35dab clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) /usr/src/debug/clang/clang-19.1.7.src/lib/CodeGen/BackendUtil.cpp:1341:0
#32 0x00007885d92ffb7a llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>::release() /usr/include/llvm/ADT/IntrusiveRefCntPtr.h:232:9
#33 0x00007885d92ffb7a llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>::~IntrusiveRefCntPtr() /usr/include/llvm/ADT/IntrusiveRefCntPtr.h:196:34
#34 0x00007885d92ffb7a clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /usr/src/debug/clang/clang-19.1.7.src/lib/CodeGen/CodeGenAction.cpp:354:20
#35 0x00007885d7dd9c86 __gnu_cxx::__normal_iterator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>*, std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>>::__normal_iterator(std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>* const&) /usr/include/c++/14.2.1/bits/stl_iterator.h:1068:0
#36 0x00007885d7dd9c86 std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>::begin() /usr/include/c++/14.2.1/bits/stl_vector.h:874:0
#37 0x00007885d7dd9c86 void clang::finalize<std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>>(std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>&, clang::Sema const&) /usr/src/debug/clang/clang-19.1.7.src/include/clang/Sema/TemplateInstCallback.h:54:0
#38 0x00007885d7dd9c86 clang::ParseAST(clang::Sema&, bool, bool) /usr/src/debug/clang/clang-19.1.7.src/lib/Parse/ParseAST.cpp:191:0
#39 0x00007885d9afa221 clang::FrontendAction::Execute() /usr/src/debug/clang/clang-19.1.7.src/lib/Frontend/FrontendAction.cpp:1078:21
#40 0x00007885d9a9008f llvm::Error::getPtr() const /usr/include/llvm/Support/Error.h:282:12
#41 0x00007885d9a9008f llvm::Error::operator bool() /usr/include/llvm/Support/Error.h:242:22
#42 0x00007885d9a9008f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /usr/src/debug/clang/clang-19.1.7.src/lib/Frontend/CompilerInstance.cpp:1061:42
#43 0x00007885d9b54ba5 std::__shared_ptr<clang::FrontendOptions, (__gnu_cxx::_Lock_policy)2>::get() const /usr/include/c++/14.2.1/bits/shared_ptr_base.h:1667:0
#44 0x00007885d9b54ba5 std::__shared_ptr_access<clang::FrontendOptions, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get() const /usr/include/c++/14.2.1/bits/shared_ptr_base.h:1364:0
#45 0x00007885d9b54ba5 std::__shared_ptr_access<clang::FrontendOptions, (__gnu_cxx::_Lock_policy)2, false, false>::operator*() const /usr/include/c++/14.2.1/bits/shared_ptr_base.h:1350:0
#46 0x00007885d9b54ba5 clang::CompilerInvocation::getFrontendOpts() /usr/src/debug/clang/clang-19.1.7.src/include/clang/Frontend/CompilerInvocation.h:259:0
#47 0x00007885d9b54ba5 clang::CompilerInstance::getFrontendOpts() /usr/src/debug/clang/clang-19.1.7.src/include/clang/Frontend/CompilerInstance.h:312:0
#48 0x00007885d9b54ba5 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /usr/src/debug/clang/clang-19.1.7.src/lib/FrontendTool/ExecuteCompilerInvocation.cpp:281:0
#49 0x00006343124e3832 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /usr/src/debug/clang/clang-19.1.7.src/tools/driver/cc1_main.cpp:284:40
#50 0x00006343124e804e ExecuteCC1Tool /usr/src/debug/clang/clang-19.1.7.src/tools/driver/driver.cpp:215:20
#51 0x00007885d9764bb5 operator() /usr/src/debug/clang/clang-19.1.7.src/lib/Driver/Job.cpp:440:32
#52 0x00007885d9764bb5 callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef> >, std::string*, bool*) const::<lambda()> > /usr/include/llvm/ADT/STLFunctionalExtras.h:45:52
#53 0x00007885cf70176a llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) /usr/src/debug/llvm/llvm-19.1.7.src/lib/Support/CrashRecoveryContext.cpp:428:1
#54 0x00007885d976bb76 clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const /usr/src/debug/clang/clang-19.1.7.src/lib/Driver/Job.cpp:444:10
#55 0x00007885d9722709 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const /usr/src/debug/clang/clang-19.1.7.src/lib/Driver/Compilation.cpp:200:3
#56 0x00007885d9722afc clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const /usr/src/debug/clang/clang-19.1.7.src/lib/Driver/Compilation.cpp:253:5
#57 0x00007885d9792a64 llvm::SmallVectorBase<unsigned int>::empty() const /usr/include/llvm/ADT/SmallVector.h:95:46
#58 0x00007885d9792a64 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) /usr/src/debug/clang/clang-19.1.7.src/lib/Driver/Driver.cpp:1946:28
#59 0x00006343124e9d8c llvm::SmallVectorBase<unsigned int>::size() const /usr/include/llvm/ADT/SmallVector.h:92:32
#60 0x00006343124e9d8c llvm::SmallVectorTemplateCommon<std::pair<int, clang::driver::Command const*>, void>::end() /usr/include/llvm/ADT/SmallVector.h:283:41
#61 0x00006343124e9d8c clang_main(int, char**, llvm::ToolContext const&) /usr/src/debug/clang/clang-19.1.7.src/tools/driver/driver.cpp:393:26
#62 0x00006343124dbf65 main /usr/src/debug/clang/clang-19.1.7.src/build/tools/driver/clang-driver.cpp:18:1
#63 0x00007885cea35488 __libc_start_call_main /usr/src/debug/glibc/glibc/csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#64 0x00007885cea3554c call_init /usr/src/debug/glibc/glibc/csu/../csu/libc-start.c:128:20
#65 0x00007885cea3554c __libc_start_main /usr/src/debug/glibc/glibc/csu/../csu/libc-start.c:347:5
#66 0x00006343124dbfc5 _start (/usr/bin/clang+++0xafc5)
clang++: error: clang frontend command failed with exit code 139 (use -v to see invocation)
clang version 19.1.7
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
clang++: note: diagnostic msg:

nsc800-db67d1.zip