[CD] Windows Wheel builds CUDA 12.9.1 Stack Overflow during build

[atalman]

🐛 Describe the bug

PR: #155748

We are observing errors: LLVM ERROR: out of memory, nvcc error : '""%CICC_PATH%\cicc"' died with status 0xC0000409

6888/7628] Building CUDA object caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduce.cu.obj
FAILED: caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/cuda/SegmentReduce.cu.obj 
C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\randomtemp.exe C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\sccache.exe C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin\nvcc.exe -forward-unknown-to-host-compiler -DAT_PER_OPERATOR_HEADERS -DEXPORT_AOTI_FUNCTIONS -DFMT_HEADER_ONLY=1 -DIDEEP_USE_MKL -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DNOMINMAX -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTORCH_CUDA_BUILD_MAIN_LIB -DTORCH_CUDA_USE_NVTX3 -DUSE_C10D_GLOO -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_EXTERNAL_MZCRC -DUSE_MEM_EFF_ATTENTION -DUSE_MIMALLOC -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_UCRT_LEGACY_INFINITY -Dtorch_cuda_EXPORTS -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build -IC:\actions-runner\_work\pytorch\pytorch\pytorch -IC:\actions-runner\_work\pytorch\pytorch\pytorch\nlohmann -IC:\actions-runner\_work\pytorch\pytorch\pytorch\moodycamel -IC:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\mimalloc\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\THC -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\cuda -IC:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\fmt\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\..\..\..\third_party\cutlass\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\..\..\..\third_party\cutlass\tools\util\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build\caffe2\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\c10\cuda\..\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\c10\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\torch\csrc\api -IC:\actions-runner\_work\pytorch\pytorch\pytorch\torch\csrc\api\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\build\third_party\gloo -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\gloo -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\googletest\googlemock\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\googletest\googletest\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\protobuf\src -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Library\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\XNNPACK\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ittapi\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\eigen -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\include" -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ideep\mkl-dnn\include\oneapi\dnnl -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ideep\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\INTERFACE -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\nlohmann\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\concurrentqueue -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\NVTX\c\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\cudnn_frontend\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\magma_cuda129_release\include -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -Xcompiler  /Zc:__cplusplus -Xcompiler /w -w -Xcompiler /FS -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch --use-local-env -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_100,code=sm_100 -gencode arch=compute_120,code=sm_120 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --Werror cross-execution-space-call --no-host-device-move-forward --expt-relaxed-constexpr --expt-extended-lambda -Xfatbin -compress-all -Xcompiler=/wd4819,/wd4503,/wd4190,/wd4244,/wd4251,/wd4275,/wd4522 -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -Xcompiler="-O2 -Ob2" -DNDEBUG -Xcompiler /MD -std=c++17 -Xcompiler=-MD -Xcompiler=-Z7 -DMKL_HAS_SBGEMM -DMKL_HAS_SHGEMM -DCAFFE2_USE_GLOO -MD -MT caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduce.cu.obj -MF caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduce.cu.obj.d -x cu -c C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduce.cu -o caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduce.cu.obj -Xcompiler=-Fdcaffe2\CMakeFiles\torch_cuda.dir\,-FS
LLVM ERROR: out of memory
SegmentReduce.cu
nvcc error   : '""%CICC_PATH%\cicc"' died with status 0xC0000409 
Retry attempt: 1
LLVM ERROR: out of memory
SegmentReduce.cu
nvcc error   : '""%CICC_PATH%\cicc"' died with status 0xC0000409 
Retry attempt: 2
LLVM ERROR: out of memory
SegmentReduce.cu
nvcc error   : '""%CICC_PATH%\cicc"' died with status 0xC0000409 
Retry attempt: 3
LLVM ERROR: out of memory
SegmentReduce.cu
nvcc error   : '""%CICC_PATH%\cicc"' died with status 0xC0000409 

.....
ninja: build stopped: subcommand failed.
-- Building version 2.8.0.dev20250617+cu129
-- Checkout nccl release tag: v2.27.3-1
cmake -GNinja -DBUILD_ENVIRONMENT=windows-binary-wheel -DBUILD_PYTHON=True -DBUILD_PYTHONLESS= -DBUILD_TEST=True -DBUILD_TYPE=release -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_COMPILER=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.9/bin/nvcc.exe -DCMAKE_CUDA_COMPILER_LAUNCHER=C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\randomtemp.exe;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\sccache.exe -DCMAKE_GENERATOR=Ninja -DCMAKE_INSTALL_PREFIX=C:\actions-runner\_work\pytorch\pytorch\pytorch\torch -DCMAKE_PREFIX_PATH=C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Lib\site-packages;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Library\;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Lib\site-packages\cmake\data\bin;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Scripts;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Scripts;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\libnvvp;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\Amazon\cfn-bootstrap;C:\ProgramData\chocolatey\bin;C:\Program Files\Amazon\AWSCLIV2;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Users\runneruser\AppData\Local\Microsoft\WindowsApps -DCUDA_NVCC_EXECUTABLE=C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\/tmp_bin/nvcc.bat -DINSTALL_TEST=0 -DPython_EXECUTABLE=C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\python.exe -DTORCH_BUILD_VERSION=2.8.0.dev20250617+cu129 -DTORCH_CUDA_ARCH_LIST=7.5;8.0;8.6;9.0;10.0;12.0 -DUSE_FBGEMM=1 -DUSE_GLOO_WITH_OPENSSL=ON -DUSE_GOLD_LINKER=OFF -DUSE_NUMPY=True -DUSE_SCCACHE=1 -DUSE_SPLIT_BUILD= C:\actions-runner\_work\pytorch\pytorch\pytorch
cmake --build . --target install --config Release -j 12

Looks like we are experiencing this issue: https://discuss.pytorch.org/t/segment-reduce-memory-problems-while-building/220488

Versions

2.8.0

cc @malfet @seemethere @peterjc123 @mszhanyi @skyline75489 @nbcsm @iremyux @Blackhex @ptrblck @msaroufim @eqy @jerryzh168

[atalman]

Mitigation tried: Using worker with larger memory footprint r5n.4xlarge, Set MAX_JOBS 12->4

[atalman]

Verbose error log:

2025-06-20T05:54:20.0895622Z [6921/7633] Building CUDA object caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj
2025-06-20T05:54:20.0896525Z FAILED: caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/cuda/SegmentReduceForward.cu.obj 
2025-06-20T05:54:20.0928047Z C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\randomtemp.exe C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\sccache.exe C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin\nvcc.exe -forward-unknown-to-host-compiler -DAT_PER_OPERATOR_HEADERS -DEXPORT_AOTI_FUNCTIONS -DFMT_HEADER_ONLY=1 -DIDEEP_USE_MKL -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DNOMINMAX -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTORCH_CUDA_BUILD_MAIN_LIB -DTORCH_CUDA_USE_NVTX3 -DUSE_C10D_GLOO -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_EXTERNAL_MZCRC -DUSE_MEM_EFF_ATTENTION -DUSE_MIMALLOC -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_UCRT_LEGACY_INFINITY -Dtorch_cuda_EXPORTS -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build -IC:\actions-runner\_work\pytorch\pytorch\pytorch -IC:\actions-runner\_work\pytorch\pytorch\pytorch\nlohmann -IC:\actions-runner\_work\pytorch\pytorch\pytorch\moodycamel -IC:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\mimalloc\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\THC -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\cuda -IC:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\fmt\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\..\..\..\third_party\cutlass\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\..\..\..\third_party\cutlass\tools\util\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build\caffe2\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\c10\cuda\..\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\c10\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\torch\csrc\api -IC:\actions-runner\_work\pytorch\pytorch\pytorch\torch\csrc\api\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\build\third_party\gloo -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\gloo -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\googletest\googlemock\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\googletest\googletest\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\protobuf\src -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Library\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\XNNPACK\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ittapi\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\eigen -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\include" -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ideep\mkl-dnn\include\oneapi\dnnl -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ideep\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\INTERFACE -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\nlohmann\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\concurrentqueue -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\NVTX\c\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\cudnn_frontend\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\magma_cuda129_release\include -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -v -Xcompiler  /Zc:__cplusplus -Xcompiler /w -w -Xcompiler /FS -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch --use-local-env -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_100,code=sm_100 -gencode arch=compute_120,code=sm_120 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --Werror cross-execution-space-call --no-host-device-move-forward --expt-relaxed-constexpr --expt-extended-lambda -Xfatbin -compress-all -v -Xcompiler=/wd4819,/wd4503,/wd4190,/wd4244,/wd4251,/wd4275,/wd4522 -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -Xcompiler="-O2 -Ob2" -DNDEBUG -Xcompiler /MD -std=c++17 -Xcompiler=-MD -Xcompiler=-Z7 -DMKL_HAS_SBGEMM -DMKL_HAS_SHGEMM -DCAFFE2_USE_GLOO -MD -MT caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj -MF caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj.d -x cu -c C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu -o caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj -Xcompiler=-Fdcaffe2\CMakeFiles\torch_cuda.dir\,-FS
2025-06-20T05:54:20.0948969Z #$ _NVVM_BRANCH_=nvvm
2025-06-20T05:54:20.0949201Z #$ _SPACE_= 
2025-06-20T05:54:20.0949415Z #$ _CUDART_=cudart
2025-06-20T05:54:20.0949663Z #$ _HERE_=C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin
2025-06-20T05:54:20.0950012Z #$ _THERE_=C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin
2025-06-20T05:54:20.0950312Z #$ _TARGET_SIZE_=
2025-06-20T05:54:20.0950531Z #$ _TARGET_DIR_=
2025-06-20T05:54:20.0950736Z #$ _TARGET_SIZE_=64
2025-06-20T05:54:20.0950955Z #$ _WIN_PLATFORM_=x64
2025-06-20T05:54:20.0951208Z #$ TOP=C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/..
2025-06-20T05:54:20.0951587Z #$ CICC_PATH=C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../nvvm/bin
2025-06-20T05:54:20.0952078Z #$ NVVMIR_LIBRARY_DIR=C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../nvvm/libdevice
2025-06-20T05:54:20.0960877Z #$ PATH=C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../nvvm/bin;C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin;C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../lib;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.38.33130\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\VC\VCPackages;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\bin\Roslyn;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\;C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\\x64;C:\Program Files (x86)\Windows Kits\10\bin\\x64;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\\MSBuild\Current\Bin\amd64;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\Tools\;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\tmp_bin;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Lib\site-packages\cmake\data\bin;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Scripts;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Scripts;C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\libnvvp;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\Amazon\cfn-bootstrap;C:\ProgramData\chocolatey\bin;C:\Program Files\Amazon\AWSCLIV2;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Users\runneruser\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\VC\Linux\bin\ConnectionManagerExe
2025-06-20T05:54:20.0969665Z #$ INCLUDES="-IC:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../include"  
2025-06-20T05:54:20.0970218Z #$ LIBRARIES=  "/LIBPATH:C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../lib/x64"
2025-06-20T05:54:20.0970614Z #$ CUDAFE_FLAGS=
2025-06-20T05:54:20.0970833Z #$ PTXAS_FLAGS=
2025-06-20T05:54:20.0971057Z SegmentReduceForward.cu
2025-06-20T05:54:20.0989701Z #$ resource file C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-13.res: [-D__CUDA_ARCH__=800 -D__CUDA_ARCH_LIST__=800,860,900,1000,1200 -D__NV_LEGACY_LAUNCH -nologo -E -TP  -DCUDA_DOUBLE_MATH_FUNCTIONS -EHsc -D__CUDACC__ -D__NVCC__ -D__CUDACC_EXTENDED_LAMBDA__ -D__CUDACC_RELAXED_CONSTEXPR__  /Zc:__cplusplus /w /FS /wd4819 /wd4503 /wd4190 /wd4244 /wd4251 /wd4275 /wd4522 -O2 -Ob2 /MD -MD -Z7 -Fdcaffe2\CMakeFiles\torch_cuda.dir\ -FS -w -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/nlohmann" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/moodycamel" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/mimalloc/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/THC" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/cuda" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/fmt/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/tools/util/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/caffe2/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/cuda/../.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api/include" "-IC:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../include"   -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googlemock/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googletest/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/protobuf/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/Python/Library/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/XNNPACK/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ittapi/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/eigen" -I"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.9/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/mkl-dnn/include/oneapi/dnnl" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/INTERFACE" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/nlohmann/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/concurrentqueue" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/NVTX/c/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/cudnn_frontend/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/magma_cuda129_release/include"  -D "AT_PER_OPERATOR_HEADERS" -D "EXPORT_AOTI_FUNCTIONS" -D "FMT_HEADER_ONLY=1" -D "IDEEP_USE_MKL" -D "MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS" -D "NOMINMAX" -D "ONNXIFI_ENABLE_EXT=1" -D "ONNX_ML=1" -D "ONNX_NAMESPACE=onnx_torch" -D "TORCH_CUDA_BUILD_MAIN_LIB" -D "TORCH_CUDA_USE_NVTX3" -D "USE_C10D_GLOO" -D "USE_CUDA" -D "USE_DISTRIBUTED" -D "USE_EXTERNAL_MZCRC" -D "USE_MEM_EFF_ATTENTION" -D "USE_MIMALLOC" -D "WIN32_LEAN_AND_MEAN" -D "_CRT_SECURE_NO_DEPRECATE=1" -D "_UCRT_LEGACY_INFINITY" -D "torch_cuda_EXPORTS" -D "LIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS" -D "ONNX_NAMESPACE=onnx_torch" -D "CUB_WRAPPED_NAMESPACE=at_cuda_detail" -D "CUDA_HAS_FP16=1" -D "__CUDA_NO_HALF_OPERATORS__" -D "__CUDA_NO_HALF_CONVERSIONS__" -D "__CUDA_NO_HALF2_OPERATORS__" -D "__CUDA_NO_BFLOAT16_CONVERSIONS__" -D "NDEBUG" -D "MKL_HAS_SBGEMM" -D "MKL_HAS_SHGEMM" -D "CAFFE2_USE_GLOO" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=9 -D__CUDACC_VER_BUILD__=86 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=9 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -D__CUDACC_DEVICE_ATOMIC_BUILTINS__=1 -FI "cuda_runtime.h" "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" ]
2025-06-20T05:54:20.1009255Z #$ cl.exe /std:c++17 @"C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-13.res" > "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-12_SegmentReduceForward.compute_80.cpp1.ii" 
2025-06-20T05:54:20.1010440Z SegmentReduceForward.cu
2025-06-20T05:54:20.1010944Z #$ erase C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-13.res
2025-06-20T05:54:20.1030028Z #$ resource file C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-15.res: [-D__CUDA_ARCH__=860 -D__CUDA_ARCH_LIST__=800,860,900,1000,1200 -D__NV_LEGACY_LAUNCH -nologo -E -TP  -DCUDA_DOUBLE_MATH_FUNCTIONS -EHsc -D__CUDACC__ -D__NVCC__ -D__CUDACC_EXTENDED_LAMBDA__ -D__CUDACC_RELAXED_CONSTEXPR__  /Zc:__cplusplus /w /FS /wd4819 /wd4503 /wd4190 /wd4244 /wd4251 /wd4275 /wd4522 -O2 -Ob2 /MD -MD -Z7 -Fdcaffe2\CMakeFiles\torch_cuda.dir\ -FS -w -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/nlohmann" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/moodycamel" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/mimalloc/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/THC" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/cuda" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/fmt/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/tools/util/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/caffe2/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/cuda/../.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api/include" "-IC:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../include"   -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googlemock/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googletest/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/protobuf/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/Python/Library/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/XNNPACK/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ittapi/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/eigen" -I"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.9/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/mkl-dnn/include/oneapi/dnnl" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/INTERFACE" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/nlohmann/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/concurrentqueue" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/NVTX/c/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/cudnn_frontend/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/magma_cuda129_release/include"  -D "AT_PER_OPERATOR_HEADERS" -D "EXPORT_AOTI_FUNCTIONS" -D "FMT_HEADER_ONLY=1" -D "IDEEP_USE_MKL" -D "MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS" -D "NOMINMAX" -D "ONNXIFI_ENABLE_EXT=1" -D "ONNX_ML=1" -D "ONNX_NAMESPACE=onnx_torch" -D "TORCH_CUDA_BUILD_MAIN_LIB" -D "TORCH_CUDA_USE_NVTX3" -D "USE_C10D_GLOO" -D "USE_CUDA" -D "USE_DISTRIBUTED" -D "USE_EXTERNAL_MZCRC" -D "USE_MEM_EFF_ATTENTION" -D "USE_MIMALLOC" -D "WIN32_LEAN_AND_MEAN" -D "_CRT_SECURE_NO_DEPRECATE=1" -D "_UCRT_LEGACY_INFINITY" -D "torch_cuda_EXPORTS" -D "LIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS" -D "ONNX_NAMESPACE=onnx_torch" -D "CUB_WRAPPED_NAMESPACE=at_cuda_detail" -D "CUDA_HAS_FP16=1" -D "__CUDA_NO_HALF_OPERATORS__" -D "__CUDA_NO_HALF_CONVERSIONS__" -D "__CUDA_NO_HALF2_OPERATORS__" -D "__CUDA_NO_BFLOAT16_CONVERSIONS__" -D "NDEBUG" -D "MKL_HAS_SBGEMM" -D "MKL_HAS_SHGEMM" -D "CAFFE2_USE_GLOO" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=9 -D__CUDACC_VER_BUILD__=86 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=9 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -D__CUDACC_DEVICE_ATOMIC_BUILTINS__=1 -FI "cuda_runtime.h" "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" ]
2025-06-20T05:54:20.1049556Z #$ cl.exe /std:c++17 @"C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-15.res" > "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-14_SegmentReduceForward.compute_86.cpp1.ii" 
2025-06-20T05:54:20.1050740Z SegmentReduceForward.cu
2025-06-20T05:54:20.1051250Z #$ erase C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-15.res
2025-06-20T05:54:20.1070240Z #$ resource file C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-17.res: [-D__CUDA_ARCH__=900 -D__CUDA_ARCH_LIST__=800,860,900,1000,1200 -D__NV_LEGACY_LAUNCH -nologo -E -TP  -DCUDA_DOUBLE_MATH_FUNCTIONS -EHsc -D__CUDACC__ -D__NVCC__ -D__CUDACC_EXTENDED_LAMBDA__ -D__CUDACC_RELAXED_CONSTEXPR__  /Zc:__cplusplus /w /FS /wd4819 /wd4503 /wd4190 /wd4244 /wd4251 /wd4275 /wd4522 -O2 -Ob2 /MD -MD -Z7 -Fdcaffe2\CMakeFiles\torch_cuda.dir\ -FS -w -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/nlohmann" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/moodycamel" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/mimalloc/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/THC" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/cuda" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/fmt/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/tools/util/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/caffe2/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/cuda/../.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api/include" "-IC:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../include"   -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googlemock/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googletest/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/protobuf/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/Python/Library/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/XNNPACK/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ittapi/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/eigen" -I"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.9/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/mkl-dnn/include/oneapi/dnnl" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/INTERFACE" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/nlohmann/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/concurrentqueue" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/NVTX/c/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/cudnn_frontend/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/magma_cuda129_release/include"  -D "AT_PER_OPERATOR_HEADERS" -D "EXPORT_AOTI_FUNCTIONS" -D "FMT_HEADER_ONLY=1" -D "IDEEP_USE_MKL" -D "MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS" -D "NOMINMAX" -D "ONNXIFI_ENABLE_EXT=1" -D "ONNX_ML=1" -D "ONNX_NAMESPACE=onnx_torch" -D "TORCH_CUDA_BUILD_MAIN_LIB" -D "TORCH_CUDA_USE_NVTX3" -D "USE_C10D_GLOO" -D "USE_CUDA" -D "USE_DISTRIBUTED" -D "USE_EXTERNAL_MZCRC" -D "USE_MEM_EFF_ATTENTION" -D "USE_MIMALLOC" -D "WIN32_LEAN_AND_MEAN" -D "_CRT_SECURE_NO_DEPRECATE=1" -D "_UCRT_LEGACY_INFINITY" -D "torch_cuda_EXPORTS" -D "LIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS" -D "ONNX_NAMESPACE=onnx_torch" -D "CUB_WRAPPED_NAMESPACE=at_cuda_detail" -D "CUDA_HAS_FP16=1" -D "__CUDA_NO_HALF_OPERATORS__" -D "__CUDA_NO_HALF_CONVERSIONS__" -D "__CUDA_NO_HALF2_OPERATORS__" -D "__CUDA_NO_BFLOAT16_CONVERSIONS__" -D "NDEBUG" -D "MKL_HAS_SBGEMM" -D "MKL_HAS_SHGEMM" -D "CAFFE2_USE_GLOO" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=9 -D__CUDACC_VER_BUILD__=86 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=9 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -D__CUDACC_DEVICE_ATOMIC_BUILTINS__=1 -FI "cuda_runtime.h" "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" ]
2025-06-20T05:54:20.1089658Z #$ cl.exe /std:c++17 @"C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-17.res" > "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-16_SegmentReduceForward.compute_90.cpp1.ii" 
2025-06-20T05:54:20.1090840Z SegmentReduceForward.cu
2025-06-20T05:54:20.1091344Z #$ erase C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-17.res
2025-06-20T05:54:20.1110260Z #$ resource file C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-19.res: [-D__CUDA_ARCH__=1000 -D__CUDA_ARCH_LIST__=800,860,900,1000,1200 -D__NV_LEGACY_LAUNCH -nologo -E -TP  -DCUDA_DOUBLE_MATH_FUNCTIONS -EHsc -D__CUDACC__ -D__NVCC__ -D__CUDACC_EXTENDED_LAMBDA__ -D__CUDACC_RELAXED_CONSTEXPR__  /Zc:__cplusplus /w /FS /wd4819 /wd4503 /wd4190 /wd4244 /wd4251 /wd4275 /wd4522 -O2 -Ob2 /MD -MD -Z7 -Fdcaffe2\CMakeFiles\torch_cuda.dir\ -FS -w -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/nlohmann" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/moodycamel" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/mimalloc/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/THC" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/cuda" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/fmt/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/tools/util/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/caffe2/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/cuda/../.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api/include" "-IC:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../include"   -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googlemock/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googletest/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/protobuf/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/Python/Library/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/XNNPACK/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ittapi/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/eigen" -I"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.9/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/mkl-dnn/include/oneapi/dnnl" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/INTERFACE" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/nlohmann/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/concurrentqueue" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/NVTX/c/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/cudnn_frontend/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/magma_cuda129_release/include"  -D "AT_PER_OPERATOR_HEADERS" -D "EXPORT_AOTI_FUNCTIONS" -D "FMT_HEADER_ONLY=1" -D "IDEEP_USE_MKL" -D "MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS" -D "NOMINMAX" -D "ONNXIFI_ENABLE_EXT=1" -D "ONNX_ML=1" -D "ONNX_NAMESPACE=onnx_torch" -D "TORCH_CUDA_BUILD_MAIN_LIB" -D "TORCH_CUDA_USE_NVTX3" -D "USE_C10D_GLOO" -D "USE_CUDA" -D "USE_DISTRIBUTED" -D "USE_EXTERNAL_MZCRC" -D "USE_MEM_EFF_ATTENTION" -D "USE_MIMALLOC" -D "WIN32_LEAN_AND_MEAN" -D "_CRT_SECURE_NO_DEPRECATE=1" -D "_UCRT_LEGACY_INFINITY" -D "torch_cuda_EXPORTS" -D "LIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS" -D "ONNX_NAMESPACE=onnx_torch" -D "CUB_WRAPPED_NAMESPACE=at_cuda_detail" -D "CUDA_HAS_FP16=1" -D "__CUDA_NO_HALF_OPERATORS__" -D "__CUDA_NO_HALF_CONVERSIONS__" -D "__CUDA_NO_HALF2_OPERATORS__" -D "__CUDA_NO_BFLOAT16_CONVERSIONS__" -D "NDEBUG" -D "MKL_HAS_SBGEMM" -D "MKL_HAS_SHGEMM" -D "CAFFE2_USE_GLOO" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=9 -D__CUDACC_VER_BUILD__=86 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=9 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -D__CUDACC_DEVICE_ATOMIC_BUILTINS__=1 -FI "cuda_runtime.h" "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" ]
2025-06-20T05:54:20.1130128Z #$ cl.exe /std:c++17 @"C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-19.res" > "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-18_SegmentReduceForward.compute_100.cpp1.ii" 
2025-06-20T05:54:20.1131320Z SegmentReduceForward.cu
2025-06-20T05:54:20.1131841Z #$ erase C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-19.res
2025-06-20T05:54:20.1151134Z #$ resource file C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-21.res: [-D__CUDA_ARCH__=1200 -D__CUDA_ARCH_LIST__=800,860,900,1000,1200 -D__NV_LEGACY_LAUNCH -nologo -E -TP  -DCUDA_DOUBLE_MATH_FUNCTIONS -EHsc -D__CUDACC__ -D__NVCC__ -D__CUDACC_EXTENDED_LAMBDA__ -D__CUDACC_RELAXED_CONSTEXPR__  /Zc:__cplusplus /w /FS /wd4819 /wd4503 /wd4190 /wd4244 /wd4251 /wd4275 /wd4522 -O2 -Ob2 /MD -MD -Z7 -Fdcaffe2\CMakeFiles\torch_cuda.dir\ -FS -w -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/nlohmann" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/moodycamel" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/mimalloc/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/THC" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/cuda" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/fmt/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/tools/util/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/caffe2/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/cuda/../.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api/include" "-IC:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../include"   -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googlemock/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googletest/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/protobuf/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/Python/Library/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/XNNPACK/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ittapi/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/eigen" -I"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.9/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/mkl-dnn/include/oneapi/dnnl" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/INTERFACE" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/nlohmann/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/concurrentqueue" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/NVTX/c/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/cudnn_frontend/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/magma_cuda129_release/include"  -D "AT_PER_OPERATOR_HEADERS" -D "EXPORT_AOTI_FUNCTIONS" -D "FMT_HEADER_ONLY=1" -D "IDEEP_USE_MKL" -D "MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS" -D "NOMINMAX" -D "ONNXIFI_ENABLE_EXT=1" -D "ONNX_ML=1" -D "ONNX_NAMESPACE=onnx_torch" -D "TORCH_CUDA_BUILD_MAIN_LIB" -D "TORCH_CUDA_USE_NVTX3" -D "USE_C10D_GLOO" -D "USE_CUDA" -D "USE_DISTRIBUTED" -D "USE_EXTERNAL_MZCRC" -D "USE_MEM_EFF_ATTENTION" -D "USE_MIMALLOC" -D "WIN32_LEAN_AND_MEAN" -D "_CRT_SECURE_NO_DEPRECATE=1" -D "_UCRT_LEGACY_INFINITY" -D "torch_cuda_EXPORTS" -D "LIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS" -D "ONNX_NAMESPACE=onnx_torch" -D "CUB_WRAPPED_NAMESPACE=at_cuda_detail" -D "CUDA_HAS_FP16=1" -D "__CUDA_NO_HALF_OPERATORS__" -D "__CUDA_NO_HALF_CONVERSIONS__" -D "__CUDA_NO_HALF2_OPERATORS__" -D "__CUDA_NO_BFLOAT16_CONVERSIONS__" -D "NDEBUG" -D "MKL_HAS_SBGEMM" -D "MKL_HAS_SHGEMM" -D "CAFFE2_USE_GLOO" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=9 -D__CUDACC_VER_BUILD__=86 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=9 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -D__CUDACC_DEVICE_ATOMIC_BUILTINS__=1 -FI "cuda_runtime.h" "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" ]
2025-06-20T05:54:20.1171845Z #$ cl.exe /std:c++17 @"C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-21.res" > "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-20_SegmentReduceForward.compute_120.cpp1.ii" 
2025-06-20T05:54:20.1173677Z SegmentReduceForward.cu
2025-06-20T05:54:20.1174197Z #$ erase C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-21.res
2025-06-20T05:54:20.1193310Z #$ resource file C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-22.res: [-D__CUDA_ARCH_LIST__=800,860,900,1000,1200 -D__NV_LEGACY_LAUNCH -nologo -E -TP -EHsc -D__CUDACC__ -D__NVCC__ -D__CUDACC_EXTENDED_LAMBDA__ -D__CUDACC_RELAXED_CONSTEXPR__  /Zc:__cplusplus /w /FS /wd4819 /wd4503 /wd4190 /wd4244 /wd4251 /wd4275 /wd4522 -O2 -Ob2 /MD -MD -Z7 -Fdcaffe2\CMakeFiles\torch_cuda.dir\ -FS -w -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/nlohmann" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/moodycamel" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/mimalloc/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/THC" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/cuda" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/fmt/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/../../../third_party/cutlass/tools/util/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/caffe2/aten/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/cuda/../.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/c10/.." -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/torch/csrc/api/include" "-IC:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin/../include"   -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/build/third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/gloo" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googlemock/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/googletest/googletest/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/protobuf/src" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/Python/Library/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/XNNPACK/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ittapi/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/eigen" -I"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.9/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/mkl-dnn/include/oneapi/dnnl" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/ideep/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/INTERFACE" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/nlohmann/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/concurrentqueue" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/NVTX/c/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/cmake/../third_party/cudnn_frontend/include" -I"C:/actions-runner/_work/pytorch/pytorch/pytorch/.ci/pytorch/windows/magma_cuda129_release/include"  -D "AT_PER_OPERATOR_HEADERS" -D "EXPORT_AOTI_FUNCTIONS" -D "FMT_HEADER_ONLY=1" -D "IDEEP_USE_MKL" -D "MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS" -D "NOMINMAX" -D "ONNXIFI_ENABLE_EXT=1" -D "ONNX_ML=1" -D "ONNX_NAMESPACE=onnx_torch" -D "TORCH_CUDA_BUILD_MAIN_LIB" -D "TORCH_CUDA_USE_NVTX3" -D "USE_C10D_GLOO" -D "USE_CUDA" -D "USE_DISTRIBUTED" -D "USE_EXTERNAL_MZCRC" -D "USE_MEM_EFF_ATTENTION" -D "USE_MIMALLOC" -D "WIN32_LEAN_AND_MEAN" -D "_CRT_SECURE_NO_DEPRECATE=1" -D "_UCRT_LEGACY_INFINITY" -D "torch_cuda_EXPORTS" -D "LIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS" -D "ONNX_NAMESPACE=onnx_torch" -D "CUB_WRAPPED_NAMESPACE=at_cuda_detail" -D "CUDA_HAS_FP16=1" -D "__CUDA_NO_HALF_OPERATORS__" -D "__CUDA_NO_HALF_CONVERSIONS__" -D "__CUDA_NO_HALF2_OPERATORS__" -D "__CUDA_NO_BFLOAT16_CONVERSIONS__" -D "NDEBUG" -D "MKL_HAS_SBGEMM" -D "MKL_HAS_SHGEMM" -D "CAFFE2_USE_GLOO" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=9 -D__CUDACC_VER_BUILD__=86 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=9 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -D__CUDACC_DEVICE_ATOMIC_BUILTINS__=1 -FI "cuda_runtime.h" "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" ]
2025-06-20T05:54:20.1214074Z #$ cl.exe /std:c++17 @"C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-22.res" > "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-6_SegmentReduceForward.cpp4.ii" 
2025-06-20T05:54:20.1215309Z SegmentReduceForward.cu
2025-06-20T05:54:20.1215876Z #$ erase C:\actions-runner\_work\pytorch\pytorch\pytorch\build\.tmpD8a8rJ/tmpxft_00001a24_00000000-22.res
2025-06-20T05:54:20.1216805Z #$ -- Filter Dependencies -- > caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj.d
2025-06-20T05:54:20.1224214Z #$ cudafe++ --ms_c++17 --microsoft_version=1938 --msvc_target_version=1938 --compiler_bindir "C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/HostX64/x64/../../../../../../.." --sdk_dir "C:/Program Files (x86)/Windows Kits/10/" --diag_error h_hd_illegal_call --diag_error h_hd_illegal_call_generic -w --display_error_number --orig_src_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/native/cuda/SegmentReduceForward.cu" --orig_src_path_name "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" --allow_managed --extended-lambda --no_host_device_move_forward --relaxed_constexpr --diag_suppress=cc_clobber_ignored --diag_suppress=field_without_dll_interface --diag_suppress=base_class_has_different_dll_interface --diag_suppress=dll_interface_conflict_none_assumed --diag_suppress=dll_interface_conflict_dllexport_assumed --diag_suppress=bad_friend_decl --m64 --parse_templates --gen_c_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-7_SegmentReduceForward.compute_120.cudafe1.cpp" --stub_file_name "tmpxft_00001a24_00000000-7_SegmentReduceForward.compute_120.cudafe1.stub.c" --gen_module_id_file --module_id_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-5_SegmentReduceForward.module_id" "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-6_SegmentReduceForward.cpp4.ii" 
2025-06-20T05:54:20.1240356Z #$ ""%CICC_PATH%\cicc" --ms_c++17 --microsoft_version=1938 --msvc_target_version=1938 --compiler_bindir "C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/HostX64/x64/../../../../../../.." --sdk_dir "C:/Program Files (x86)/Windows Kits/10/" --diag_error h_hd_illegal_call --diag_error h_hd_illegal_call_generic -w --display_error_number --orig_src_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/aten/src/ATen/native/cuda/SegmentReduceForward.cu" --orig_src_path_name "C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu" --allow_managed --extended-lambda --no_host_device_move_forward --relaxed_constexpr --diag_suppress=cc_clobber_ignored --diag_suppress=field_without_dll_interface --diag_suppress=base_class_has_different_dll_interface --diag_suppress=dll_interface_conflict_none_assumed --diag_suppress=dll_interface_conflict_dllexport_assumed --diag_suppress=bad_friend_decl  -arch compute_80 -m64 --no-version-ident -ftz=0 -prec_div=1 -prec_sqrt=1 -fmad=1 --include_file_name "tmpxft_00001a24_00000000-4_SegmentReduceForward.fatbin.c" -tused --module_id_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-5_SegmentReduceForward.module_id" --gen_c_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-11_SegmentReduceForward.compute_80.cudafe1.c" --stub_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-11_SegmentReduceForward.compute_80.cudafe1.stub.c" --gen_device_file_name "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-11_SegmentReduceForward.compute_80.cudafe1.gpu"  "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-12_SegmentReduceForward.compute_80.cpp1.ii" -o "C:/actions-runner/_work/pytorch/pytorch/pytorch/build/.tmpD8a8rJ/tmpxft_00001a24_00000000-11_SegmentReduceForward.compute_80.ptx""
2025-06-20T05:54:20.1250272Z LLVM ERROR: out of memory
2025-06-20T05:54:20.1250671Z nvcc error   : '""%CICC_PATH%\cicc"' died with status 0xC0000409 
2025-06-20T05:54:20.1251062Z # --error 0xc0000409 --
2025-06-20T05:54:20.1251347Z Retry attempt: 1

Nvcc command that failed:

2025-06-20T02:02:12.9277849Z FAILED: caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/cuda/SegmentReduceForward.cu.obj 
2025-06-20T02:02:12.9328475Z C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\randomtemp.exe C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\\tmp_bin\sccache.exe C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin\nvcc.exe -forward-unknown-to-host-compiler -DAT_PER_OPERATOR_HEADERS -DEXPORT_AOTI_FUNCTIONS -DFMT_HEADER_ONLY=1 -DIDEEP_USE_MKL -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DNOMINMAX -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTORCH_CUDA_BUILD_MAIN_LIB -DTORCH_CUDA_USE_NVTX3 -DUSE_C10D_GLOO -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_EXTERNAL_MZCRC -DUSE_MEM_EFF_ATTENTION -DUSE_MIMALLOC -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_UCRT_LEGACY_INFINITY -Dtorch_cuda_EXPORTS -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build -IC:\actions-runner\_work\pytorch\pytorch\pytorch -IC:\actions-runner\_work\pytorch\pytorch\pytorch\nlohmann -IC:\actions-runner\_work\pytorch\pytorch\pytorch\moodycamel -IC:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\mimalloc\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\THC -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\cuda -IC:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\fmt\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\..\..\..\third_party\cutlass\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\..\..\..\third_party\cutlass\tools\util\include -IC:\actions-runner\_work\pytorch\pytorch\pytorch\build\caffe2\aten\src -IC:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\c10\cuda\..\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\c10\.. -IC:\actions-runner\_work\pytorch\pytorch\pytorch\torch\csrc\api -IC:\actions-runner\_work\pytorch\pytorch\pytorch\torch\csrc\api\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\build\third_party\gloo -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\gloo -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\googletest\googlemock\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\googletest\googletest\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\protobuf\src -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\Python\Library\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\XNNPACK\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ittapi\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\eigen -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\include" -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ideep\mkl-dnn\include\oneapi\dnnl -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\ideep\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\INTERFACE -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\nlohmann\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\concurrentqueue -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\third_party\NVTX\c\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\cmake\..\third_party\cudnn_frontend\include -isystem C:\actions-runner\_work\pytorch\pytorch\pytorch\.ci\pytorch\windows\magma_cuda129_release\include -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -v -Xcompiler  /Zc:__cplusplus -Xcompiler /w -w -Xcompiler /FS -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch --use-local-env -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_100,code=sm_100 -gencode arch=compute_120,code=sm_120 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --Werror cross-execution-space-call --no-host-device-move-forward --expt-relaxed-constexpr --expt-extended-lambda -Xfatbin -compress-all -Xcompiler=/wd4819,/wd4503,/wd4190,/wd4244,/wd4251,/wd4275,/wd4522 -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -Xcompiler="-O2 -Ob2" -DNDEBUG -Xcompiler /MD -std=c++17 -Xcompiler=-MD -Xcompiler=-Z7 -DMKL_HAS_SBGEMM -DMKL_HAS_SHGEMM -DCAFFE2_USE_GLOO -MD -MT caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj -MF caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj.d -x cu -c C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\SegmentReduceForward.cu -o caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduceForward.cu.obj -Xcompiler=-Fdcaffe2\CMakeFiles\torch_cuda.dir\,-FS

[SoShiny]

In case it helps in tracking it down: I encountered the same issue when attempting to build the 2.7.1 release of troch using CUDA 12.9.1 on our Windows build server. I used MAX_JOBS=2 and ran into the same LLVM ERROR: out of memory in SegmentReduce.cu. The server has 256 GB of RAM. As host compiler I used MSVC build tools 17.10.