[Windows] Run native unit tests in CI #13923

GregoryComer · 2025-09-03T23:19:38Z

Summary

Build and run C++ tests in the unit test CI job on Windows. Fix a few lingering issues on Windows that surfaced.

Changes:

Add CMake build + CTest execution the Windows unit test job.
Gate Core ML export recipe tests (and coremltools imports) to non-Windows platforms.
Add some explicit cases for size_t 0 literals to make MSVC overload resolution happy.
Remove 2> /dev/null from some CMake PTE export custom commands. I'm not entirely sure why this is there in the first place (keep the log a little cleaner?), but it doesn't work on Windows. Easiest to just remove it.
Update the temp file utility code (in extension/testing_util/temp_file.h) to use standard library file IO (as opposed to posix).
(Critical): Replace long with int64_t in a few portable kernels that deal with indices. It should be int64_t, as this is the correct type for long tensors. On Windows, for example, long is actually 32-bit for historical reasons, and it causes memory corruption / crashes when trying to read/write ScalarType::Long tensors with long* pointers.
Update a bunch of custom commands to use platform-specific syntax or replace with a cross-platform approach.
Use non-strict export for test models. We can support strict export on Windows, but it take a bit of env setup. This is covered in CI in places, but non-strict is the default so it makes it a bit easier to build tests this way.

I've also excluded method_test and tensor_parser_test for now, due to the env issues with some of the PTE test models. The tests pass locally - but I need to debug the build-time PTE creation for these tests. I'm disabling to get the remaining 98% of the tests up first, and will fix these shortly. Tracking in #13883.

Test plan

CI - the tests are covered by the unittest jobs (debug, release, and editable).

pytorch-bot · 2025-09-03T23:19:42Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13923

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 42 Pending

As of commit 04826a7 with merge base 9af908d ():

NEW FAILURE - The following job has failed:

pull / android / run-emulator (gh)
The process '/usr/bin/sh' failed with exit code 255

This comment was automatically generated by Dr. CI and updates every 15 minutes.

JacobSzwejbka · 2025-09-11T17:17:04Z

kernels/portable/cpu/op_argmax.cpp

@@ -49,21 +49,21 @@ Tensor& argmax_out(
  static constexpr const char op_name[] = "argmax.out";

  ET_SWITCH_REALHBF16_TYPES(in.scalar_type(), ctx, op_name, CTYPE, [&] {
-    long* out_data = out.mutable_data_ptr<long>();
+    int64_t* out_data = out.mutable_data_ptr<int64_t>();


Nice. We should have a lint for this cc @manuelcandales

test/models/export_delegated_program.py

runtime/kernel/test/CMakeLists.txt

runtime/core/exec_aten/testing_util/tensor_util.cpp

extension/evalue_util/test/print_evalue_test.cpp

JacobSzwejbka · 2025-09-11T17:22:35Z

export/tests/test_target_recipes.py


    def tearDown(self) -> None:
        super().tearDown()

+    @unittest.skipIf(sys.platform == "win32", "Core ML is not available on Windows.")


This seems like evidence that we shouldnt have a mono recipe test with real backends.

Yeah, might make sense to refactor it as a follow up. CC @abhinaykukkadapu Do you have any preferences on handling of platform-specific recipes?

I was under the impression that lowering works across all platforms, although i haven't thought about windows tbf.

This seems like evidence that we shouldnt have a mono recipe test with real backends.

these tests only call forward if those specific et backends are available, check here: https://github.com/pytorch/executorch/blob/main/export/tests/test_target_recipes.py#L103, otherwise these tests just lowers the model.

@metascroy do you know if coreml tools work on a windows platform?

Thanks. From what I read, coremltools AOT flow is not available on Windows.

export/target_recipes.py

GregoryComer · 2025-09-11T18:48:08Z

@JacobSzwejbka I've made a few small updates to address comments. Do you have any blocking concerns?

ghstack-source-id: 1dca42f ghstack-comment-id: 3251228740 Pull-Request: pytorch#13925

GregoryComer · 2025-09-11T21:05:27Z

After rebase and minor cleanup, I'm seeing one (flaky) test failure relating to etdump generation due to being unable to access a temp file because there's another open handle to it. Tracking this in #13883. I've updated and am re-running CI now. If no additional objections, I hope to land this shortly, pending final approval from reviewers.

This reverts commit f1ca55a.

…rch#14286) This reverts commit 10e93fb.

Differential Revision: D82345075 Pull Request resolved: #14287

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 3, 2025

GregoryComer force-pushed the windows-native-test-ci branch 16 times, most recently from 8028d96 to 43355de Compare September 10, 2025 04:10

GregoryComer added the release notes: windows label Sep 10, 2025

GregoryComer force-pushed the windows-native-test-ci branch 11 times, most recently from 62784a7 to 9364aea Compare September 11, 2025 05:04

GregoryComer requested a review from manuelcandales as a code owner September 11, 2025 08:14