[windows] add a check for a matching python architecture #1942

charles-zablit · 2025-06-27T14:22:43Z

On Windows, if there is a mismatch between the architecture of the installed Python and the architecture of the toolchain, swift repl will crash immediately with 0xC000007B. This patch adds a check to warn the user of the architecture mismatch right before invoking the REPL, which will crash.

We should also look into adding a warning at the end of the Swift GUI installer.

charles-zablit · 2025-06-27T14:22:55Z

@swift-ci please test

charles-zablit · 2025-06-27T18:23:17Z

@swift-ci please test

adrian-prantl · 2025-06-27T18:27:22Z

Sources/SwiftDriver/Driver/Driver.swift

@@ -1434,6 +1438,42 @@ extension Driver {

    return (.subcommand(subcommand), updatedArgs)
  }
+
+  private static func checkIfMatchingPythonArch() {


Can you add a comment here explaining that this exists because users who install the x86 toolchain on arm64 windows will get an extremely cryptic error message otherwise?

Completely forgot to add it after our discussion. Fixed 👍

adrian-prantl · 2025-06-27T18:28:26Z

This is going to be huge usability improvement. I'd like @artemcm to sign off though due the unusual nature of the patch.

charles-zablit · 2025-06-27T22:50:48Z

@swift-ci please test

artemcm · 2025-06-27T23:04:15Z

@swift-ci please test Windows platform

artemcm

Is it possible to find a way to do this without executing Python code? e.g. can we locate the python.exe executable and do something equivalent to what file does on UNIX?

artemcm · 2025-06-27T23:10:34Z

Sources/SwiftDriver/Driver/Driver.swift

+  /// of the Python installation.
+  ///
+  /// When installing the x86 toolchain on ARM64 Windows, if the user does not
+  /// install an x86 version of Python, they will get acryptic error message


Suggested change

/// install an x86 version of Python, they will get acryptic error message

/// install an x86 version of Python, they will get a cryptic error message

It is quite non-cryptic IMO. 0xC000007B is a NTSTATUS; 0xC => Kernel; 0x7B = 123 = ERROR_INVALID_NAME. This is the same behaviour as Linux when the loader is not found.

I meant cryptic from the point of view of the user. They only get this error message with no indication of what is failing and how to fix it (see the screenshot below). I will change the wording in the comment 👍

swift repl crashes immediately on Windows ARM64 swift#79981

artemcm · 2025-06-27T23:14:36Z

Sources/SwiftDriver/Driver/Driver.swift

+          return;
+        }
+        if !output.lowercased().contains(arch) {
+          stderrStream.send("There is an architecture mismatch between the installed toolchain and the resolved Python's architecture:\n")


Could you please use the driver's diagnosticEngine and define a new .warning for this message instead of using the stderr stream directly?

Switched to using the diagnosticsEngine.

artemcm · 2025-06-27T23:22:06Z

Sources/SwiftDriver/Driver/Driver.swift

+    #else
+    return;
+    #endif
+    let process = Process()


By the time we are in invocationRunMode we have an instance of the DriverExecutor, so we should plumb that here and use it to run this subprocess instead of using Process APIs directly. You can see an example of this being done in xcrunFind.

Switched to using DriverExecutor.

compnerd · 2025-06-27T23:31:19Z

Sources/SwiftDriver/Driver/Driver.swift

@@ -1425,6 +1426,9 @@ extension Driver {
    if firstArg == "repl" {
        updatedArgs.remove(at: 1)
        updatedArgs.append("-repl")
+        #if os(Windows)
+        checkIfMatchingPythonArch()
+        #endif


Does this problem not exist on Darwin as well? If you run arch -m x86_64 swift repl that should fail similarly.

I was not able to get the problem to happen on Darwin. arch -x86_64 swift repl starts the REPL correctly.

Maybe we should bring this check to other platforms as well. I focused on Windows in this patch because all the reports I received were from Windows users on ARM64 machines.

compnerd · 2025-06-27T23:32:39Z

Sources/SwiftDriver/Driver/Driver.swift

+  /// of the Python installation.
+  ///
+  /// When installing the x86 toolchain on ARM64 Windows, if the user does not
+  /// install an x86 version of Python, they will get acryptic error message


It is quite non-cryptic IMO. 0xC000007B is a NTSTATUS; 0xC => Kernel; 0x7B = 123 = ERROR_INVALID_NAME. This is the same behaviour as Linux when the loader is not found.

compnerd · 2025-06-27T23:34:08Z

Sources/SwiftDriver/Driver/Driver.swift

+    #endif
+    let process = Process()
+    process.executableURL = URL(fileURLWithPath: "C:\\Windows\\System32\\cmd.exe")
+    process.arguments = ["/c", "python.exe", "-c", "import platform; print(platform.machine())"]


This seems like a pretty bad way to check if the DLL matches. Are we checking if python matches the architecture or if the current runtime as indicated by PYTHONHOME matchess?

We are checking that the Python that is resolved from the Path matches the architecture. I have tried some combinations of configurations:

Everything below was run on an ARM64 Windows VM with the x86 toolchain installed.

Python Path pointing to PYTHONHOME pointing to Output

ARM64 Nothing Crash with warning

ARM64 ARM64 Crash with warning

ARM64 AMD64 Crash with warning

AMD64 Nothing Launches fine

AMD64 ARM64 Fatal Python error: init_fs_encoding

AMD64 AMD64 Launches fine

I think that checking if the Python that is in the Path is the correct one is enough. If there is a mismatch between the Python that is in the Path and the one that is in PYTHONHOME, that is another problem and there are enough information printed to the user to help them figure it out.

charles-zablit · 2025-06-30T14:40:04Z

Is it possible to find a way to do this without executing Python code? e.g. can we locate the python.exe executable and do something equivalent to what file does on UNIX?

There is no non third party alternative to file on Windows. There is a way to check programmatically that python.exe is the correct architecture according to this SO thread. I'm happy to try to implement this in Swift, but I think it will add some complexity to the code compared to just calling Python?

artemcm · 2025-06-30T17:22:16Z

Is it possible to find a way to do this without executing Python code? e.g. can we locate the python.exe executable and do something equivalent to what file does on UNIX?

There is no non third party alternative to file on Windows. There is a way to check programmatically that python.exe is the correct architecture according to this SO thread. I'm happy to try to implement this in Swift, but I think it will add some complexity to the code compared to just calling Python?

If we can reduce the set of dependencies the driver needs in order to perform all the tasks it needs to then we should. Even though swift repl does depend on having Python on the system, it would be really nice to avoid the driver having to - especially by firing up the interpreter and executing code.

Furthermore, as-is, when executing this check in the driver on a system where Python is missing, the user will still get this warning with some surprising pythonArchitecture contents printed.

I know there is Dumpbin.exe that is present on the system when Windows SDK is installed, which can act similarly to file, but it also doesn't seem all that bad to try and read the file machine type from the COFF header - that way we can ensure the binary exists in the first place and emit an appropriate error in that case too.

artemcm · 2025-06-30T19:57:35Z

To add to the above comment, the driver today on most code paths plans the build without launching any subprocesses at all, which is crucial in some environments where the driver is used as a library in an environment where it is not allowed to launch processes itself.

While I'm not aware of this being a hard requirement on Windows today, it's still worthwhile to avoid adding new process launch sites when we can.

charles-zablit · 2025-07-01T11:49:50Z

@swift-ci please test

charles-zablit · 2025-07-01T11:51:27Z

The COFF header approach seems to work fine 👍

I am unsure what's the best way to resolve the path to python. My current approach manually searches for it with lookupExecutablePath to avoid executing a subprocess.
However, we could use where.exe to achieve the same thing and use the actual resolution mechanism from Windows.

compnerd · 2025-07-01T14:56:07Z

The COFF header approach seems to work fine 👍

I am unsure what's the best way to resolve the path to python. My current approach manually searches for it with lookupExecutablePath to avoid executing a subprocess.

However, we could use where.exe to achieve the same thing and use the actual resolution mechanism from Windows.

Looking up in Path is the right thing to do - if it is not in Path, it won't be found anyway.

artemcm

I have a couple of small nits, but otherwise this looks good, thank you!

Sources/SwiftDriver/Driver/Driver.swift

Package.resolved

Sources/SwiftDriver/Driver/Driver.swift

charles-zablit · 2025-07-01T17:00:13Z

I have a couple of small nits, but otherwise this looks good, thank you!

Thanks! Fixed the issues and added documentation to the new methods.

artemcm · 2025-07-01T17:01:24Z

@swift-ci test

artemcm · 2025-07-01T17:02:58Z

@swift-ci test Windows platform

artemcm · 2025-07-01T19:28:05Z

@swift-ci test Windows platform

charles-zablit · 2025-07-01T23:12:32Z

Odd, the build error happens only on Windows:

C:\Users\swift-ci\jenkins\workspace\swift-driver-PR-windows\swift-driver\Sources\SwiftDriver\Driver\Driver.swift:870:13: error: cannot find 'checkIfMatchingPythonArch' in scope
 868 |     #if os(Windows)
 869 |       if args[1] == "-repl" {  // a `-` gets automatically added to `swift repl`.
 870 |         try checkIfMatchingPythonArch(
     |             `- error: cannot find 'checkIfMatchingPythonArch' in scope
 871 |           cwd: ProcessEnv.cwd, env: env, diagnosticsEngine: diagnosticsEngine)
 872 |       }

Do I need to manually import the method?

EDIT: Added the file to CMakeList.txt.

charles-zablit · 2025-07-02T07:41:18Z

@swift-ci please test windows platform

charles-zablit · 2025-07-02T11:07:05Z

@swift-ci please test

artemcm · 2025-07-02T16:13:58Z

@swift-ci please test windows platform

jakepetroules · 2025-07-03T17:04:56Z

Sources/SwiftDriver/Utilities/PythonArchitecture.swift

+  static func readWindowsExecutableArchitecture(
+    cwd: AbsolutePath?, env: [String: String], filename: String
+  ) -> Self {
+    let searchPaths = getEnvSearchPaths(pathString: env["Path"], currentWorkingDirectory: cwd)


Environment on Windows is case-insensitive and this WILL lead to very subtle bugs in cases where Path happens to be cased as PATH.

Do we have an Environment abstraction available to SwiftDriver like we have in SwiftPM that we can use instead of a raw [String: String] dictionary?

We have TSCBasic.ProcessEnvironmentBlock which I just transitioned most uses in the driver to: fe4a66c.

Though this is missing a bunch of good stuff that SPM's Environment has, it does use ProcessEnvironmentKey which appears to differentiate case sensitivity for Windows in particular. @charles-zablit we should switch to that here.

jakepetroules · 2025-07-03T17:05:27Z

Sources/SwiftDriver/Utilities/PythonArchitecture.swift

+/// When installing the x86 toolchain on ARM64 Windows, if the user does not
+/// install an x86 version of Python, they will get acryptic error message
+/// when running lldb (`0xC000007B`). Calling this function before invoking
+/// lldb gives them a warning to help troublshoot the issue.


nit: "troublshoot" mis-spelled

jakepetroules · 2025-07-03T17:05:38Z

Sources/SwiftDriver/Utilities/PythonArchitecture.swift

+/// of the Python installation.
+///
+/// When installing the x86 toolchain on ARM64 Windows, if the user does not
+/// install an x86 version of Python, they will get acryptic error message


nit: "acryptic" => "a cryptic"

charles-zablit self-assigned this Jun 27, 2025

charles-zablit requested a review from adrian-prantl June 27, 2025 14:23

charles-zablit force-pushed the charles-zablit/windows/warn-architecture-mismatch-python branch from a1ef87c to def835d Compare June 27, 2025 18:22

adrian-prantl requested a review from artemcm June 27, 2025 18:25

adrian-prantl reviewed Jun 27, 2025

View reviewed changes

charles-zablit force-pushed the charles-zablit/windows/warn-architecture-mismatch-python branch from def835d to f163a58 Compare June 27, 2025 22:50

artemcm requested changes Jun 27, 2025

View reviewed changes

artemcm requested a review from compnerd June 27, 2025 23:29

compnerd reviewed Jun 27, 2025

View reviewed changes

charles-zablit requested review from artemcm and compnerd July 1, 2025 11:51

artemcm approved these changes Jul 1, 2025

View reviewed changes

artemcm reviewed Jul 1, 2025

View reviewed changes

Sources/SwiftDriver/Driver/Driver.swift Outdated Show resolved Hide resolved

charles-zablit force-pushed the charles-zablit/windows/warn-architecture-mismatch-python branch from 47bc64b to ede165c Compare July 1, 2025 16:59

charles-zablit enabled auto-merge July 1, 2025 17:00

artemcm approved these changes Jul 1, 2025

View reviewed changes

[windows] add a check for a matching python architecture

4c094d5

charles-zablit force-pushed the charles-zablit/windows/warn-architecture-mismatch-python branch from ede165c to 4c094d5 Compare July 2, 2025 11:06

charles-zablit mentioned this pull request Jul 2, 2025

Verify LLDB python version swiftlang/vscode-swift#539

Open

jakepetroules requested changes Jul 3, 2025

View reviewed changes

	/// install an x86 version of Python, they will get acryptic error message
	/// install an x86 version of Python, they will get a cryptic error message

Python Path pointing to	PYTHONHOME pointing to	Output
ARM64	Nothing	Crash with warning
ARM64	ARM64	Crash with warning
ARM64	AMD64	Crash with warning
AMD64	Nothing	Launches fine
AMD64	ARM64	Fatal Python error: init_fs_encoding
AMD64	AMD64	Launches fine

[windows] add a check for a matching python architecture #1942

Are you sure you want to change the base?

[windows] add a check for a matching python architecture #1942

Uh oh!

Conversation

charles-zablit commented Jun 27, 2025

Uh oh!

charles-zablit commented Jun 27, 2025

Uh oh!

charles-zablit commented Jun 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adrian-prantl commented Jun 27, 2025

Uh oh!

charles-zablit commented Jun 27, 2025

Uh oh!

artemcm commented Jun 27, 2025

Uh oh!

artemcm left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

charles-zablit Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

charles-zablit commented Jun 30, 2025

Uh oh!

artemcm commented Jun 30, 2025

Uh oh!

artemcm commented Jun 30, 2025

Uh oh!

charles-zablit commented Jul 1, 2025

Uh oh!

charles-zablit commented Jul 1, 2025

Uh oh!

compnerd commented Jul 1, 2025

Uh oh!

artemcm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

charles-zablit commented Jul 1, 2025

Uh oh!

artemcm commented Jul 1, 2025

Uh oh!

artemcm commented Jul 1, 2025

Uh oh!

artemcm commented Jul 1, 2025

Uh oh!

artemcm left a comment •

edited

Loading

charles-zablit Jun 30, 2025 •

edited

Loading

charles-zablit commented Jul 1, 2025 •

edited

Loading