Add support for testable executables on Windows

[jakepetroules]

This uses the /ALTERNATENAME flag to link.exe, which is roughly equivalent to -alias/--defsym. This has been verified to work in a sample project.

See https://devblogs.microsoft.com/oldnewthing/20200731-00/ for more info.

Closes #6367

[jakepetroules]

@compnerd Would appreciate your input here on whether you think this is a reasonable solution, at least for the interim.

For the longer term once Swift Build becomes the default build system we could consider generating an assembly file to define a hard alias if there are any downsides to this approach, but I'm not thinking of any right now.

[jakepetroules]

@swift-ci test

[jakepetroules]

@swift-ci test windows self hosted

[compnerd]

I'm not sure that this is the right approach. I've known about /alternatename:. It doesn't provide the same semantics as -alias. It is designed for strongly weak-linking the symbol. That is, if main is not found (this is never the case), then ..._main would be considered as an equivalent.

The long term solution here is to fix LLVM. When clang migrated to the New Pass Manager, it dropped the symbol rewriter from the module pass (mostly because of ARM64). We should restore that, and then generate the yaml configuration to rename the symbol during compile time at the LLVM IR level. This will provide us with a clean, completely portable solution.

[jakepetroules]

I've known about /alternatename:. It doesn't provide the same semantics as -alias. It is designed for strongly weak-linking the symbol. That is, if main is not found (this is never the case), then ..._main would be considered as an equivalent.

I know it doesn't have exactly the same semantics, but it might be close enough to work for our specific use case.

The long term solution here is to fix LLVM. When clang migrated to the New Pass Manager, it dropped the symbol rewriter from the module pass (mostly because of ARM64). We should restore that, and then generate the yaml configuration to rename the symbol during compile time at the LLVM IR level. This will provide us with a clean, completely portable solution.

Meaning we'll no longer need to pass -alias/--defsym to the linker, either?

How much effort and how far out is that? Do you think it's worth landing this as an interim solution in the meantime? And/or the approach of generating a symbol alias in an assembly file?

[jakepetroules]

@swift-ci test windows self hosted

[jakepetroules]

And, what about using /ENTRY for Windows instead of /ALTERNATENAME? That also works (whereas -e/--entry won't on macOS/Linux if linking crt1.o).

[MaxDesiatov]

@swift-ci test self hosted windows

[daveinglis]

Can we remove this in the Package.swift now?

// Workaround SwiftPM's attempt to link in executables which does not work on all
// platforms.
#if !os(Windows)

There some tests in there that specifically test this too. (see testTestsCanLinkAgainstExecutable)

[compnerd]

I've known about /alternatename:. It doesn't provide the same semantics as -alias. It is designed for strongly weak-linking the symbol. That is, if main is not found (this is never the case), then ..._main would be considered as an equivalent.

I know it doesn't have exactly the same semantics, but it might be close enough to work for our specific use case.

Does it? If that works with link.exe, that is an interesting proposition. When I tried it, it did not work for me.

The long term solution here is to fix LLVM. When clang migrated to the New Pass Manager, it dropped the symbol rewriter from the module pass (mostly because of ARM64). We should restore that, and then generate the yaml configuration to rename the symbol during compile time at the LLVM IR level. This will provide us with a clean, completely portable solution.

Meaning we'll no longer need to pass -alias/--defsym to the linker, either?

How much effort and how far out is that? Do you think it's worth landing this as an interim solution in the meantime? And/or the approach of generating a symbol alias in an assembly file?

I'm hoping to work on that soon. I think that we could apply it downstream first before upstream to speed it up some. I don't think that it should be too bad to do that. It would be way better to do that over the symbol aliases or assembly thunks as it is pretty much target agnostic and entirely avoids some of the semantic issues around simply creating and using the alias and hoping that the main entry point goes away.

And, what about using /ENTRY for Windows instead of /ALTERNATENAME? That also works (whereas -e/--entry won't on macOS/Linux if linking crt1.o).

AFAIK, that would be bad. You would need define the contents of ..._main, ensuring that it is identical to the default main that is provided. /entry: also serves as a GC root for /OPT:REF, so you could potentially change the semantics of the program if you are not 100% in sync with the MSFT release.

[jakepetroules]

Can we remove this in the Package.swift now?

Not until this change lands in a released Swift toolchain, if we decide to move forward with it.

Does it? If that works with link.exe, that is an interesting proposition. When I tried it, it did not work for me.

Yeah, I tried an example with a test target that linked an executable, and running the executable and running the tests which referenced symbols from that executable both worked.

From what I understand, for executable targets, the Swift frontend flag -entry-point-function-name will generate the named symbol X from @main or top-level code, instead of the default main. Then, at link time, the object files constituting the executable have no main, so mainCRTStartup looks for main, and because none exists, looks at the /ALTERNATENAME mapping which matches the X value given to -entry-point-function-name, substitutes X for main, and the entry path becomes mainCRTStartup => X => user main code.

AFAIK, that would be bad. You would need define the contents of ..._main, ensuring that it is identical to the default main that is provided. /entry: also serves as a GC root for /OPT:REF, so you could potentially change the semantics of the program if you are not 100% in sync with the MSFT release.

Good to know! I guess this would also bypass initialization in mainCRTStartup, which is probably bad too.

[jakepetroules]

I'm not sure that this is the right approach. I've known about /alternatename:. It doesn't provide the same semantics as -alias. It is designed for strongly weak-linking the symbol. That is, if main is not found (this is never the case), then ..._main would be considered as an equivalent.

Also, just want to clarify -- due to use of -entry-point-function-name it is always the case that main is not found, which is why this works.

[compnerd]

I'm not sure that this is the right approach. I've known about /alternatename:. It doesn't provide the same semantics as -alias. It is designed for strongly weak-linking the symbol. That is, if main is not found (this is never the case), then ..._main would be considered as an equivalent.

Also, just want to clarify -- due to use of -entry-point-function-name it is always the case that main is not found, which is why this works.

Ah, hah! That was the missing bit - okay, that makes sense now. I guess that the last piece that I think needs to be confirmed is that this would not impact the user's use of -entry-point-function-name: https://github.com/compnerd/swift-win32-application/blob/main/Package.swift#L5-L26 for an example.

But, with that additional context, I think that this makes sense as an interim solution while we can get the symbol rewriter hooked up again.

[jakepetroules]

Why does -Xfrontend -entry-point-function-name -Xfrontend wWinMain work? WinMain/wWinMain have different signatures than main/wmain.

I'm not able to build your swift-win32-application example with Swift 6.1 as I get:

Updating https://github.com/compnerd/swift-win32.git
Updated https://github.com/compnerd/swift-win32.git (0.90s)
error: the package manifest at '\[email protected]' cannot be accessed (invalid absolute path '[email protected]') in https://github.com/compnerd/swift-win32.git

[jakepetroules]

@swift-ci test

[jakepetroules]

@swift-ci test windows self hosted

[jakepetroules]

OK, I don't understand why -Xfrontend -entry-point-function-name -Xfrontend wWinMain works due to the different signature (but it appears to work), but I think what I can do is look for -entry-point-function-name in unsafeFlags, and do nothing if that's present and specifies a symbol name other than main, since in that case testable executables won't run into the duplicate symbols issue since the main symbol will already be something other than "main".

I updated the patch to do this; I think this should cover all bases and be ready to land.

[compnerd]

Why does -Xfrontend -entry-point-function-name -Xfrontend wWinMain work? WinMain/wWinMain have different signatures than main/wmain.

They have different signatures, however, the signature that Swift cares about is:

void main()

This effectively matches any signature as a happy side-effect.

I'm not able to build your swift-win32-application example with Swift 6.1 as I get:

Updating https://github.com/compnerd/swift-win32.git
Updated https://github.com/compnerd/swift-win32.git (0.90s)
error: the package manifest at '\[email protected]' cannot be accessed (invalid absolute path '[email protected]') in https://github.com/compnerd/swift-win32.git

Ugh, symlinks have been an annoyance, but the inability to evolve the package manifest with compatibility is just as annoying.

[jakepetroules]

@swift-ci test windows

[jakepetroules]

@swift-ci test windows

[jakepetroules]

@swift-ci test windows self hosted

[jakepetroules]

@swift-ci test windows

[jakepetroules]

@swift-ci test windows self hosted

[jakepetroules]

@swift-ci test windows

[jakepetroules]

@swift-ci test self hosted windows