Introduce `CallConvSwift` to represent the Swift ABI calling convention

[AaronRobinsonMSFT]

Background and Motivation

The Swift programming language has a different ABI, runtime environment, and object model, making it challenging to call from the .NET without the runtime support. Ideally, we want to avoid generating extra wrappers in the objection tools and attempt to directly call all kinds of Swift functions.

For interop developers of managed APIs wrapping native Swift API layers, reconciling the Swift ABI with the existing P/Invoke options is close to intractable. The proposal would be to add built-in support for the Swift ABI via P/Invokes and Reverse P/Invokes through UnmanagedCallConvAttribute. This would directly help out binding efforts in the MAUI space.

Proposed API

According to the calling convention, the self context has dedicated registers, and it is always passed through them since it's heavily used. Methods calling other methods on the same object can share the self context.

Here are cases when self context is passed via register:

• Instance methods on class types: pointer to self
• Class methods: pointer to type metadata (which may be subclass metadata)
• Mutating method on value types: pointer to the value (i.e. value is passed indirectly)
• Non-mutating methods on value types: self may fit in one or more registers, else passed indirectly
• Thick closures, i.e. closures requiring a context: the closure context

Error handling is also handled through registers, so the caller needs to check for error and throw if necessary. Implementing P/Invoke thunks should simplify registers juggling by using predefined set of registers in these scenarios.

namespace System.Runtime.CompilerServices
{
+    public class CallConvSwift
+    {
+        public CallConvSwift() { }
+    }
}

Based on the design doc we want to introduce types to represent each of the special registers.

+ namespace System.Runtime.InteropServices.Swift
+ {
+     public readonly struct SwiftSelf
+     {
+         public SwiftSelf(IntPtr value)
+         {
+             Value = value;
+         }
+
+         public IntPtr Value { get; }
+     }
+
+     public readonly struct SwiftError
+     {
+         public SwiftError(IntPtr value)
+         {
+             Value = value;
+         }
+
+         public IntPtr Value { get; }
+     }
+ }

Usage Examples

This API would be used in the P/Invoke and Reverse P/Invoke syntax as follows:

using System.Runtime.InteropServices.Swift;

[UnmanagedCallConv(CallConvs = new Type[] { typeof(CallConvSwift) })]
[DllImport("SwiftLibrary", EntryPoint = "export")]
public unsafe static extern nint conditionallyThrowError(bool willThrow, SwiftError* error);

[UnmanagedCallConv(CallConvs = new Type[] { typeof(CallConvSwift) })]
[DllImport("SwiftLibrary", EntryPoint = "export")]
public static extern nint getMagicNumber(SwiftSelf self);

Using the CallConv* approach would also permit use in C# unmanaged function pointers. This would follow the Swift ABI during dispatch. Documentation for the Swift ABI can be found here and its design philosophy here.

/cc @dotnet/jit-contrib @dotnet/interop-contrib @Redth

[Redth]

/cc @dalexsoto @rolfbjarne

[jkotas]

What would be the differences between regular C x64/arm64 Apple calling conventions and the CallConvSwift calling conventions? (I have glanced over the documents at https://github.com/apple/swift/tree/main/docs/ABI and I do not see any obvious differences.)

[mandel-macaque]

@jkotas the register usage is different AFAIK but I'm not an expert, I know because of @stephen-hawley who can give a lot of details.

[AaronRobinsonMSFT]

@jkotas There is definitely some overlap, but there are also novel semantics we would need to deal with if we were support the ABI in the runtime. For example, the error register would be something we would need to respect when calling. How we responded to a non-zero value could vary but we would need to respond in some way – fail fast, convert to .NET exception, etc.

[jkotas]

or example, the error register would be something we would need to respect when calling. How we responded to a non-zero value could vary

Agree, the error register looks special. Do all Swift methods set the error register or just some of them? Do we need additional APIs to configure how to deal with the errors returned in the error register?

[stephen-hawley]

Will this handle generic types and especially with regards to passing protocol witness tables?

[AaronRobinsonMSFT]

Yay! Thanks for joining @stephen-hawley. I created this issue to lure someone who has far more knowledge about the Swift ABI to hopefully chime in. @jkotas brings up a great question about how we would handle the contents of the error register, this will require understanding our options. It would also be interesting to think can users suggest or take over that responsibility from the runtime.

Will this handle generic types and especially with regards to passing protocol witness tables?

This is another excellent question. If I understand correctly, Swift has a fair number of features that C# doesn't have obvious mappings for. I assume the passing protocol witness tables is one of them. The desire of this API would be to help core and performance sensitive APIs, for example the native UI stack supporting MAUI. If this couldn't be used to help improve the MAUI experience, then I think it would be less compelling, at least in the near term. I'd defer to your thoughts on how much of the Swift ABI we would need and if "all of it" is the answer, then the answer to the above question would need to be "yes" I think.

Once we get the list of what we need to support our next step would be to ensure that we could represent it in C# and if not, we can start that ask. This issue is really about pushing the conversation and getting clarity in the open on needs both first party and from the community.

[stephen-hawley]

With regards to the error register, if you have a method in swift that looks like this:

public func foo () throws -> SomeType
{
}

If the method throws it will set the error register to the error. If it does not then the error register is zero and the return value is valid.

In Binding Tools for Swift, I work around this by writing an adapter to the method. For the example, the adapter looks like this:

public func adapt_foo(return: UnsafeMutablePointer<SomeType, Error, Bool>) 
{
    do {
      var return = try foo ()
      setExceptionNotThrown(value:return, retval:retval);
   } catch let err {
     setExceptionThrown(err:err, retval: retval);
   }
}

In addition, how will this handle closures, which in addition to the calling conventions will also have a dedicated register (the self register) which is used for accessing data bound by the closure? Will it also handle closures that are escaping and non-escaping?

I understand that you're looking at Maui, but I'm trying to get as close to 100% coverage to the swift language from C# as I can.

Do you have plans for handling the ARC in value types? For example, since swift has semantics for what happens to an object when it gets copied and destroyed, I can't model value types as C# value types, so instead I model them as a class implementing IDisposable with a byte array to opaque data and accessors to get at the elements. When I need to marshal them to swift, I always pass by reference and make a copy of the payload into a stack allocated array using the method in the value type to copy the data in a way that follows ARC (and do the reverse on the way back).

I have so many questions. In a good way.

[stephen-hawley]

With regards to protocol witness tables, these come into play in a couple of different contexts.
If I have this function in swift:

public func foo<T:SomeProtocol>(a: Int) {
}

This actually gets implemented as under the hood like this:

public func foo<T>(T:SomeProtocol, tType: T.Self, pwt: ProtocolWitnessTableForSomeProtocolWithRespectToT) {
}

In other words, for every generic type, there is an extra argument added in that is the swift equivalent of the C# Type object then for each protocol constraint, there is a protocol witness table pointer that is effectively the vtable for the protocol with respect to the generic type.

Where this gets particularly interesting is when I need to model a class with a generic virtual method:

open class SomeClass {
    public init () { }
    open func someMethod<T: SomeProtocol>(value: T) -> T {
    }
}

To model this, I write an adapter class that overrides each of the virtual methods, then vectors into C# through a table of function pointers that receive the arguments. The problem is that I need to pass in the type of T (easy, I can get it through T.Self, but I also need the protocol witness table which is not accessible since it's implicit. I handle this through a set of C functions that are written to take arguments, but I surface them in swift as taking no arguments. So that for any protocol, I can write the following in swift:

public func protocolWitnessTableOf<T:SomeProtocol>(t: T) -> OpaquePointer {
    return swiftAsmArg2(); // returns pwt
   // void *swiftAsmArg2(void *arg0, void *arg1, void *arg2) { return arg2; }
}

With regards to the task of calling virtual methods in swift, I'm torn on this. Virtual methods use a self pointer. I have to wrap any virtual method like this:

public func virtual_someMethodOnFoo(this: Foo, arg0: SomeValueType) {
    this.someMethod(arg0);
}

The problem with this is that someMethod puts the instance pointer into the self register (R13 IIRC on x64) and then moves arg0 into rsi. In the case of the adapter, this requires maximal registerjuggling.
I'd like to avoid this and dispatch through the class' vtable directly. Unfortunately, the ordering of the vtable is undocumented and as of Swift 5, in order to maintain forward compatibility, Apple has said outright that vtables are fragile and they won't use them in any caller from outside the class.

And if it helps, I have a lot of this already documented here.

Feel free to reach out with questions. For sure I think we should get our stories straight in terms of how to ensure that marshalling value types works. For example, all my mappings of value types implement the interface ISwiftValueType which is defined like this:

	public interface ISwiftValueType : IDisposable {
		byte [] SwiftData { get; set; }
	}

And all swift class types implement the interface ISwiftObject:

	public interface ISwiftObject : IDisposable {
		IntPtr SwiftObject { get; }
	}

It would be super nice if I could write a pinvoke for the previous example of a method wrapper as:

[UnmanagedCallConv(CallConvs = new Type[] { typeof(CallConvSwift) })]
[DllImport("SwiftLibrary", EntryPoint = "export")]
internal static void virtual_someMethodOnFoo(ISwiftObject foo, ISwiftValueType value); // or maybe I use the real types here?

[AaronRobinsonMSFT]

Do you have plans for handling the ARC in value types?
For sure I think we should get our stories straight in terms of how to ensure that marshalling value types works.

The proposal wouldn't be about enabling marshalling support in the same way we support marshalling a value type to a C/C++ struct. This would be about ensuring we place arguments in the correct registers and following calling convention semantics. The actual marshalling logic would be source generated – see DllImport generator. However, if we couldn't source generate marshalling code that would be an indication the runtime needs to provide something, but only after ensuring there was no source generation solution.

I think the best example of this would be the error register above. Are there others?

[AaronRobinsonMSFT]

Feel free to reach out with questions.

Yes! That needs to happen. It is on my list, be prepared :-)