[ffi] Implement leaf FFI calls

[sjindel-google]

Add support for FFI leaf calls:

• No need to enter/leave safepoint - avoid few locked instructions
• No need to create exit frame or update Thread state
• No need to block all registers; callee-saved registers can still be live across the call
• Replace existing InvokeMathCFunction with FFICall

[mraleph]

Additional benefit is that you can pass some pointers down directly (of course assuming that native side is not going to store them somewhere), e.g. pass an internal pointer into typed data arrays or strings, because you know that GC can't move them. For example this opens possibility to just call memcpy to quickly copy memory from typed array or a string into some other place.

[ghost]

Some informal benchmarking suggests that replacing functions like dart:math's sqrt and sin with FFI native calls, result in a run time of about 200%, compare to the current implementation.

[dcharkes]

Some informal benchmarking suggests that replacing functions like dart:math's sqrt and sin with FFI native calls, result in a run time of about 200%, compare to the current implementation.

That is without implementing leaf-calls I presume @cskau-g.

It might be worth exploring implementing leaf calls. Both for dart:math, as well as for example exposing the inner pointers of TypedData as <...>* in FFI trampolines (StackOverflow).

[ghost]

Exactly. My previous comment was to further back the need for leaf-calls.

[mkustermann]

As discussed in todays meeting we might want to use optional parameter: DynamicLibrary.lookupFunction<..>(..., isLeaf: true) Pointer<NativeFunction<...>>.asFunction(..., isLeaf: true).

[mkustermann]

Let's keep this issue open until the register spilling subtask has also been implemented.

[Keithcat1]

Is there any planned support for allowing to pass pointers to typed data to C functions when doing a leaf call?
Might also be nice to allow passing pointers to Dart integers at least when the expected native type is Pointer, instead of implementing #42509

[mraleph]

@Keithcat1

Is there any planned support for allowing to pass pointers to typed data to C functions when doing a leaf call?

Yes, we want to support that eventually. It's an important functionality.

Might also be nice to allow passing pointers to Dart integers at least when the expected native type is Pointer, instead of implementing #42509

I am not sure what you mean here. Dart integers are immutable and (often) unboxed so pointer-to-an-integer does not really have a well defined meaning.

[Keithcat1]

What I mean is that it's fairly common for a C library to expect something like a Pointer or some other integer type, and the only way Dart can allocate such a pointer currently is with malloc, whereas C libraries tend to just allocate those integers on the stack.
Whenever I'm encountering this problem, I use:

static late final _floatPtr = ffi.malloc<Float>();

Even though it's unsafe, I wish there was a way to allocate memory on the stack.
Or maybe, if Dart functions ever support returning multiple values, it might be possible to write code like:

final lib = DynamicLibrary.open("mylib.dll");
final nativeFunctionThatNeedsIntPointer = lib.lookupFunction<Void Function(Pointer<Uint32>), (void, int) Function()>("add_inplace")

This code tells the FFI that instead of the Dart function accepting a Pointer as expected, Dart should allocate a temporary Uint32 on the stack whenever this function is called, pass a pointer to that Uint32 to the add_inplace, then return whatever value is now in the integer. This is safer than allocating memory on the stack, but gives you less control. Not that FFI is ever truly safe to begin with.

[mraleph]

@Keithcat1 I see what you mean. You want a pointer to the variable more or less, just like it is possible in C.

Yeah, I also found myself wanting something like that when working with APIs which involve output parameters. This request is tracked by #42509 as you point out yourself. I don't think it is in any way tied to leafness of the call.

[Keithcat1]

Will I only be able to pass a Typed data pointer to C when using a leaf call, or can I also do this in a non-leaf call? Also, how would it work?

[ghost]

Will I only be able to pass a Typed data pointer to C when using a leaf call, or can I also do this in a non-leaf call?

Anything you can do with a leaf call you can do with a non-leaf call.

Leaf calls is an optimisation where the VM foregoes some of the safety measures for the sake of speed.

[dcharkes]

Are we not guaranteed GC doesn't move objects in a leaf call? In that case we can do more in a leaf call than a non-leaf call.

[ghost]

Hm. That's a good question. I don't know if we strictly guarantee that as a part of the contract with leaf calls (and promise that that will never change)?

I believe the "sales pitch" for leaf calls from the user's perspective so far has been what I mentioned above.

[mraleph]

I don't know if we strictly guarantee that as a part of the contract with leaf calls?

Yes, we do. We don't enter a safepoint around leaf calls - which means GC can't happen inside. That's the whole idea behind leaf calls. User is also not allowed to call back into Dart (hence "leaf" in the name).

That's highlighted in the comment above: #36707 (comment)

Will I only be able to pass a Typed data pointer to C when using a leaf call, or can I also do this in a non-leaf call? Also, how would it work?

Only on leaf calls. As explained above we can't allow that on non-leaf calls.

[Keithcat1]

What would the syntax for passing a typed data pointer be?

[mraleph]

What would the syntax for passing a typed data pointer be?

If I were to make a suggestion then I'd suggest the following rule: if parameter in the native signature is Pointer<T> and parameter in the Dart signature is subtype of TypedData and function is leaf then callee receives pointer to typed data storage rather than handle to the typed data itself.

So something like this:

 final memcpy = DynamicLibrary.process().lookupFunction<
    Pointer<Void> Function(Pointer<Void>, Pointer<Void>, Size),
    Pointer<Void> Function(TypedData, TypedData, int)>('memcpy', isLeaf: true);

final size = 1024*1024;
final a = Uint8List(size);
final b = Uint8List(size);
memcpy(a, b, size);

[dcharkes]

That prohibits passing Pointers. Maybe we should consider introducing a base class in Dart land that has TypedData and Pointer as subtypes. (In the VM we have PointerBase.)

[Keithcat1]

I don't really understand why typed data pointers can't be passed when doing a non-leaf call. Go lets you pass slice pointers, provided the C function only holds on to the pointer until it returns.
Is it because the typed data could be garbage collected while the C function call is in progress? Then just keep a reference to it until the C function call is done, right?

[dcharkes]

@Keithcat1 during a non-leaf call, the Dart GC might move a TypedData to another location in memory.

If what you say about Go is correct, then Go does not run the GC during a call to C. This is similar to what we have for leaf calls. We expect leaf calls to be short, so we don't want to waste cycles on telling the GC that it can run. But non-leaf calls can be long-running, so we there we would like the GC to be able to run. Does that answer your question?

[mraleph]

If what you say about Go is correct, then Go does not run the GC during a call to C.

FWIW Go GC is currently non-moving.

Even in moving GC you can make this work but it requires GC to support object pinning (support situations when certain objects can't move because they are pinned until after they are unpinned). Object pinning is a complexity that we want to avoid as much as possible (at least with the current GC architecture which assumes that any object in new space can be moved freely).

[dcharkes]

Let's keep this issue open until the register spilling subtask has also been implemented.

I believe this was addressed in ba341d7.

For extra features, or more optimizations for leaf calls please open new issues.

Related open issues:

• [vm/ffi] Implement TypedData unwrapping to Pointers in Leaf calls #44589
• [vm/ffi] Investigate copy-free Strings #39787
• [vm/ffi] Unwrapping Strings in FFI calls #47992