Add support for external memory

[SupaMaggie70Incorporated]

In order to allocate memory that can be exported in vulkan, a vkExportMemoryAllocateInfoKHR struct must be passed to the pNext chain of the create info. unfortunately gpu-allocator doesn't allow for this. I see 3 ways to approach this:

• Add support for external memory specifically
• Add support for a custom pNext object for the vulkan allocator
• Add some way to access gpu-allocator's internal allocator, so that the user can manage the allocation themself but still let gpu-allocator select the memory type and whatnot

I think option number 3 would be the most useful overall but would take the most work. I'd be interested in seeing either that or option 2 implemented here.

I'm just gonna leave this issue here until I get feedback on which approach should be taken.

[Jasper-Bekkers]

I'm leaning most towards 1, that way we can also nicely unify it across the backends.

[SupaMaggie70Incorporated]

I'm leaning most towards 1, that way we can also nicely unify it across the backends.

I would say one advantage even on a single backend of 1 is that it could make use of existing algorithms to combine multiple logical allocations under the same driver allocation. That being said, I'd be in favor of opening up gpu-allocator a little even if we go for 1, but that's out of the scope of this issue.

Only thing I'd say is that some platform specific code must be written, as using fd's is slightly different than win32 handles. And there may be cases where the driver decides that some allocations must be dedicated, so this might complicate matters slightly.

If you give the green light on 1 however, I might start working on it tomorrow or some other time soon.

[Jasper-Bekkers]

Only thing I'd say is that some platform specific code must be written, as using fd's is slightly different than win32 handles. And there may be cases where the driver decides that some allocations must be dedicated, so this might complicate matters slightly.

Nicest solution I think would be to create a small side-crate like raw-window-handle (raw-render-handle) with minimal dependencies (e.g. all types are NonNull<c_void> etc) that we make sure is well maintained, explicitly for sharing these kind of resources around. Would be nice if it's initially part of this repository so we know that maintainership doesn't get abandoned after a while.

If that's too much effort, I would suggest to just carve out a u64 as that can fit all pointers, handles, and file descriptors in practice.

If you give the green light on 1 however, I might start working on it tomorrow or some other time soon.

You have my blessing, notice that we also support Dx12 and Metal, where applicable and possible those should also be implemented.

[SupaMaggie70Incorporated]

Only thing I'd say is that some platform specific code must be written, as using fd's is slightly different than win32 handles. And there may be cases where the driver decides that some allocations must be dedicated, so this might complicate matters slightly.

Nicest solution I think would be to create a small side-crate like raw-window-handle (raw-render-handle) with minimal dependencies (e.g. all types are NonNull<c_void> etc) that we make sure is well maintained, explicitly for sharing these kind of resources around. Would be nice if it's initially part of this repository so we know that maintainership doesn't get abandoned after a while.

If that's too much effort, I would suggest to just carve out a u64 as that can fit all pointers, handles, and file descriptors in practice.

If you give the green light on 1 however, I might start working on it tomorrow or some other time soon.

You have my blessing, notice that we also support Dx12 and Metal, where applicable and possible those should also be implemented.

I had thought of this separate crate idea like raw-window-handle previously. I think it would work nicely with what we want. Unfortunately, I don't own a Mac so for the short term I could only contribute to work for win32 and fd memory sharing. For now, I'll probably just write a prototype without any fancy handle abstraction. I'm working a little on external memory with wgpu too so I'd like to get more ecosystem feedback before starting work on that.

[Jasper-Bekkers]

Unfortunately, I don't own a Mac so for the short term I could only contribute to work for win32 and fd memory sharing

Just leave macOS as a todo!() for the time being then; maybe somebody else in the community will want to pick it up.

[MarijnS95]

I did actually consider adding such a thing directly to raw-window-handle long ago, but never found the time to propose it upstream.

While a RawWindowHandle is generally useful for a lot of OS-specific things, in the context of computer graphics these handles commonly represent a Surface, e.g. something that rendered buffers can be presented from (and also first dequeued from). raw-window-handle however doesn't have an abstraction for such individual buffers however.

This initially came up for me in the context of GBM rendering with EGL interop, where a GBM Surface could be shared using that crate abstraction, but a GBM Buffer Object (single image) could not. I'm curious to see how FDs (the non-opaque variant) and Win32 handles and Composition Surface Textures and Android Hardware buffers all fit into this.

[SupaMaggie70Incorporated]

@MarijnS95 Unfortunately I'm mainly interested in this purely for the memory sharing so I can send data from a compute library to a graphical visualizer. As such, I won't be dealing with textures or advanced use cases, maybe not even external semaphores. If I were to get some work started though, it's good to know that you might be able to advise a little here and there.

[Jasper-Bekkers]

I did actually consider adding such a thing directly to raw-window-handle long ago, but never found the time to propose it upstream.

What we're discussing is a crate similar to raw window handle, but separate.