sync : llama.cpp

[ggerganov]

No description provided.

[dg0yt]

I think this causes an error in vcpkg CI now, android with static library linkage.

ld.lld: error: duplicate symbol: vk::detail::defaultDispatchLoaderDynamic
>>> defined at test-cmake.cpp:6 (/mnt/vcpkg-ci/b/vcpkg-ci-ggml/src/e98bd99269-a8d27b3c76.clean/examples/test-cmake/test-cmake.cpp:6)
>>>            CMakeFiles/test-cmake.dir/test-cmake.cpp.o:(vk::detail::defaultDispatchLoaderDynamic)
>>> defined at ggml-vulkan.cpp:14 (/mnt/vcpkg-ci/b/ggml/src/v0.9.1-72a0cec7be.clean/src/ggml-vulkan/ggml-vulkan.cpp:14)
>>>            ggml-vulkan.cpp.o:(.data._ZN2vk6detail28defaultDispatchLoaderDynamicE+0x0) in archive /mnt/vcpkg-ci/installed/arm64-android/debug/lib/libggml-vulkan.a
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

So ggml now exposes a symbol from the vk namespace.

I'm not a Vulkan expert. Is this the right thing to do in a library under C++ ODR rules? In a ggml test port in vcpkg this is done the "executable" for Android. AFAIU the final executable is the only way to do it reliably with regard to ODR.

(Admittedly this is a linker error about duplicate symbols, not a compiler error about ODR.)

[dg0yt]

FTR the test port builds the example executable with this change:

diff --git a/examples/test-cmake/CMakeLists.txt b/examples/test-cmake/CMakeLists.txt
index d6bc0cc4..395a63c7 100644
--- a/examples/test-cmake/CMakeLists.txt
+++ b/examples/test-cmake/CMakeLists.txt
@@ -8,3 +8,9 @@ find_package(ggml CONFIG REQUIRED)
 set(TEST_TARGET test-cmake)
 add_executable(test-cmake test-cmake.cpp)
 target_link_libraries(test-cmake PRIVATE ggml::ggml)
+
+if(ANDROID AND TARGET ggml::ggml-vulkan)
+    # Instantiates VULKAN_HPP_DEFAULT_DISPATCH_LOADER_DYNAMIC_STORAGE
+    find_package(Vulkan REQUIRED)
+    target_link_libraries(test-cmake PRIVATE Vulkan::Vulkan)
+endif()
diff --git a/examples/test-cmake/test-cmake.cpp b/examples/test-cmake/test-cmake.cpp
index 029c8898..9d4bbe19 100644
--- a/examples/test-cmake/test-cmake.cpp
+++ b/examples/test-cmake/test-cmake.cpp
@@ -1,5 +1,11 @@
 #include "ggml-backend.h"
 
+#if defined(ANDROID)
+#define VULKAN_HPP_DISPATCH_LOADER_DYNAMIC 1
+#include <vulkan/vulkan.hpp>
+VULKAN_HPP_DEFAULT_DISPATCH_LOADER_DYNAMIC_STORAGE
+#endif
+
 int main(void) {
     ggml_backend_load_all();
     return 0;

[talregev]

The problem is from vcpkg side. because on android we force these VULKAN_HPP_DISPATCH_LOADER_DYNAMIC and VULKAN_HPP_DEFAULT_DISPATCH_LOADER_DYNAMIC_STORAGE while compile with vulkan. This create the duplication of the symbols. I fix it in this PR: microsoft/vcpkg#47464

[dg0yt]

If a second lib does the same, we are back at start.
And that's why it cannot be in the libs.

[talregev]

You should mention that in the PR when I ask for solution and elaborate more. Now I understand.

[dg0yt]

Actually this is the implementation.