Bug: GGML can no longer be statically linked to llama.cpp due to the source code reorganization

[lcarrere]

What happened?

Since the commit #8006 GGML is now compiled as Dynamic library (vs static library, before).

I can't find any option to reintroduce the previous mode. There is a GGML_STATIC option into the CMakeLists.txt of the ggml solution but it seems to do nothing.

I there a way to reintroduce static compilation mode?

Thanks a lot!

Loïc

Name and Version

latest.

cmake .. -DGGML_NATIVE=OFF -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_SERVER=OFF -DBUILD_SHARED_LIBS=ON -DGGML_AVX2=ON -DGGML_AVX512=OFF -DGGML_STATIC=ON

What operating system are you seeing the problem on?

No response

Relevant log output

No response

[slaren]

It should be linked statically without the -DBUILD_SHARED_LIBS=ON parameter (edit: BUILD_SHARED_LIBS is the default in most cases). What is not possible at the moment is to build llama.cpp as a shared library and ggml as a static library, but I am not sure that's really a problem.

[lcarrere]

Thanks a lot for the reply @slaren!

This flag indeed links GGML statically into llama.cpp. However, in this configuration, llama.cpp is also generated as a static library. I'm trying to compile llama.cpp as a dynamic library.

For me, the ideal option would be to have both GGML_BUILD_SHARED_LIBS and LLAMA_BUILD_SHARED_LIBS.

[mudler]

I'm hitting the same issue here - however it's more dodgy in my case as I'm linking also grpc - and, before the mentioned PR that's how the linker was looking like on the resulting binaries, and everything was working fine:

       linux-vdso.so.1 (0x00007ffe3a6b6000)
        libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f784dbfc000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f784d800000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f784db15000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f784dae6000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f784d400000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f784f882000)

now, with shared mode on there are more shared libraries used, but that breaks badly my linker now:

/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld: ../../bin/grpc-server: hidden symbol `_ZN6google8protobuf2io19EpsCopyOutputStream16WriteRawFallbackEPKviPh' in /home/mudler/_git/LocalAI/backend/cpp/grpc/installed_packages/lib64/libprotobuf.a(coded_stream.cc.o) is referenced by DSO

and yeah, the symbol is hidden in libprotobuf.a - but the pity of that is that I can't always control how libprotobuf is built (indeed, it fails in mac with homebrew now)

It should be linked statically without the -DBUILD_SHARED_LIBS=ON parameter. What is not possible at the moment is to build llama.cpp as a shared library and ggml as a static library, but I am not sure that's really a problem.

Sadly that has no effect here - I never enabled that explicitly before. I 'm going to try to disable it explicitly and see if that's helping

Update: Thanks for the hint @slaren ! specifying -DBUILD_SHARED_LIBS=OFF did the trick here - I guess now -DBUILD_SHARED_LIBS=ON is implied by default?

[slaren]

Some features of llama.cpp require using ggml. If llama.cpp is built as a shared library linking statically to ggml, then applications cannot use these features without linking their own copy of ggml - which may not work at all. Is it really that much of a problem to bundle the ggml shared library alongside llama.cpp?

@mudler BUILD_SHARED_LIBS is the default in most cases:
https://github.com/ggerganov/llama.cpp/blob/f675b20a3b7f878bf3be766b9a737e2c8321ff0d/CMakeLists.txt#L31-L41

[mudler]

Some features of llama.cpp require using ggml. If llama.cpp is built as a shared library linking statically to ggml, then applications cannot use these features without linking their own copy of ggml - which may not work at all. Is it really that much of a problem to bundle the ggml shared library alongside llama.cpp?

In my case it is, because I'm using llama.cpp with gRPC, and that is by default static - and mixing gRPC static with shared ggml and llama cpp breaks the linker as some of the protobuf symbols are hidden when building from a DSO.

@mudler BUILD_SHARED_LIBS is the default in most cases:

https://github.com/ggerganov/llama.cpp/blob/f675b20a3b7f878bf3be766b9a737e2c8321ff0d/CMakeLists.txt#L31-L41

gotcha - thanks @slaren ! - it clicked here now thanks to your previous comment. I thought that was defaulting to ON before as well when I was looking at the PR diff, but it looks it wasn't (at least here), so now I had to disable it explicitly to get things going. I have yet to see what LocalAI's CI tells, but locally at least now builds fine as before.

[lcarrere]

Thanks folks for what you've shared.

Some features of llama.cpp require using ggml. If llama.cpp is built as a shared library linking statically to ggml, then applications cannot use these features without linking their own copy of ggml

I don't believe that was the case unless I missed something. I used to use pretty much all of the llama.cpp API by compiling llama.cpp as a shared library, statically linking ggml, at least on Windows, macOS, and Linux systems.

Is it really that much of a problem to bundle the ggml shared library alongside llama.cpp?

Good point. I would not argue that it is a real problem. Plus I understand the willingness to keep ggml as a separate binary for the long run and/or for building apps using llama.cpp, Whisper.cpp, and probably other upcoming projects.

-> I'm reviewing my compilation and deployment pipelines to follow this approach.

If you don't mind, I'm marking this case as resolved.

[hmartinez82]

I'm having the same issue. Tried with both GCC and Clang in MSYS2.
Even when setting GGML_STATIC to true, there's still dependencies to libc++, pthread ,etc

LLMA_STATIC from before was working and doing what we needed.

What are the flags now to get llama-server linked statically to libc++, etc?

[mudler]

LLAMA_STATIC never really worked here as it always linked on libgomp and failed linking, same GGML_STATIC (but maybe caused by my requirements? ). Did you tried to set BUILD_SHARED_LIBS to OFF?

[hmartinez82]

Yes. I agree with you. I got lbc++ statically linking, but gomp and openblas are always dynamically linked. You can reclose this if you want.