native-image: support statically linking JNI libraries

[smasher164]

native-image can currently produce static or mostly static executables, except for object files loaded with System/loadLibrary when using JNI. This makes it inconvenient to distribute binaries for applications where the user doesn't have the JNI library present on their system.

The only alternative appears to be to build GraalVM from source with library linked, but I would prefer a solution where I could tell native-image a list of object files to statically link into the resulting binary.

[kristofdho]

You can do this using -H:AdditionalLinkerOptions=<path-to-your-lib>, not sure if that's strictly the right way to do it.
from the output of native-image --expert-options-all:
-H:AdditionalLinkerOptions="" String which would be appended to the linker call.

Alternatively you can write your own feature for your lib and tell native-image exactly what to do with it:

graal/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/SecurityServicesFeature.java

Lines 245 to 258 in fbab70f

	private static void linkSunEC(DuringAnalysisAccess a) {
	NativeLibraries nativeLibraries = ((DuringAnalysisAccessImpl) a).getNativeLibraries();
	/* We statically link sunec thus we classify it as builtIn library */
	PlatformNativeLibrarySupport.singleton();
	NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("sunec");
	/* and ensure native calls to sun_security_ec* will be resolved as builtIn. */
	PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_security_ec");
	nativeLibraries.addStaticJniLibrary("sunec");
	if (isPosix()) {
	/* Library sunec depends on stdc++ */
	nativeLibraries.addDynamicNonJniLibrary("stdc++");
	}
	}

[smasher164]

Thanks for following up @kristofdho!
I made a repo for a minimal proof-of-concept here: https://github.com/smasher164/staticjni.
I passed in AdditionalLinkerOptions with the absolute path to the library, and I no longer compile the library with -shared, however I still see the following error:

root@8b3118f6d2a6:~/staticjni# ./helloworld
Exception in thread "main" java.lang.UnsatisfiedLinkError: no HelloWorld in java.library.path
	at com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryRelative(NativeLibrarySupport.java:132)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:275)
	at java.lang.Runtime.loadLibrary0(Runtime.java:830)
	at java.lang.System.loadLibrary(System.java:1871)
	at HelloWorld.main(HelloWorld.java:5)

(Note, I ran this example inside an ubuntu docker container)

Should I not be using loadLibrary? Or is there something I'm missing here?

[kristofdho]

I'm having troubles getting it to work with -H:AdditionalLinkerOptions to the point that I'm doubting if I ever got it working that way..

The second example does work however. I am on windows btw, so I had to figure out some stuff before I could run your example.

In the end, to get it statically linked into the image, I had to add this custom feature to my classpath:

import com.oracle.svm.core.annotate.AutomaticFeature;
import com.oracle.svm.core.jdk.NativeLibrarySupport;
import com.oracle.svm.core.jdk.PlatformNativeLibrarySupport;
import com.oracle.svm.hosted.FeatureImpl;
import com.oracle.svm.hosted.c.NativeLibraries;
import org.graalvm.nativeimage.hosted.Feature;

@AutomaticFeature
public class HelloWorldFeature implements Feature {

    @Override
    public void beforeAnalysis(BeforeAnalysisAccess access) {
        NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("HelloWorld");
        PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("HelloWorld");
        NativeLibraries nativeLibraries = ((FeatureImpl.BeforeAnalysisAccessImpl) access).getNativeLibraries();
        nativeLibraries.addStaticJniLibrary("HelloWorld");
    }
}

For this to compile, you need this dependency:

<dependency>
    <groupId>org.graalvm.nativeimage</groupId>
    <artifactId>svm</artifactId>
    <version>21.0.0</version>
    <scope>provided</scope>
</dependency>

After this, native-image will itself put HelloWorld.lib (since I'm on windows) in the linker command, so all we have to do is make sure the linker can find it, so I had to add -H:CLibraryPath="<path to folder containing library>" to the native-image command.

In the end, my native-image command looked like this:

native-image --no-fallback --no-server --initialize-at-build-time -H:Name=hello -cp HelloWorld.jar -H:CLibraryPath="<path to folder containing library>"

The part that is missing when using AdditionalLinkerOptions is probably the part where you tell native-image that the Java_HelloWorld* symbols are to be resolved internally, which the addBuiltinPkgNativePrefix call does. But don't quote me on that, it's merely a slightly educated guess.

[smasher164]

Thanks @kristofdho, that works! Did you mean -jar HelloWorld.jar instead of -cp HelloWorld.jar in the native-image command? I've updated my repo accordingly.

[kristofdho]

Ah yes, I didn't have a manifest, so I actually used -cp HelloWorld.jar HelloWorld, but I only partially changed it back to your use case. Should indeed be -jar HelloWorld.jar. Glad I could help you!

[smasher164]

Great. I'll go ahead and close this issue.

[smasher164]

Actually, is it possible to put this information in the GraalVM docs somewhere? I understand the user must implement a Feature, but I feel that this is a common enough use-case to warrant its own place in the docs.

[kristofdho]

Unfortunately that's not something I can help you with. And strictly speaking, I think those calls are implementation details, and not part of the API. So for an actual API conform solution some code changes and indeed, definitely documentation, would be required.

[smasher164]

Gotcha. In that case, would the original purpose of this issue still be valid? Since there is no stable (or at least a way we would want to document) way to currently do this.
I could imagine a flag like -H:RegisterBuiltin=<comma-delimited list of libraries to resolve internally>