[Android] Error running a transformer decoder-based neural network

[adonnini]

Hi,

@kirklandsign I have been developing on Android for more than 15 years. I am familiar with dependencies etc., etc. The issue 8636 had to do with packaging.

I guess you did not see my last comment in
#8636

Here it is again:

Hi Update #2

I worked around the classNotFound exception by creating a java.org.pytorch.executorch package containing all the source files from
executorch/extension/android/src/main/java/org/pytorch/executorch

Now execution fails producing the error log reported below.

I built ```libexecutorch.so``` following the instructions in
https://pytorch.org/executorch/stable/demo-apps-android.html

Package name is the same as expected (and used in the android demo app), i.e. ```org.pytorch.executorch```

The dependencies are

                implementation 'com.facebook.fbjni:fbjni-java-only:0.2.2'
                implementation 'com.facebook.soloader:nativeloader:0.10.5'

as in the android demo app

At this point, I am at a loss as to what I am doing wrong and what I should try next.

Should I add and android module to my project just as there is one in the android demo app. In the android demo app, the android module contains all the modules necessary to use the executorch runtime (e.g. NativePeer) in the application.

I think the issue may be related to multi-project set-up which I do not use in my application (although I do have multiple modules within the project.

Generally speaking, I still have the sense that the set-up to use the executorch runtime on Android applications could/should be simpler, for example not requiring the use of the wrapper modules in org.pytorch.executorch

Please let me know if you need additional information.

Thanks

ERROR LOG

02-23 16:41:41.176: W/libc(25151): Access denied finding property "ro.mediatek.platform"
02-23 16:41:41.176: W/libc(25151): Access denied finding property "ro.chipname"
02-23 16:41:41.177: E/audit(1210): type=1400 audit(1740325301.170:52396): avc:  denied  { read } for  pid=25151 comm="Thread-2" name="u:object_r:media_plat_prop:s0" dev="tmpfs" ino=315 scontext=u:r:untrusted_app_30:s0:c45,c258,c512,c768 tcontext=u:object_r:media_plat_prop:s0 tclass=file permissive=0 SEPF_SM-S908B_12_0001 audit_filtered
02-23 16:41:41.177: E/audit(1210): type=1400 audit(1740325301.170:52397): avc:  denied  { read } for  pid=25151 comm="Thread-2" name="u:object_r:chipname_prop:s0" dev="tmpfs" ino=165 scontext=u:r:untrusted_app_30:s0:c45,c258,c512,c768 tcontext=u:object_r:chipname_prop:s0 tclass=file permissive=0 SEPF_SM-S908B_12_0001 audit_filtered
02-23 16:41:41.179: D/nativeloader(25151): Load /data/app/~~CzYQ5jB-bXszoW-cWE7uUg==/com.android.contextq-B_Rwji2_VyK2Jx3NvUwN1A==/base.apk!/lib/arm64-v8a/libexecutorch.so using ns clns-5 from class loader (caller=/data/app/~~CzYQ5jB-bXszoW-cWE7uUg==/com.android.contextq-B_Rwji2_VyK2Jx3NvUwN1A==/base.apk): ok
02-23 16:41:41.179: W/lNetworkService(25151): CheckJNI: method to register "initHybrid" not in the given class. This is slow, consider changing your RegisterNatives calls.
02-23 16:41:41.183: E/lNetworkService(25151): ----- class 'Lorg/pytorch/executorch/NativePeer;' cl=0x140133a8 -----
02-23 16:41:41.183: E/lNetworkService(25151):   objectSize=240 (224 from super)
02-23 16:41:41.183: E/lNetworkService(25151):   access=0x0000.0000
02-23 16:41:41.183: E/lNetworkService(25151):   super='java.lang.Class<java.lang.Object>' (cl=0x0)
02-23 16:41:41.183: E/lNetworkService(25151):   interfaces (1):
02-23 16:41:41.183: E/lNetworkService(25151):      0: java.lang.Class<org.pytorch.executorch.INativePeer> (cl=0x140133a8)
02-23 16:41:41.183: E/lNetworkService(25151):   vtable (2 entries, 11 in super):
02-23 16:41:41.183: E/lNetworkService(25151):      0: org.pytorch.executorch.EValue org.pytorch.executorch.NativePeer.forward(org.pytorch.executorch.EValue[])
02-23 16:41:41.183: E/lNetworkService(25151):      1: void org.pytorch.executorch.NativePeer.resetNative()
02-23 16:41:41.183: E/lNetworkService(25151):   direct methods (3 entries):
02-23 16:41:41.183: E/lNetworkService(25151):      0: void org.pytorch.executorch.NativePeer.<clinit>()
02-23 16:41:41.183: E/lNetworkService(25151):      1: void org.pytorch.executorch.NativePeer.<init>(java.lang.String, java.util.Map)
02-23 16:41:41.183: E/lNetworkService(25151):      2: com.facebook.jni.HybridData org.pytorch.executorch.NativePeer.initHybrid(java.lang.String, java.util.Map)
02-23 16:41:41.183: E/lNetworkService(25151):   instance fields (1 entries):
02-23 16:41:41.183: E/lNetworkService(25151):      0: com.facebook.jni.HybridData org.pytorch.executorch.NativePeer.mHybridData
02-23 16:41:41.183: E/lNetworkService(25151): Failed to register native method org.pytorch.executorch.NativePeer.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData; in /data/data/com.android.contextq/code_cache/.overlay/base.apk/classes14.dex
02-23 16:41:41.184: E/log(25151): error java.lang.NoSuchMethodError: no static or non-static method "Lorg/pytorch/executorch/NativePeer;.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData;"
02-23 16:41:41.185: A/lNetworkService(25151): java_vm_ext.cc:616] JNI DETECTED ERROR IN APPLICATION: JNI NewGlobalRef called with pending exception java.lang.NoSuchMethodError: no static or non-static method "Lorg/pytorch/executorch/NativePeer;.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData;"
02-23 16:41:41.185: A/lNetworkService(25151): java_vm_ext.cc:616] (Throwable with no stack trace)
02-23 16:41:41.185: A/lNetworkService(25151): java_vm_ext.cc:616] 
02-23 16:41:41.185: A/lNetworkService(25151): java_vm_ext.cc:616]     in call to NewGlobalRef
02-23 16:41:41.185: A/lNetworkService(25151): java_vm_ext.cc:616]     from java.lang.String java.lang.Runtime.nativeLoad(java.lang.String, java.lang.ClassLoader, java.lang.Class)

cc @kirklandsign @cbilgin @mergennachin @byjlw

[guangy10]

Can you please include your dev env and commit hash on main, in case developers need to reproduce the issue?

[kirklandsign]

Hi @adonnini

Currently the demo app doesn't copy java sources (modules) (e.g. NativePeer) but use the AAR instead

executorch/examples/demo-apps/android/LlamaDemo/setup.sh

Line 21 in f1a7d60

cp "$BUILD_AAR_DIR/executorch.aar" "$BASEDIR"/app/libs/executorch.aar

and the AAR contains .so and .jar (which contains all compiled java file)

And you can include AAR in your project by

executorch/extension/benchmark/android/benchmark/app/build.gradle.kts

Line 38 in f1a7d60

implementation(files("libs/executorch.aar"))

[adonnini]

Hi @guangy10 ,
I do not have a commit hash for this. I am not sure which commit hash you are referring to. Could you please clarify?
My development environment:

• Debian bookworm
• Android Studio ladybug patch 2
  Do you need any other information regarding my development environment?
  Thanks

[adonnini]

@kirklandsign Thanks.
I will add the dependency. Where can I find a copy of the .aar? There is no libs directory in executorch/extension/benchmark/android/benchmark/app
Did I look in the wrong place? Or, do I need to just build the benchmark app?
Sorry about these questions. Thanks,

P.S. Is the information in your message anywhere in the documentation? Could you please send me a link, if there is? Thanks

[adonnini]

Hi, sorry about this. I couldn't find setup.sh I the llama demo directory. I didn't realize that the llama set-up applies also to non llama apps. Does it?Again sorry about my clumsiness Thanks.On Feb 25, 2025 7:25 AM, Hansong ***@***.***> wrote: Hi @adonnini Currently the demo app doesn't copy java sources (modules) (e.g. NativePeer) but use the AAR instead https://github.com/pytorch/executorch/blob/f1a7d60e823104d1753b114656f798334fc728f3/examples/demo-apps/android/LlamaDemo/setup.sh#L21 and the AAR contains .so and .jar (which contains all compiled java file)—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***> kirklandsign left a comment (pytorch/executorch#8665) Hi @adonnini Currently the demo app doesn't copy java sources (modules) (e.g. NativePeer) but use the AAR instead https://github.com/pytorch/executorch/blob/f1a7d60e823104d1753b114656f798334fc728f3/examples/demo-apps/android/LlamaDemo/setup.sh#L21 and the AAR contains .so and .jar (which contains all compiled java file) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[adonnini]

@kirklandsign

I apologize in advance for the (very) long message. Please read it to the end.

I tried running setup.sh from both the LlamaDemo and ExecutorchDemo directories. They both failed.

• setup.sh in ExecutorchDemo directory failed with this error:

CMake Error: The source directory "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPredictionContextQ/executorch/examples/demo-apps/android/ExecuTorchDemo" does not appear to contain CMakeLists.txt.

I checked. There is no CMakeLists.txt in the directory

• setup.sh in lLamaDemo directory fails with this error:

/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPredictionContextQ/executorch/examples/demo-apps/android/LlamaDemo/../../../../build/build_android_llm_demo.sh: line 11: pushd: extension/android: No such file or directory

the executorch/extension/android does exist.

In setting up exeutorch I followed the instruction in
https://pytorch.org/executorch/stable/getting-started-setup.html
Everything worked with no errors.

However, I must have done something wrong, not sure what. I am not familiar shell scripting. I did not make any changes to the two scripts.

As a workaround, I ran download_prebuilt_lib.sh found in the LlamaDemo directory. Then, I copied the executorch.aar which was downloaded to the libs folder in my application and added the dependency.

Android Studio reports the following error:

Cannot resolve method 'toTensor' in 'EValue'

in this line

outputTensor = mModule.forward(EValue.from(arrDataPytorch)).toTensor();

Note that there is a line like this in
executorch/examples/demo-apps/android/ExecuTorchDemo/app/src/main/java/com/example/executorchdemo/MainActivity.java
Tensor outputTensor = mModule.forward(EValue.from(inputTensor))[0].toTensor();

I also checked the contents of EValue.class in executorch.aar. It does contain a toTensor method.

However, application build was successful.

When I ran the application, execution still failed when attempting to load the module. Please see the error log below.

Is the executorch.aar I downloaded not the right one for non-lLama applications?

Please let me know what I should do next.

Thanks

ERROR LOG

02-25 16:04:43.057: W/lNetworkService(22959): CheckJNI: method to register "initHybrid" not in the given class. This is slow, consider changing your RegisterNatives calls.
02-25 16:04:43.060: E/lNetworkService(22959): ----- class 'Lorg/pytorch/executorch/NativePeer;' cl=0x14013570 -----
02-25 16:04:43.060: E/lNetworkService(22959):   objectSize=240 (224 from super)
02-25 16:04:43.060: E/lNetworkService(22959):   access=0x0000.0000
02-25 16:04:43.060: E/lNetworkService(22959):   super='java.lang.Class<java.lang.Object>' (cl=0x0)
02-25 16:04:43.060: E/lNetworkService(22959):   interfaces (1):
02-25 16:04:43.060: E/lNetworkService(22959):      0: java.lang.Class<org.pytorch.executorch.INativePeer> (cl=0x14013570)
02-25 16:04:43.060: E/lNetworkService(22959):   vtable (2 entries, 11 in super):
02-25 16:04:43.060: E/lNetworkService(22959):      0: org.pytorch.executorch.EValue org.pytorch.executorch.NativePeer.forward(org.pytorch.executorch.EValue[])
02-25 16:04:43.060: E/lNetworkService(22959):      1: void org.pytorch.executorch.NativePeer.resetNative()
02-25 16:04:43.060: E/lNetworkService(22959):   direct methods (3 entries):
02-25 16:04:43.060: E/lNetworkService(22959):      0: void org.pytorch.executorch.NativePeer.<clinit>()
02-25 16:04:43.060: E/lNetworkService(22959):      1: void org.pytorch.executorch.NativePeer.<init>(java.lang.String, java.util.Map)
02-25 16:04:43.060: E/lNetworkService(22959):      2: com.facebook.jni.HybridData org.pytorch.executorch.NativePeer.initHybrid(java.lang.String, java.util.Map)
02-25 16:04:43.060: E/lNetworkService(22959):   instance fields (1 entries):
02-25 16:04:43.060: E/lNetworkService(22959):      0: com.facebook.jni.HybridData org.pytorch.executorch.NativePeer.mHybridData
02-25 16:04:43.060: E/lNetworkService(22959): Failed to register native method org.pytorch.executorch.NativePeer.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData; in /data/app/~~S2GUziV5tYf4zysXextUOw==/com.android.contextq-TWkeFJwpSdRUaY6I1r-k_A==/base.apk!classes14.dex
02-25 16:04:43.061: E/log(22959): error java.lang.NoSuchMethodError: no static or non-static method "Lorg/pytorch/executorch/NativePeer;.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData;"
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616] JNI DETECTED ERROR IN APPLICATION: JNI NewGlobalRef called with pending exception java.lang.NoSuchMethodError: no static or non-static method "Lorg/pytorch/executorch/NativePeer;.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData;"
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616] (Throwable with no stack trace)
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616] 
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616]     in call to NewGlobalRef
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616]     from java.lang.String java.lang.Runtime.nativeLoad(java.lang.String, java.lang.ClassLoader, java.lang.Class)
02-25 16:04:43.250: A/lNetworkService(22959): runtime.cc:708] Runtime aborting...

[guangy10]

@kirklandsign

I apologize in advance for the (very) long message. Please read it to the end.

I tried running setup.sh from both the LlamaDemo and ExecutorchDemo directories. They both failed.

• setup.sh in ExecutorchDemo directory failed with this error:

CMake Error: The source directory "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPredictionContextQ/executorch/examples/demo-apps/android/ExecuTorchDemo" does not appear to contain CMakeLists.txt.

I checked. There is no CMakeLists.txt in the directory

• setup.sh in lLamaDemo directory fails with this error:

/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPredictionContextQ/executorch/examples/demo-apps/android/LlamaDemo/../../../../build/build_android_llm_demo.sh: line 11: pushd: extension/android: No such file or directory

the executorch/extension/android does exist.

In setting up exeutorch I followed the instruction in https://pytorch.org/executorch/stable/getting-started-setup.html Everything worked with no errors.

However, I must have done something wrong, not sure what. I am not familiar shell scripting. I did not make any changes to the two scripts.

As a workaround, I ran download_prebuilt_lib.sh found in the LlamaDemo directory. Then, I copied the executorch.aar which was downloaded to the libs folder in my application and added the dependency.

Android Studio reports the following error:

Cannot resolve method 'toTensor' in 'EValue'

in this line

outputTensor = mModule.forward(EValue.from(arrDataPytorch)).toTensor();

Note that there is a line like this in executorch/examples/demo-apps/android/ExecuTorchDemo/app/src/main/java/com/example/executorchdemo/MainActivity.java Tensor outputTensor = mModule.forward(EValue.from(inputTensor))[0].toTensor();

I also checked the contents of EValue.class in executorch.aar. It does contain a toTensor method.

However, application build was successful.

When I ran the application, execution still failed when attempting to load the module. Please see the error log below.

Is the executorch.aar I downloaded not the right one for non-lLama applications?

Please let me know what I should do next.

Thanks

ERROR LOG

02-25 16:04:43.057: W/lNetworkService(22959): CheckJNI: method to register "initHybrid" not in the given class. This is slow, consider changing your RegisterNatives calls.
02-25 16:04:43.060: E/lNetworkService(22959): ----- class 'Lorg/pytorch/executorch/NativePeer;' cl=0x14013570 -----
02-25 16:04:43.060: E/lNetworkService(22959):   objectSize=240 (224 from super)
02-25 16:04:43.060: E/lNetworkService(22959):   access=0x0000.0000
02-25 16:04:43.060: E/lNetworkService(22959):   super='java.lang.Class<java.lang.Object>' (cl=0x0)
02-25 16:04:43.060: E/lNetworkService(22959):   interfaces (1):
02-25 16:04:43.060: E/lNetworkService(22959):      0: java.lang.Class<org.pytorch.executorch.INativePeer> (cl=0x14013570)
02-25 16:04:43.060: E/lNetworkService(22959):   vtable (2 entries, 11 in super):
02-25 16:04:43.060: E/lNetworkService(22959):      0: org.pytorch.executorch.EValue org.pytorch.executorch.NativePeer.forward(org.pytorch.executorch.EValue[])
02-25 16:04:43.060: E/lNetworkService(22959):      1: void org.pytorch.executorch.NativePeer.resetNative()
02-25 16:04:43.060: E/lNetworkService(22959):   direct methods (3 entries):
02-25 16:04:43.060: E/lNetworkService(22959):      0: void org.pytorch.executorch.NativePeer.<clinit>()
02-25 16:04:43.060: E/lNetworkService(22959):      1: void org.pytorch.executorch.NativePeer.<init>(java.lang.String, java.util.Map)
02-25 16:04:43.060: E/lNetworkService(22959):      2: com.facebook.jni.HybridData org.pytorch.executorch.NativePeer.initHybrid(java.lang.String, java.util.Map)
02-25 16:04:43.060: E/lNetworkService(22959):   instance fields (1 entries):
02-25 16:04:43.060: E/lNetworkService(22959):      0: com.facebook.jni.HybridData org.pytorch.executorch.NativePeer.mHybridData
02-25 16:04:43.060: E/lNetworkService(22959): Failed to register native method org.pytorch.executorch.NativePeer.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData; in /data/app/~~S2GUziV5tYf4zysXextUOw==/com.android.contextq-TWkeFJwpSdRUaY6I1r-k_A==/base.apk!classes14.dex
02-25 16:04:43.061: E/log(22959): error java.lang.NoSuchMethodError: no static or non-static method "Lorg/pytorch/executorch/NativePeer;.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData;"
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616] JNI DETECTED ERROR IN APPLICATION: JNI NewGlobalRef called with pending exception java.lang.NoSuchMethodError: no static or non-static method "Lorg/pytorch/executorch/NativePeer;.initHybrid(Ljava/lang/String;I)Lcom/facebook/jni/HybridData;"
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616] (Throwable with no stack trace)
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616] 
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616]     in call to NewGlobalRef
02-25 16:04:43.061: A/lNetworkService(22959): java_vm_ext.cc:616]     from java.lang.String java.lang.Runtime.nativeLoad(java.lang.String, java.lang.ClassLoader, java.lang.Class)
02-25 16:04:43.250: A/lNetworkService(22959): runtime.cc:708] Runtime aborting...

Thank you for reporting the issue with detailed steps for repro. The Android apps are built and running continuously in our CI, so likely what you see is due to some setup issue. To help us better understand your setup, you can attach you dev env by running the following commands:

wget https://github.com/raw/pytorch/pytorch/main/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

[kirklandsign]

Hi @adonnini

I tried running setup.sh from both the LlamaDemo and ExecutorchDemo directories. They both failed.

Please run from root directory.

For prebuilt AAR, maybe use this 0.5 release?

Docs: https://pytorch.org/executorch/0.5/android-prebuilt-library.html#using-prebuilt-libraries

BTW the AAR is same for both llama and non-llama. The docs above is not updated. I will update docs.