🐛 Black screen no some devices using Frame Processor

[maxximee]

What's happening?

Using frame processor shows black image on emulator and some devices (can't reproduce the real devices, read it from android play store reviews).

With frameProcessor

Without frameProcessor

Note: The image is black when negative is false and white when negative is true, meaning the shader is applied correctly, but the image isn't displayed. Also relevant, the resulting image taken is correct and shows the real picture. It's just the live preview with frame processor that isn't working.

Reproduceable Code

using following shader:


const invertColorsFilter = Skia.RuntimeEffect.Make(`
 uniform shader image;
 half4 main(vec2 pos) {
   vec4 color = image.eval(pos);
   return vec4((1.0 - color).rgb, 1.0);
 }
`)
const shaderBuilder = Skia.RuntimeShaderBuilder(invertColorsFilter)
const imageFilter = Skia.ImageFilter.MakeRuntimeShader(shaderBuilder, null, null)
const paint = Skia.Paint()
const defaultPaint = Skia.Paint()
paint.setImageFilter(imageFilter)

set here:

    const frameProcessor = useSkiaFrameProcessor(frame => {
        'worklet'
        if (negativeColors) {
            frame.render(paint)
        } else {
            frame.render(defaultPaint)
        }
    }, [paint, defaultPaint, negativeColors])

to my component:

 <ReanimatedCamera
                                        style={StyleSheet.absoluteFill}
                                        device={device}
                                        isActive={isActive}
                                        ref={camera}
                                        onInitialized={onInitialized}
                                        onError={onError}
                                        onStarted={() => console.log('Camera started!')}
                                        onStopped={() => console.log('Camera stopped!')}
                                        onPreviewStarted={() => console.log('Preview started!')}
                                        onPreviewStopped={() => console.log('Preview stopped!')}
                                        onOutputOrientationChanged={(o) => console.log(`Output orientation changed to ${o}!`)}
                                        onPreviewOrientationChanged={(o) => console.log(`Preview orientation changed to ${o}!`)}
                                        onUIRotationChanged={(degrees) => console.log(`UI Rotation changed: ${degrees}°`)}
                                        format={format}
                                        fps={30}
                                        photoHdr={photoHdr}
                                        photoQualityBalance="speed"
                                        lowLightBoost={device.supportsLowLightBoost}
                                        enableZoomGesture={false}
                                        animatedProps={cameraAnimatedProps}
                                        exposure={0}
                                        enableFpsGraph={false}
                                        outputOrientation="device"
                                        photo={true}
                                        video={false}
                                        audio={false}
                                        enableLocation={false}
                                        frameProcessor={frameProcessor}
                                    />

### Relevant log output

```shell
2025-01-13 09:30:53.407  5114-5114  unknown:co...agerHelper com.klim.klimfilmscanner             E  Unhandled SoftException
                                                                                                    com.facebook.react.bridge.ReactNoCrashSoftException: Cannot get UIManager because the context doesn't contain an active CatalystInstance.
                                                                                                    	at com.facebook.react.uimanager.UIManagerHelper.getUIManager(UIManagerHelper.java:82)
                                                                                                    	at com.facebook.react.uimanager.UIManagerHelper.getEventDispatcher(UIManagerHelper.java:134)
                                                                                                    	at com.facebook.react.uimanager.UIManagerHelper.getEventDispatcherForReactTag(UIManagerHelper.java:112)
                                                                                                    	at com.mrousavy.camera.react.CameraView_EventsKt.sendEvent(CameraView+Events.kt:173)
                                                                                                    	at com.mrousavy.camera.react.CameraView_EventsKt.invokeOnPreviewStarted(CameraView+Events.kt:46)
                                                                                                    	at com.mrousavy.camera.react.CameraView$createPreviewView$1$1.invoke(CameraView.kt:301)
                                                                                                    	at com.mrousavy.camera.react.CameraView$createPreviewView$1$1.invoke(CameraView.kt:294)
                                                                                                    	at com.mrousavy.camera.react.CameraView.createPreviewView$lambda$2$lambda$1(CameraView.kt:294)
                                                                                                    	at com.mrousavy.camera.react.CameraView.$r8$lambda$ZoRs9LXhNa6nHYEs80y02k4qaAQ(Unknown Source:0)
                                                                                                    	at com.mrousavy.camera.react.CameraView$$ExternalSyntheticLambda1.onChanged(D8$$SyntheticClass:0)
                                                                                                    	at androidx.lifecycle.LiveData.considerNotify(LiveData.java:133)
                                                                                                    	at androidx.lifecycle.LiveData.dispatchingValue(LiveData.java:151)
                                                                                                    	at androidx.lifecycle.LiveData.setValue(LiveData.java:309)
                                                                                                    	at androidx.lifecycle.MutableLiveData.setValue(MutableLiveData.java:50)
                                                                                                    	at androidx.lifecycle.LiveData$1.run(LiveData.java:93)
                                                                                                    	at android.os.Handler.handleCallback(Handler.java:959)
                                                                                                    	at android.os.Handler.dispatchMessage(Handler.java:100)
                                                                                                    	at android.os.Looper.loopOnce(Looper.java:232)
                                                                                                    	at android.os.Looper.loop(Looper.java:317)
                                                                                                    	at android.app.ActivityThread.main(ActivityThread.java:8705)
                                                                                                    	at java.lang.reflect.Method.invoke(Native Method)
                                                                                                    	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:580)
                                                                                                    	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:886)
2025-01-13 09:30:53.537  5114-5114  EGL_emulation           com.klim.klimfilmscanner             E  eglQueryContext 32c0  EGL_BAD_ATTRIBUTE
2025-01-13 09:30:53.538  5114-5114  EGL_emulation           com.klim.klimfilmscanner             E  tid 5114: eglQueryContext(2161): error 0x3004 (EGL_BAD_ATTRIBUTE)
2025-01-13 09:30:53.580  5114-5114  Surface                 com.klim.klimfilmscanner             E  freeAllBuffers: 1 buffers were freed while being dequeued!
2025-01-13 09:30:54.593  5114-5114  Surface                 com.klim.klimfilmscanner             E  getSlotFromBufferLocked: unknown buffer: 0x757fa93df0b0
2025-01-13 09:30:54.603  5114-5114  klimfilmscanner         com.klim.klimfilmscanner             E  [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:30:55.175  5114-5114  klimfilmscanner         com.klim.klimfilmscanner             E  [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:30:55.227  5114-5114  klimfilmscanner         com.klim.klimfilmscanner             E  [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:30:59.141  5114-5114  klimfilmscanner         com.klim.klimfilmscanner             E  [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:31:02.147  5114-5114  klimfilmscanner         com.klim.klimfilmscanner             E  [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:31:02.521  5114-5114  klimfilmscanner         com.klim.klimfilmscanner             E  [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "limited",
  "maxZoom": 1,
  "minZoom": 1,
  "maxExposure": 6,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "ultra-wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 0,
  "minExposure": -6,
  "name": "10 (BACK) androidx.camera.camera2",
  "hasFlash": false,
  "hasTorch": false,
  "position": "back",
  "id": "10"
}

Device

android emulator with API 35 and "VirturalScene"

VisionCamera Version

4.6.3 with skia 1.7.0

Can you reproduce this issue in the VisionCamera Example app?

No, I cannot reproduce the issue in the Example app

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[maintenance-hans]

Guten Tag, Hans here 🍻.

Thanks for the detailed report! It looks like you're experiencing an issue with the frame processor displaying a black image on some devices. However, mrousavy will need more relevant logs to investigate this further, particularly runtime logs from adb logcat.

You can gather those by running adb logcat in your terminal while the app is open on the device or emulator. This will help us troubleshoot the issue more effectively.

Feel free to update the issue once you have the logs!

Note: If you think I made a mistake, please ping @mrousavy to take a look.

[maxximee]

@mrousavy updated the issue with relevant adb logs pointing to the exception

[maxximee]

@mrousavy According wcandillon fromSkia it's an issue related to devices who are using Vulkan and Angle for OpenGL support, including a lot of samsung devices.
It was fixed in skia 1.7.7, maybe updating skia from 1.4.2 to 1.7.7 (or higher) would fix all those black screen issues?

[mrousavy]

1. Please stop pinging me, I am not free support.
2. Skia is a * peerDependency - it is not pinned to a version. If you install 1.7.7., VisionCamera will use Skia 1.7.7. - not 1.4.2.

[cleber-dev]

Has anyone managed to solve this problem?

[luicfrr]

In some tests I made black screen is related to canvas translation:

react-native-vision-camera/package/src/skia/useSkiaFrameProcessor.ts

Lines 98 to 99 in b194ef1

	canvas.translate(frame.height, frame.width)
	canvas.rotate(270, 0, 0)

If I use canvas.translate(0, frame.width) black screen is solved but preview is inverted (left is right and right is left).
Also, drawing on frame is buggy too because seems the frame is -90deg rotated while on IOS it works fine

EDIT:
To fix black screen + mirroring on android I changed this:

react-native-vision-camera/package/src/skia/useSkiaFrameProcessor.ts

Lines 98 to 99 in b194ef1

	canvas.translate(frame.height, frame.width)
	canvas.rotate(270, 0, 0)

To this:

canvas.scale(1, -1)
canvas.translate(frame.height, -frame.width)
canvas.rotate(90, 0, 0)

But this makes drawing on frame with skia 90deg rotated...

[maxximee]

Nothing works on my side, also I logged and am getting 'landscape-right' by default, not landscape-left.
Black screen on some samsung devices like Pixel 9 with api 35 and crash on samsung s8.

[seppemp]

I also experience crashes using the Pixel 7 Pro and 6 with API 35 and frame processors, but I haven't been able to resolve the issue. Has something worked for anyone?

[maxximee]

I also experience crashes using the Pixel 7 Pro and 6 with API 35 and frame processors, but I haven't been able to resolve the issue. Has something worked for anyone?

The issue is still there, lots of issues with skia frame processor (useSkiaFrameProcessor). The bug is open in the rn skia side, but hasn't been looked at yet.