VC4-KMS-V3D Screen tearing artifacts in video player and OpenGL apps

[qrp73]

The bug description

First I notice that there is ugly screen tearing during video playback or running OpenGL application. I found this discussion: https://forums.raspberrypi.com/viewtopic.php?t=246291 they suggested to disable xcompmgr composition manager in raspi-config. I disabled it and at first look screen tearing disappears. But later I found that it is still here, just less noticeable.

So I wrote a small test app for testing vblank wait behavior of the video driver. With this app I found that screen tearing is still here, but its position depends on rendering window size and position. In a fullscreen mode screen tearing disappears, but it may be just due to specific conditions, needs to test it on different display resolution.

At a glance it looks that the video driver performs buffers swap with some unacceptable delay relative to hardware vblank event, so it happens in the middle of video buffer scan and as result, the display frame is teared and contains mixed content combined from several video buffers.

Steps to reproduce the behaviour

1. create file test-vblank.c and fill it with following program:

#include <stdio.h>
#include <time.h>
#include <GL/freeglut.h>

// gcc test-vblank.c -o test-vblank -Wall -lGL -lglut

void fillRect(float x, float y, float w, float h)
{
    GLfloat rect[] = { x, y+h, x+w, y+h, x+w, y, x, y };
    glEnableClientState(GL_VERTEX_ARRAY);
    glVertexPointer(2, GL_FLOAT, 0, rect);
    glDrawArrays(GL_TRIANGLE_FAN, 0, 4);    
}

char _buf[128];
int _counter = 0;
int _cnt1 = 0;
struct timespec _ts1, _ts2;
float _fps = 0;
#define TS2NS(ts) (((uint64_t)ts.tv_sec)*1e9 + (uint64_t)ts.tv_nsec)

void displayFunc(void)
{
    // calculate fps
    timespec_get(&_ts2, TIME_UTC);
    _counter++;
    uint64_t dt_ns = TS2NS(_ts2) - TS2NS(_ts1);
    if (dt_ns > 1e9)
    {
        // update fps every second
        int frameCount = _counter - _cnt1;
        _ts1 = _ts2;
        _cnt1 = _counter;
        _fps = (frameCount*1e9) / (float)dt_ns;
    }
    
    glLoadIdentity();
    glClearColor(0.0f,0.0f,0.0f,1.0f);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    // fill rectangle with different color on every next frame (multicolor effect)
    // it helps to detect inconsistency or instability of vblank wait and buffer update 
    switch (_counter & 3)
    {
        case 0: glColor3f(0.1f, 0.1f, 0.1f); break;
        case 1: glColor3f(1.0f, 1.0f, 1.0f); break;
        case 2: glColor3f(0.1f, 0.1f, 1.0f); break;
        case 3: glColor3f(0.0f, 1.0f, 0.0f); break;
    }
    fillRect(-1.0f, -1.0f, 2, 2);
    
    // print resolution, bpp and fps
    sprintf(_buf, "%dx%d:%d\n%.1f fps\n", 
        glutGet(GLUT_WINDOW_WIDTH), 
        glutGet(GLUT_WINDOW_HEIGHT), 
        glutGet(GLUT_WINDOW_BUFFER_SIZE),
        _fps);
    glColor4f(1.0f, 0.0f, 0.0f, 1.0f);
    glPushMatrix();
    glTranslatef(-0.95f, 0.85f, 0.0f);
    glScalef(0.001,0.001,1);
    glutStrokeString(GLUT_STROKE_ROMAN, (const unsigned char*)_buf);
    glPopMatrix();

    // swap buffers and request glut for next display() call for continuous update
    glutSwapBuffers();
    glutPostRedisplay();
}

int _isFullscreen;
int _wndWidth, _wndHeight;
void keyboardFunc(unsigned char key, int x, int y) 
{
    int mod = glutGetModifiers(); //1=SHIFT, 2=CTRL, 4=ALT
    if (key == ' ' && mod == 0)
    {
        // Use space key to switch fullscreen/window mode
        _isFullscreen ^= 1;
        if (_isFullscreen)
        {
            _wndWidth = glutGet(GLUT_WINDOW_WIDTH); 
            _wndHeight = glutGet(GLUT_WINDOW_HEIGHT); 
            glutFullScreen();
        }
        else
        {
            glutReshapeWindow(_wndWidth, _wndHeight);
        }
    }
}


int main(int argc, char** argv)
{
    glutInit(&argc, argv);
    glutInitDisplayMode(GLUT_DOUBLE | GLUT_RGBA);

    glutInitWindowSize(800, 600);
    glutInitWindowPosition(0, 0);
    glutCreateWindow("test-vblank");

    glutDisplayFunc( displayFunc );
    glutKeyboardFunc( keyboardFunc );

    // explicit matrices init
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();
    
    glLineWidth(2);         // use bold lines for glutStrokeString
    
    glutMainLoop();
    return 0;
}

2. compile it with a command line gcc test-vblank.c -o test-vblank -Wall -lGL -lglut
3. run it with ./test-vblank
4. try to move window position/size and notice that at some position/size screen tearing appears and you can see video artifacts on the display. Use button SPACE to switch between windowed and fullscreen mode for testing different modes.

The expected result: the window is flickering at constant and stable speed and doesn't contains video artifacts, it should change color for entire window surface simultaneously on every video frame. The color of windows surface should be solid with no artifacts.

The actual result: the window contains video artifacts, the display shows mixed frame which contains parts from several video buffers.

Note: you cannot take screenshot of these video artifacts, because it appears on hardware display, but not in the video buffer. So all screenshot will be good, but the displayed picture will contains video artifacts.

Device (s)

Raspberry Pi 4 Mod. B

System

$ cat /etc/rpi-issue
Raspberry Pi reference 2023-05-03
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 7c750947a959fb626a70c09fd17c65815df192ac, stage4

$ vcgencmd version
Mar 17 2023 10:50:39
Copyright (c) 2012 Broadcom
version 82f3750a65fadae9a38077e3c2e217ad158c8d54 (clean) (release) (start)

$ uname -a
Linux raspi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

Logs

N/A

Additional context

glxinfo:
OpenGL renderer string: V3D 4.2
OpenGL version string: 2.1 Mesa 20.3.5

See photo for details:

It looks that this bug happens only in window mode, because I cannot reproduce it in fullscreen video mode. The fullscreen mode definitely has a different behavior, because it also shows much higher fps with vblank_mode=0 and no tearing (even with no vsync)...

Screen tearing happens for both cases - when xcompmgr is enabled or disabled. But tearing position is different - it depends on xcompmgr state. Also when xcompmgr is enabled, screen tearing happens even in fullscreen mode.

[qrp73]

just tried it on 32-bit raspi os - the same bug, also catch tearing in fullscreen mode, so on 32-bit OS this bug is even worse...

[qrp73]

just tried it on ubuntu arm64 23.04, and screen tearing issue is missing from ubuntu, it shows all frames with no tearing. But unfortunately ubuntu has much slower peroformance than Raspi OS, so I returned back. Hope screen tearing will be fixed, because it is very noticeable when playing video, opengl app or even when scrolling text in the browser...

[popcornmix]

This is unlikely to be a kernel issue, so this isn't the right place for the report.

Are you using mutter? My understanding is that is required to avoid tearing, but due to increased memory use, is only enabled on Pi4 with 2GB or more. What is yours?

You can check with:

pi@pi4:~ $ ps aux | grep mutter | grep -v grep
pi           874  2.1  1.8 664172 71440 ?        Sl   11:33   0:01 mutter

[qrp73]

On my machine it gives empty output.
I think this issue is due to some bug in video driver and/or kernel thread scheduler.

I tested ubuntu and found that it doesn't have such screen tearing, but on the other hand ubuntu has a lot of frame drops, just 40 hardware frames of 60 is displayed on ubuntu, while raspberry pi os shows all 60 frames per second, but with screen tearing.

Also tried to switch to fkms driver, it has the same screen tearing. I notice that glmark2 crashes when started with fkms driver. With kms driver glmark2 works ok and shows result 355.

[popcornmix]

but due to increased memory use, is only enabled on Pi4 with 2GB or more. What is yours?

(i.e. what does vcgencmd get_mem arm report?)

[qrp73]

pi@raspi:~ $ vcgencmd get_mem arm
arm=896M
pi@raspi:~ $ vcgencmd get_mem gpu
gpu=128M

I'm using RPi4 with 4GB (revision 1.5, reading from MIDR_EL1 returns 0x410fd083)

Do you have screen tearing with bullseye arm64?

[popcornmix]

Actually vcgencmd get_config total_mem would be a better command.

Have you enabled VNC in raspi-config? That also disables mutter.

systemctl status vncserver-x11-serviced and see if it is active.

[qrp73]

pi@raspi:~ $ vcgencmd get_config total_mem
total_mem=4096

yes I enabled VNC, vncserver is active

[pelwell]

So does it still tear when you disable VNC?

[qrp73]

I tried to disable VNC and screen tearing really disappears (in window mode).

But now screen tearing appears in fullscreen mode. :( Just run test program, press space and wait for 1-2 sec. At beginning, when you switch to a fullscreen mode, screen tearing is missing, but after second or two it appears at the top of screen...

Also I catch strange bug. When I switch to fullscreen and back for several times in test app (see source code in the issue description) it suddenly freeze the system... The window is no more updated and UI system doesn't respond to any user actions anymore. The mouse pointer is still moves and I can switch to a text tty with Ctrl+Alt+F1 and kill process, after killing process the system starts to respond to a user actions.

I can reproduce this bug several times, even after reboot I can reproduce it... When vncserver was active, I didn't seen such system freeze.

Some additional update, freeze happens if I switch from fullscreen mode to windowed mode when screen tearing is started (in fullscreen mode). If I switch it quickly it doesn't lead to a system freeze.

And another finding. When system freeze happens I can resume normal system run just by switch to a text tty with Ctrl+Alt+F1 and then switch back to UI with Ctrl+Alt+F7. The window starts to update again with no need to restart and system starts to respond to user actions...

It looks that some kind of deadlock happens in the thread which waits for vblank event... And when I switch to a text mode and then back it leave that deadlock and system continue to run.

[qrp73]

Also I found that hdmi display detect on the fly doesn't works when vncserver is active. If you connect display to another hdmi port it don't recognize it until reboot. With disabled vncserver it started works on the fly, I can switch hdmi display and it is detected and started to use immediately.

[qrp73]

Also with disabled vncserver chromium-browser starts to works more smoothly, when vncserver was active, it has some lags (short freezes), I thought this is because RPI4 has not enough performance. But now, when I disabled vncserver, it works smoothly... The only issue is these strange freezes when exit from fullscreen mode and that fullscreen mode has screen tearing now. With active vncserver there was no screen tearing in fullscreen mode (they appears in windowed mode).

[vanfanel]

@qrp73 Save your time for coding: you will never get proper tearing-less movement on X11 using the Pi4 video stack.
Use Wayland or KMS/DRM instead. No tearing there and way less latency.
I can recommend labwc as your Wayland compositor. You'll have to build it yourself, but you get good performance (vs the default desktop) and "every frame is perfect" on the Raspberry Pi 4.

Seriously, don't go down the X11 hole, it's pure time wasting. That technology is just broken by design.
If someone tells you he doesn't get tearing in X11 on the Pi, it's simply because some people can't see it.

Why hasn't the foundation moved Pi OS to Wayland, it's a unknown to me. Performance is way better, no tearing in any situation, better input lag using Vulkan + 2 video buffers, etc.
But X11 still plagues the default OS as of 2023.

[popcornmix]

Why hasn't the foundation moved Pi OS to Wayland, it's a unknown to me. Performance is way better, no tearing in any situation, better input lag using Vulkan + 2 video buffers, etc.

Bookworm (currently in beta testing) uses wayland by default.

[vanfanel]

Why hasn't the foundation moved Pi OS to Wayland, it's a unknown to me. Performance is way better, no tearing in any situation, better input lag using Vulkan + 2 video buffers, etc.

Bookworm (currently in beta testing) uses wayland by default.

Hell yeah, that's what I call good news! About time.. :)

What compositor is being used?

[popcornmix]

wayfire/wlroots.

[qrp73]

Where I can get RPI4 image with Bookworm wayfire/wlroots and opengl hw accelerated support for testing?

[popcornmix]

You need to be a beta tester. Or wait for the public release.

[vanfanel]

Or you can build you own: burn latest DietPi image, then build latest stable libDRM, latest stable MESA, latest stable libwayland+wlroots+labwc... here you go! That's what I did and works great.

[qrp73]

Just found that if I switch to dtoverlay=vc4-fkms-v3d in config.txt, the opengl performance remains exactly the same (for both high load scene and lightweight scene), but it leads to much-much worse screen tearing...

I think there is some issue in vc4-kms-v3d and vc4-fkms-v3d drivers which cause screen tearing issue... vc4-kms-v3d works better, but still has some screen tearing issues...

I just wonder - is it open source driver?

[popcornmix]

fkms is closed source and deprectated. It has known issues with tearing (that won't be fixed).

kms is open source and the default. It has no known issues with tearing. But it can be driven in a single buffered way, which will have tearing. That is a decision of the windowing framework/compositor.

[qrp73]

But it can be driven in a single buffered way, which will have tearing.

What this means? Is it uses single buffer for update and at the same time this buffer is scanned by video circuit in parallel?

Is there any way to configure it to use separate video buffers for video update?

There is a setting max_framebuffers=2 in config.txt, so it should use at least 3 video buffers and don't touch the buffer which is currently scanned by video circuit. Isn't it?

How tearing happens if it properly implemented? Because primary buffer should not be updated directly when max_framebuffers=2 or max_framebuffers=1

[popcornmix]

max_framebuffers determines whether a framebuffer is allocated for each of the hdmi displays.
Nothing to do with double/tripe buffering. And it's not used by kms anyway.

[qrp73]

Hm, just tried to set max_framebuffers=0, it leads to increase opengl app performance for +19% and there is no tearing in window mode. Fullscreen mode still has a fixed offset screen tearing at the top of screen.

There is also some bug when I switch between fullscreen mode and window mode, it can lead to broken window appearance and video artefacts. The same bug happens in VLC player. It looks like compositor bug. This is strange, but this bug is missing when VNC is enabled.

[popcornmix]

I've tested openarea with a timedemo on Pi4 and get:
max_framebuffers=2
fullscreen 42.9
windowed 38.6

max_framebuffers=0
fullscreen 42.8
windowed 38.7

so I'm not seeing the performance change you see.

[popcornmix]

There is also some bug when I switch between fullscreen mode and window mode, it can lead to broken window appearance and video artefacts. The same bug happens in VLC player. It looks like compositor bug. This is strange, but this bug is missing when VNC is enabled.

mutter is the default window manager/compositor on Pi4 with 2G or more RAM.
It doesn't play nicely with VNC, so if you enable VNC mutter is disabled.

You can check if mutter is running with:

pi@raspberrypi:~ $ ps aux | grep mutter | grep -v grep
pi           781  0.6  2.2 533788 87380 ?        Sl   13:59   0:07 mutter

when mutter is disabled openbox is the window manager.

[qrp73]

I've tested openarea with a timedemo on Pi4 and get: max_framebuffers=2 fullscreen 42.9 windowed 38.6
max_framebuffers=0 fullscreen 42.8 windowed 38.7
so I'm not seeing the performance change you see.

just tried it again, but cannot reproduce this behavior again, now it gives almost the same fps for max_framebuffers=0/1/2. I remember that I catch it with lightweight scenes, but may be it was effect of some background process.

[qrp73]

mutter is the default window manager/compositor on Pi4 with 2G or more RAM. It doesn't play nicely with VNC, so if you enable VNC mutter is disabled.

it was happened (broken window with garbage) with mutter enabled (VNC disabled). But now I cannot reproduce this bug. Previously it was easy to catch this bug with several window/fullscreen switch, but now it works ok. I'm not sure if it was fixed with some recent updates or due to some changes in my OS configuration, but at the moment it seems that this bug disappears.

[qrp73]

No, I was wrong, it still here. There is need to wait several seconds after switch into fullscreen mode until screen tearing appears at the top of screen, then switch into window mode move window and window layout become broken. But it needs to do it several times to catch it.

Also I found that sometimes opengl apps can suddenly freeze on buffer swap call.

[qrp73]

not relevant anymore