gstreamer issues (video playback jittery)

[redargon]

First of all, thanks ardera for your work on flutter-pi and for implementing gstreamer (the omxplayer is really hard to get working on new raspberry pi builds these days).

I am getting some error messages from gstreamer, the video playback is jittery and controls (play/pause/seek etc.) don't seem to work.

I'm running:
flutter 2.10.1
flutter-pi latest commit
Raspberry Pi OS 'Bullseye' 64 bit

on a:
Raspberry Pi 400
iiyama ProLite TF2415MC-B2 touch screen

If you would like to see my config.txt and cmdline.txt files, just let me know and I will post them too, but as far as I know mostly defaults.

error messages:

[gstreamer video_player plugin] respond success on_create
[gstreamer video_player plugin] on_receive_evch
[gstreamer video player] gstreamer error: code: 7, domain: gst-stream-error-quark, msg: No valid frames decoded before end of stream (debug info: ../gst-libs/gst/video/gstvideodecoder.c(1246): gst_video_decoder_sink_event_default (): /GstPipeline:pipeline0/GstURIDecodeBin:src/GstDecodeBin:decodebin0/v4l2h264dec:v4l2h264dec0:
no valid frames found)
[gstreamer video_player] Hardware decoder failed. Falling back to software decoding...
[gstreamer video_player plugin] Got video info: stream? no, w x h:  1920 x  1080, duration: 0:00:14.740000000

(flutter-pi:1068): GStreamer-CRITICAL **: 14:27:09.468: gst_event_new_seek: assertion 'rate != 0.0' failed

(flutter-pi:1068): GStreamer-CRITICAL **: 14:27:09.469: gst_element_send_event: assertion 'event != NULL' failed
[gstreamer video_player] Could not set the new playback speed / playback position (speed: 0.000000, pos: 0:00:00.001025980).

(flutter-pi:1068): GStreamer-CRITICAL **: 14:27:20.759: gst_event_new_seek: assertion 'rate != 0.0' failed

(flutter-pi:1068): GStreamer-CRITICAL **: 14:27:20.759: gst_element_send_event: assertion 'event != NULL' failed
[gstreamer video_player] Could not set the new playback speed / playback position (speed: 0.000000, pos: 0:00:11.291245185).```

[redargon]

Update:
The jittering appears only to happen with larger resolution files where the hardware decoder failed.
The same video encoded with h.264 at:
480p hardware decoder ok, no jitter (all 4 cores of CPU running at about 20%)
720p hardware decoder failed, no jitter (all 4 cores of CPU running at about 30%)
1080p hardware decoder failed, jitter(dropped frames) (all 4 cores of CPU running at about 60%)

I also tried using chewie controller and standard video_player controller and same gstreamer-critical errors occur and video controls don't function.

[ardera]

The hardware decoding failing is a known problem, see #224, I'm currently looking into that. From what I can tell it's not related to resolution or anything. I have two mp4 example videos, both using h264, same resolution and somehow one works while the other doesn't.

The controls should still work with software decoding though (except for 4k, that's problematic), do they not work at all or are they just not as responsive?

[redargon]

Thanks Ardera, I had a look at #244 and am currently investigating which encoding types and tools work best to allow for hardware decoding.

The controls do not work at all, for example if I try to play or stop video playback using video_player commands, like _controller.play() or controller.pause(), then I get these log messages:

(flutter-pi:1034): GStreamer-CRITICAL **: 14:35:32.001: gst_event_new_seek: assertion 'rate != 0.0' failed

(flutter-pi:1034): GStreamer-CRITICAL **: 14:35:32.001: gst_element_send_event: assertion 'event != NULL' failed
[gstreamer video_player] Could not set the new playback speed / playback position (speed: 0.000000, pos: 0:00:00.000000000).

(flutter-pi:1034): GStreamer-CRITICAL **: 14:35:32.003: gst_event_new_seek: assertion 'rate != 0.0' failed

(flutter-pi:1034): GStreamer-CRITICAL **: 14:35:32.003: gst_element_send_event: assertion 'event != NULL' failed
[gstreamer video_player] Could not set the new playback speed / playback position (speed: 0.000000, pos: 0:00:00.000000000).

Would you prefer to close this issue (as the description is more about video jitter which you are following up in #244) and I open a new issue for control problems only?

[ardera]

This should be fixed now, now that the hw decoding problem is fixed (and one texture bug that lead to high display latency)