Openmax AVC Override encode capabilities not supported or not implemented

[F5OEO]

Setting a OMX_IndexConfigEncLevelExtension with H264 profile / Level parameters hang out.
For example , trying to put Levl 4.2 High Profile :
Parameter<OMX_VIDEO_CONFIG_LEVEL_EXTEND> ProfileExtended;
ProfileExtended->nPortIndex=OPORT;
ProfileExtended->nCustomMaxMBPS=522240; //4.2
ProfileExtended->nCustomMaxFS=8704;
ProfileExtended->nCustomMaxBRandCPB=62500;

Setting this return no error , but hang as soon as we inquire OMX_FillThisBuffer.
: 0x80001018 OMX_ErrorIncorrectStateOperation

[6by9]

Do you have a simple test app that you can share to illustrate this? Saves us wasting time creating our own test case to investigate this?
NB Level 4.2 is not officially supported as the hardware is only rated for level 4.0. We allow you to request it, but may not keep up with realtime without overclocking.

[F5OEO]

Same issue with MMAL :
in Raspivid.c insert after line
https://github.com/raspberrypi/userland/blob/master/host_applications/linux/apps/raspicam/RaspiVid.c#L2139

MMAL_PARAMETER_VIDEO_LEVEL_EXTENSION_T paramExt;
    paramExt.hdr.id=MMAL_PARAMETER_VIDEO_LEVEL_EXTENSION;
    paramExt.hdr.size = sizeof(paramExt);
    //Levl 4 Definition 
    paramExt.custom_max_mbps=245760;
    paramExt.custom_max_fs=8192;
    paramExt.custom_max_br_and_cpb=25000;

    status = mmal_port_parameter_set(encoder_output, &paramExt.hdr);
      if (status != MMAL_SUCCESS)
      {
         vcos_log_error("Unable to set Level Extension");
         goto error;
      }

Invoke raspivid with Level parameter

./raspivid -w 1280 -h 720 -b 2000000 -o test.264 -lev 4 -pf high -fps 25 -ih -v -g 10

You should see something like

Encoder component done
Starting component connection stage
Connecting camera preview port to preview input port
Starting video preview
Connecting camera video port to encoder input port
mmal: mmal_vc_port_enable: failed to enable port vc.ril.video_encode:in:0(OPQV): EINVAL
mmal: mmal_port_enable: failed to enable connected port (vc.ril.video_encode:in:0(OPQV))0x14b9d60 (EINVAL)
mmal: mmal_connection_enable: output port couldn't be enabled
mmal: main: Failed to connect camera video port to encoder input
mmal: Argument is invalid

Important note using vcdbg shows that custom_max_br_and_cpb is the bad parameter.

[6by9]

Checking the firmware code, it validates the provided parameters and is checking:

1. max fs <= ((1920/16) * (1080/16))
2. max_bps <= ((1920/16)*(1080/16)) * 30
3. max_br <= 2000

Spot the obvious error in 3, and the nice rounding error in 1 (and then inherently in 2).
I'll fix those up, allowing up to 62.5Mbps on max_br.

NB I'll get views from others here, but as it stands max max_fs is going to be 8160 and not 8192 as specified in the AVC Level 4 spec. I still haven't tracked down exactly where the limitation comes from, but widths > 1920 fail in the hardware. I guess you could go for something strange in aspect ratio and use all 8192 macroblocks without exceeding that width restriction, but it would be a weird case.

[F5OEO]

Thx for this correction.

My prior goal was to reduce MaxBitrate to reduce Codec Picture Buffer. By reducing it (below 2M) doesn't seem to affect CPB and IDR Picture is always around 4 times the average bitrate. I can see Buffer occupancy with debug, but not a chance for now to reduce it.

• OMX_IndexParamBrcmVideoPeakRate doesn't either to affect that.

Bitrate on P picture is constant and excellent, but as soon as there is an I picture there is a high peak.

Is there a way to reduce CPB buffer or Window time size for buffer occupancy algorithm ?

[F5OEO]

Reading the header of H264 produced, I see :

profile=66, bitrate=3500000, framerate=25, i_period=100, i_period_t=0, rc_type=0, n_slice=1, mbrps=0, t_scale=1000000, max_slice_bits=0, refresh_mode=0, pvr=0, par=0, cir_mbs=0, air_mbs=1, air_th=512, buf_delay=1.000, immutable=0, min_qp=20, max_qp=51, ext_me=0, ddfi=0, ifr_qp_weight=2, hp_cab=0, constrained=0, 4byte=1, dual_ref=0, rc_slice_dqp=100, rc_bu_size=1

Some parameters are settable with OpenMax/MMAL :

• profile : OMX_IndexParamVideoProfileLevelCurrent
• bitrate ,framerate : portdef
• iperiod : OMX_IndexConfigVideoAVCIntraPeriod
• i_period_t : OMX_IndexConfigBrcmVideoIntraPeriodTime
• rc_type : OMX_IndexParamVideoBitrate (Variable/constant bitrate)
• n_slice : n slice by frame
• mbrps : macroblocks ?
• t_scale : ?
• refresh_mode : IntraRefreshType with cir_mbs, air_mbs
• air_th : ????
• buf_delay : is it the window Rate Control Delay ?
• immutable : Simple or double buffer
• min_qp, max_qp : OMX_IndexParamBrcmVideoEncodeMinQuant/OMX_IndexParamBrcmVideoEncodeMaxQuant
• ext_me, ddfi : ??
• rc_bu_size : rate control buffer size : Where can we set this important thing ???

Thanks by advance @6by9 if you can complete missing and question .

[6by9]

I'm guessing you're looking at the SEI metadata string as those don't appear to be direct H264 header parameters but values passed in to our encoder (particularly the immutable flag).
Reading out of the source.

• mbrps : macroblock rows per slice. OMX_IndexConfigBrcmVideoEncoderMBRowsPerSlice
• t_scale : timestamp_timescale
• air_th : Adaptive IntraRefresh threshold.
• buf_delay : initial_buffer_delay
• immutable : OMX_IndexParamBrcmImmutableInput. Treat source images as immutable, or use the buffer for reconstructed images.
• ext_me: alternate reference frame option.
• ddfi : disable_deblocking_filter_idc
• rc_bu_size : rate control basic unit. Does something in some of the rate control algorithms, but not clear what.

Bitrate on P picture is constant and excellent, but as soon as there is an I picture there is a high peak.
Is there a way to reduce CPB buffer or Window time size for buffer occupancy algorithm

Not that I can see exposed quickly out via IL or MMAL. There is a codec API property for VIDEO_ENCODER_IFRAME_QP_WEIGHT which may do what you need - "The weight factor used to reduce QP for I-frames (compared with the previous GOP average)". Leave this open and I'll try to find 15mins to plumb it through. No guarantees it does anything actually useful, although it is used in a function called venc_rc_get_qp_for_iframe

[F5OEO]

Thx for all this information and details. Love to play with codec API, but let you try what you can do with (Closed firmware..until) . CBR is an interesting request for low delay/transmitting in a constant bitrate channel. Having all H264 parameters to play could be very helpfull.

[popcornmix]

This is in latest rpi-update firmware.

[F5OEO]

Thx. Confirm that don't hang up as before. Not tested if it modifies behavior of CPB buffer, but could be closed.