-
Notifications
You must be signed in to change notification settings - Fork 62
Libavcodec Compression
FFmpeg provides a huge collection of codecs for use. Some of them (but not all) may be used with UltraGrid.
When no compression is specified, MJPEG is used. You can, however, pick another compression like H.264 or HEVC. Selection of other codecs is available as well (i. a. ProRes, VP9 or AV1).
Basic, but still useful and computationally decent compression is MJPEG which can be used if invoked in a following way:
uv -t testcard -c libavcodec:codec=MJPEG
uv -t testcard -c libavcodec:codec=MJPEG:q=23 # use quality 23 (approximately default)
uv -t testcard -c libavcodec # MJPEG is a default compression
Note: quality can be affected only by the "q" parameter. Other quality-affecting parameters like "bitrate" are not honored.
Usage:
uv -t deltacast -c libavcodec:codec=H.264 <address> # use H.264
uv -t deltacast -c libavcodec:codec=H.264:bitrate=20M <address> # specifies requested bitrate
uv -t deltacast -c libavcodec:codec=H.264:subsampling=420 <address> # enforce 4:2:0 subsampling (may be needed for the stream to be decodable with VAAPI)
Using NVENC encoding (NVIDIA only):
uv -t deltacast -c libavcodec:encoder=h264_nvenc
Use CUVID (HW accelerated) decoder (see below):
uv -d gl --param force-lavd-decoder=h264_cuvid
Since HEVC is relatively new and still a bit demanding compression providing a great compression ratio, you may need to tweak things a bit to achieve optimal performance.
There are multiple encoders supporting HEVC encoding, namely libx265 and hevc_nvenc (and also hevc_qsv if available). While encoding HEVC is still a bit demanding, it is advisable to use the NVENC encoder (or QSV) to encode the stream:
uv -t deltacast -c libavcodec:encoder=hevc_nvenc
Currently, the stream encoded by NVENC encoder isn’t much parallelizable by decoder, so you may want to force hardware decoder (please note that the decoder currently adds some 4 frames of latency!):
uv -d gl --param force-lavd-decoder=hevc_cuvid
Alternatively, you may reduce the bit rate, 10 or 15 Mbps is a way easier to decode:
uv -t deltacast -c libavcodec:encoder=hevc_nvenc:bitrate=15M
You can use also software encoder, which can be a bit slowish, however decoder parallelizes easily.
uv -t deltacast -c libavcodec:encoder=libx265
Once you have both hardware encoder and decoder, you can turn on spatial AQ to improve the image quality (of course it can be used also along with SW decoder, however decoding is then a bit more computationally demanding).
uv -t deltacast -c libavcodec:encoder=hevc_nvenc:spatial_aq=1
Besides H.264/HEVC, UltraGrid supports also few other codecs that may be used, eg.:
- VP8/VP9
- AV1
- J2K - FFMPEG's implementation of J2K is however very slow
uv -t deltacast -c libavcodec <addr> # use default libavcodec codec (currently MJPEG)
uv -t deltacast -c libavcodec:codec=MJPEG <addr> # use MJPEG codec explicitly
uv -t deltacast -c libavcodec:help <addr> # prints available codecs/encoders (and decoders)
UltraGrid offers some hardware compression accelerations. Encoders are always standalone, therefore specifying encoder=<name> is sufficient. For decoders there are on one hand also standalone decoders that may be used similarly but also so called hardware accelerations (currently implemented in UG only for Linux).
Please note that the HW decoders may not be able to decode arbitrary video. If decoder refuses to decode, it is useful to try to set encoder to encode 8-bit 4:2:0 YCbCr video. You can try -c libavcodec[other_opts]:subsampling=420, if it doesn't help. :disable_intra_refresh may also improve compatibility. Color depth should be ideally controlled at the source (eg. -t decklink:codec=UYVY).
See also a separate page Hw. acceleration support.
HW accelerated encoding is toggled by selecting appropriate encoder, eg. hevc_vaapi or hevc_nvenc:
uv -t testcard -c libavcodec:encoder=hevc_amf (Windows only)
uv -t testcard -c libavcodec:encoder=hevc_nvenc
uv -t testcard -c libavcodec:encoder=hevc_vaapi (Linux only)
uv -t testcard -c libavcodec:encoder=hevc_qsv
uv -t testcard -c libavcodec:encoder=hevc_videotoolbox (mac only)
VAAPI/VDPAU accelerated decoding (if supported) can be toggled with following command:
uv -d decklink --param use-hw-accel
CUVID and QuickSync are regular, HW-accelerated decoders (can be listed with -c libavcodec:help). Usage:
uv -d gl --param force-lavd-decoder=hevc_cuvid
uv -d gl --param force-lavd-decoder=hevc_qsv
or if unsure which codec arrives, you can list more decoders:
uv -d gl --param force-lavd-decoder=hevc_cuvid:h264_cuvid
Note: Using a cuvid may introduce additional latency.
Note 2: QuickSync decoder may not support intra refresh feature, in that case use -c libavcodec:encoder=<enc>:disable_intra_refresh on encoder.
Prefer CUVID/QuickSync and similar decoders for higher bit depths because current UltraGrid VAAPI/VDPAU implementation supports only 8-bits decoding.
Note: If using VAAPI for decoding, set 4:2:0 subsampling on the encoder, otherwise decoding won't work, eg.:
uv -t testcard -c libavcodec:encoder=libx264:subsampling=420 <receiver>
| API | Encode | Decode |
|---|---|---|
| NVENC | Win/Linux | Win/Linux (CUVID) |
| QuickSync | Win/Linux ¹ | Win/Linux ¹ |
| VideoToolbox | Mac | no |
| VA API ² | Linux | Linux |
| VDPAU ² | Linux | Linux |
- ¹ recent Linux distributions are required (aka Ubuntu 20.10 or Debian 11)
- ² see also Hw. acceleration support
See also:
- https://trac.ffmpeg.org/wiki/HWAccelIntro
- https://en.wikipedia.org/wiki/Quicksync
- https://en.wikipedia.org/wiki/Nvidia_NVENC
UltraGrid parameters allow more fine-grained selecting of parameters, eg. converting R12L to 10 or 12-bit 4:4:4 YCbCr to keep bit depth instead of keeping color space (default)
Then use option lavc-use-codec at encoder:
uv -t decklink:codec=R12L --param lavc-use-codec=yuv444p16le -c libavcodec:encoder=hevc_nvenc <recv>
And at the decoder, you can toggle the counterpart, option decoder-use-codec:
uv -d decklink --param decoder-use-codec=R12L <sender>
Here the codec is one that is supported by the display. If you enter an unsupported one, decoder writes you available options. Or you can use a keyword help to see display supported codecs.
It is sometimes tricky to know which pixel formats are available and provided by the decoder. Fortunately it can be probed with UltraGrid, just run (for example for x264; don't forgot to press Ctrl-C to exit UltraGrid):
uv -t testcard -c libavcodec:encoder=libx264 --verbose=7 2>&1 | grep 'supported pixel formats'
You can also check what codec would go from decoder. This is only informative, UltraGrid can do a conversion if needed. But if you expect 12-bit and there are only 8-bits available, it would need more tweaking (another decoder or a card):
uv -t testcard -c libavcodec:encoder=libx264 -d dummy --verbose=7 2>&1 | grep 'Available output pixel formats:'
Note: the available output pixel formats may differ according to input stream properties, thus pass the parameters to encoder exactly.
Sometimes is easier to get an overview of device capabilities first.
To see capabilities of a graphic card, you can download and compile this tool (requires nv-codec-headers) and see output of nvencinfo and nvdecinfo for an list of HW supported features
git clone https://github.com/FFmpeg/nv-codec-headers.git
cd nv-codec-headers && make && sudo make install
cd ..
git clone https://github.com/philipl/nv-video-info
cd nv-video-info && ./autogen.sh && make
./nvencinfo
Also this matrix may be useful.
For VDPAU and VAAPI there exist functions vdpauinfo and vainfo to check capabilities
If you have any technical or non-technical question or suggestion please feel free to contact us at