After nearly a year of development, GStreamer 1.28 has been released, a cross-platform component set for creating a wide range of multimedia applications, from media players and audio/video file converters to VoIP applications and streaming systems. GStreamer code is distributed under the LGPLv2.1 license. Updates for plugins gst-plugins-base, gst-plugins-good, gst-plugins-bad, gst-plugins-ugly, as well as the gst-libav wrapper and the gst-rtsp-server streaming server are being developed separately. At the API and ABI level, the new release is backward compatible with the 1.0 branch. Binary builds will soon be available for Android, iOS, macOS, and Windows (on Linux, it is recommended to use packages from the distribution).
Key improvements in GStreamer 1.28:
- A new HIP (Heterogeneous-computing Interface for Portability) plugin has been added to support the framework developed by AMD for heterogeneous computing, where parallel computing resources of GPUs are used alongside CPUs. HIP simplifies the implementation of support for different GPUs in a single codebase (the ROCm stack is used for execution on AMD GPUs, while a translation layer to CUDA is used for NVIDIA GPUs). The plugin provides components implemented using HIP for performing operations such as video mixing, conversion from one color space to another, video resizing, and facilitating data exchange between system memory and video memory. Additionally, a library has been prepared to integrate HIP support into applications.
- Support for the LCEVC (Low Complexity Enhancement Video Coding) video encoding standard has been added, which implements an additional layer with metadata on top of standard codecs H.265 and H.266 to improve video quality. The implementation of the LCEVC encoder and decoder is based on libraries from the V-Nova SDK.
- Support for the Vulkan graphics API capabilities has been added to accelerate video decoding in AV1 and VP9 formats, as well as video encoding in H.264 format. The Vulkan-based H.265 video decoder now supports a color depth of 10 bits per channel.
- In the gtkwaylandsink element, designed for rendering with GTK4 and Wayland, enhanced color capabilities have been added, including support for parsing and setting HDR10 metadata.
- A new element has been added for separating audio sources (e.g., separating voice from background music), based on the charon-audio library, which implements the Demucs method in Rust. Additionally, a Deepgram plugin for speech recognition, written in Rust, has been included.
- Enhanced analytics and object recognition capabilities. Added components for merging and splitting analytics. Implemented tensor-decoder classifier, facedetector for face detection, YOLOv8 object detector, an element for execution, and decoder for the YOLOX object recognition model. Added tensordecodebin element for automatic connection of decoders and a component for executing AI models using LiteRT (replacing TensorFlow Lite).
- A new element for decoding GIF images has been added, written in Rust and supporting looping animations.
- In the input-selector element, a two-phase mechanism for switching between input streams (sinkpad) has been implemented, eliminating the occurrence of race conditions during switching.
- The webrtcsink element has been enhanced with support for hardware-accelerated encoding using VA-API and the ability to renegotiate connections.
- Added components for combining and extracting streams in accordance with the ST-2038 standard for broadcasting.
- The fallbacksrc element now supports encoded streams.
- In the FLV multimedia container used for RTMP broadcasting, support for H.265 video has been improved, and multi-track audio support has been added.
- In the glupload element, support for a loader based on udmabuf has been added, allowing buffer sharing between software decoders/sources and GPUs, display engines (wayland), and other devices.
- A qml6 element has been added for rendering using Qt6 QML.
- The video processing library has been updated to include operations for cropping, scaling, rotating, and shifting.
- The GstContext object has been added to allow the use of a shared thread pool when executing elements to improve the performance of operations such as compositing and video conversion.
- The icecastsink element has been added for receiving streams from IceCast, supporting AAC format and written in Rust.
- A new voice cloning element from ElevenLabs has been added.
- The speech synthesizer has been enhanced with a mode for speeding up playback while preserving pitch.
- A vmaf element has been added for assessing video quality perception using Netflix's VMAF framework.
- A new MPEG-H audio decoding plugin has been introduced, based on the Fraunhofer MPEG-H decoder and supporting the unpacking of MP4 media containers.
- Support for hidden subtitles (CC — Closed Captions) and text processing has been expanded.
Source: opennet.ru
