Collabora has published an implementation of a machine learning system to improve the efficiency of video conferencing compression, which allows, in the case of video transmission with a participant’s face, to reduce the required bandwidth by 10 times while maintaining quality at the H.264 level. The implementation is written in Python using the PyTorch framework and is open under the GPLv3 license.
The method allows reconstructing facial details lost during transmission with a strong compression level. The machine learning model generates a talking head animation based on a separately transmitted high-quality face image and the received video, tracking the change in facial expression and head position in the video. On the sender side, the video is transmitted at a very low bitrate, and on the recipient side it is processed by a machine learning system. For additional quality enhancement, the generated video can be processed using the Super-Resolution model.
Source: opennet.ru