Google has introduced a new audio codec, Lyra, optimized for delivering maximum speech quality even over very slow communication channels. The implementation of Lyra is written in C++ and released under the Apache 2.0 license, but one of its necessary dependencies is the proprietary library libsparse_inference.so, which implements the core for mathematical computations. It is noted that the proprietary library is temporary — Google promises to develop an open-source replacement in the future and ensure support for various platforms.
In terms of voice data quality over low-speed connections, Lyra significantly surpasses traditional codecs that utilize digital signal processing methods. To achieve high-quality voice transmission under limited bandwidth conditions, Lyra employs a speech model based on a machine learning system, which reconstructs missing information based on typical speech characteristics, in addition to standard audio compression techniques and signal transformations. The sound generation model is trained using thousands of hours of voice recordings in more than 70 languages.
The codec includes an encoder and a decoder. The encoder's algorithm involves extracting voice data parameters every 40 milliseconds, compressing them, and transmitting them to the recipient over the network. A communication channel with a speed of 3 kilobits per second is sufficient for data transmission. The extracted audio parameters include logarithmic mel-spectrograms that account for the energy characteristics of speech across various frequency ranges, prepared with respect to the human auditory perception model.
The decoder employs a generative model that reconstructs speech signals based on the provided audio parameters. To reduce computational complexity, a lightweight model based on a recurrent neural network is used, which is a variant of the WaveRNN speech synthesis model. It operates at a lower sampling frequency while simultaneously generating multiple signals across different frequency ranges. The resulting signals are then combined to produce a single output signal corresponding to the specified sampling rate.
To accelerate processing, specialized processor instructions available in 64-bit ARM processors are also utilized. As a result, despite the use of machine learning, the Lyra codec can be used for real-time speech encoding and decoding on mid-range smartphones, demonstrating a signal transmission latency of around 90 milliseconds.
Source: opennet.ru
