Developers from Google presented a new project, within which a technology was created that can translate oral sentences from one language to another. The main difference between the new translator called Translatotron and its analogues is that it works exclusively with sound, without using intermediate text. This approach allowed us to significantly speed up the work of the translator. Another noteworthy point is that the system accurately mimics the speaker's frequency and tone.
It was possible to create Translatotron thanks to continuous work, which took several years. Researchers at Google have long considered the possibility of direct speech conversion, but until recently it was not possible to realize their plans.
The simultaneous translation systems used today most often work according to one algorithm. At the initial stage, the original speech is transformed into text. The text in one language is then converted into text in another language. After that, the resulting text is converted into speech in the desired language. This method works well, but it is not without drawbacks. At each stage, errors can occur that overlap each other and lead to a decrease in the quality of the translation.
To achieve the desired result, the researchers studied the sound spectrograms. They tried to make a spectrogram in one language turn into a spectrogram in another language, skipping the steps of converting sound to text.
It is worth noting that despite the complexity of such a transformation, speech processing takes place in one step, and not in three, as it was before. With a sufficient amount of computing power at its disposal, Translatotron will perform simultaneous translation much faster. Another important point is that this approach allows you to preserve the features and intonation of the original voice.
At this stage, Translatotron cannot boast the same high translation accuracy as standard systems. Despite this, the researchers say that most of the translations made are of sufficient quality. In the future, work on Translatotron will continue, as the researchers intend to improve the quality of simultaneous speech translation.
Source: 3dnews.ru