Odprt koda Spleeter, sistema za ločevanje glasbe in glasu

Streaming provider Deezer has opened the source codes of the experimental project Spleeter, which develops a machine learning system for separating audio sources from complex sound compositions. The program allows you to remove vocals from a composition and keep only the musical accompaniment, manipulate the sound of individual instruments, or discard the music and keep the voice for overlaying on another audio track, creating mixes, karaoke, or transcriptions. The project's code is written in Python using the Tensorflow engine and se distribuira pod licenco MIT.

Za prenos predlagajo pre-trained models to separate vocals (one voice) from accompaniment, as well as to split into 4 and 5 streams, including vocals, drums, bass, piano, and other sounds. Spleeter can be used as a Python library or as a standalone command line utility. In the simplest case, based on the source file two, four, or five files with vocals and components from the accompaniment (vocals.wav, drums.wav, bass.wav, piano.wav, other.wav) are created. When splitting into 2 and 4 streams, Spleeter provides very high performance; for example, when utilizing a GPU, the separation of an audio file into 4 streams takes 100 times less time than the duration of the original composition. On a system with an NVIDIA GeForce GTX 1080 GPU and 32-core Intel Xeon Gold 6134 CPU, processing a test collection musDB, lasting three hours and 27 minutes, was completed in 90 seconds.

Among the advantages of Spleeter, compared to other developments in the field of sound separation, such as the open project

Open-Unmix , is the use of higher quality models built on the basis of a vast collection of sound files. Due to copyright restrictions, researchers in the field of machine learning are limited to access to rather meager publicly available collections of musical files, while for Spleeter, the models were built using data from a vast musical catalog at Deezer.in comparison

Po primerjavi Spleeter постигло разделение примерно на 35% быстрее при тестах на CPU с Open-Unmix. Оно поддерживает MP3-файлы и генерирует значительно более качественный результат (при выделении голоса в Open-Unmix остаются следы некоторых инструментов, что, вероятно, объясняется тем, что модели Open-Unmix натренированы на коллекции из 150 композиций).

Izvor: opennet.ru