The Spleeter code, a system for separating music and vocals, is now open-source

Streaming provider Deezer has launched the source codes of the experimental project Spleeter, which develops a machine learning system for separating sound sources from complex audio compositions. The software allows users to remove vocals from tracks while keeping the musical accompaniment, manipulate the sound of individual instruments, or strip away the music to retain only the vocals for overlaying on another audio track, creating mixes, karaoke, or transcriptions. The project's code is written in Python using the Tensorflow engine and is distributed is licensed under the MIT License.

For download for installation in WSL already trained models for separating vocals (a single voice) from the accompaniment, as well as for splitting into 4 and 5 streams, including vocals, drums, bass, piano, and other sounds. Spleeter can be used both as a Python library and as a standalone command-line utility. In its simplest case, based on the input file creates two, four, or five files with the vocals and components from the accompaniment (vocals.wav, drums.wav, bass.wav, piano.wav, other.wav).

When splitting into 2 and 4 streams, Spleeter ensures very high performance; for example, using a GPU, separating an audio file into 4 streams takes 100 times less time than the duration of the original composition. On a system with an NVIDIA GeForce GTX 1080 GPU and a 32-core Intel Xeon Gold 6134 CPU, processing a test collection from musDB, which lasts three hours and 27 minutes, was completed in 90 seconds.

Among the advantages of Spleeter, compared to other developments in the field of sound separation, such as the open project Open-Unmix, is the use of higher quality models built on a vast collection of audio files. Due to copyright restrictions, researchers in the field of machine learning are limited to access to very sparse publicly available collections of music files, while for Spleeter, the models have been built using data from Deezer's extensive music catalog.

According to in comparison With Open-Unmix, the Spleeter toolkit performs separation approximately 35% faster in CPU tests, supports MP3 files, and generates noticeably higher quality results (when isolating vocals in Open-Unmix, traces of some instruments remain, which is likely due to the fact that Open-Unmix models are trained on a collection of only 150 compositions).

Source: opennet.ru