On Monday, Deezer, a French online music streaming service, released Spleeter which is a music separation engine. It comes in the form of a Python Library based on Tensorflow. Stating the reason behind Spleeter, the researchers state, “We release Spleeter to help the Music Information Retrieval (MIR) community leverage the power of source separation in various MIR tasks, such as vocal lyrics analysis from audio, music transcription, any type of multilabel classification or vocal melody extraction.”
Spleeter comes with pre-trained models for 2, 4 and 5 track separation. These include:
- Vocals (singing voice) / accompaniment separation (2 stems)
- Vocals / drums / bass / other separation (4 stems)
- Vocals / drums / bass / piano / other separation (5 stems)
It can also train source separation models or fine-tune pre-trained ones with Tensorflow if you have a dataset of isolated sources.
Deezer benchmarked Spleeter against Open-Unmix another open-source model recently released and reported slightly better performances with increased speed. It can perform separation of audio files to 4 stems 100x faster than real-time when running on a GPU.
You can use Spleeter straight from the command line as well as directly in your own development pipeline as a Python library. It can be installed with Conda, with pip or be used with Docker. Spleeter creators mention a number of potential applications of source separation engine including remixes, upmixing, active listening, educational purposes, and pre-processing for other tasks such as transcription.
Spleeter received mostly positive feedback on Twitter, as people experimented to separate vocals from music.
"Spleeter is the Deezer source separation library with pretrained models written in Python and uses TensorFlow". Meaning it separates vocals and different types of accompaniments (eg, Piano). Looks very interesting. https://t.co/IIW0pCPD7r
— João Pedro Martins 🇪🇺🇵🇹 (@lokijota) November 5, 2019
Spleeter kicks ass. Unjust used it to split vocals and instruments of ice ice baby. 🙂 pic.twitter.com/WrhRnuDzr6
— bert boerland (@bertboerland) November 3, 2019
Only yesterday I was wondering are there any tools to separate/extract different components of music tracks.
Well, this will allow you to create tools to extract voice, piano, drums, etc. from any music track using Machine Learning.
https://t.co/8ceIdiGj01— ϽΓΣⱤẛ∁ (@CholericCleric) November 3, 2019
Wavy.org also ran several songs through the two-stem filter and evaluated them in a blog post. They tried a variety of soundtracks across multiple genres. The performance of audio was much better than expected, however, vocals sometimes felt robotically autotuned. The amount of bleed was shockingly low relative to other solutions and surpassed any available free tool and rival commercial plugins and services.
This new blazingly-fast open-source library isolates vocals from music, built on a TensorFlow model trained on tens of thousands of songs. I tested it on Lizzo, Billie Eilish, Lil Nas X, Marvin Gaye, and others—listen to the results here. https://t.co/HGgUYI7MT7
— Andy Baio (@waxpancake) November 4, 2019
Spleeter will be presented and live-demoed at the 2019 ISMIR conference in Delft.
For more details refer to the official announcement.
Read Next
DeepMind AI’s AlphaStar achieves Grandmaster level in StarCraft II with 99.8% efficiency.
Google AI introduces Snap, a microkernel approach to ‘Host Networking’
Firefox 70 released with better security, CSS, and JavaScript improvements