lucataco / mvsep-mdx23-music-separation

Model for Sound demixing challenge 2023: Music Demixing Track - MDX'23

  • Public
  • 270 runs
  • GitHub
  • Paper

Input

Output

Run time and cost

This model runs on Nvidia A40 (Large) GPU hardware.

Readme

MVSEP-MDX23-music-separation-model

About

Model for Sound demixing challenge 2023: Music Demixing Track - MDX‘23. Model perform separation of music into 4 stems “bass”, “drums”, “vocals”, “other”. Model won 3rd place in challenge (Leaderboard C).

Model based on Demucs4, MDX neural net architectures and some MDX weights from Ultimate Vocal Remover project (thanks Kimberley Jensen for great high quality vocal models). Brought to you by MVSep.com.

Notes

  • If you have not enough GPU memory you can use CPU (--cpu), but it will be slow. Additionally you can use single ONNX (--single_onnx), but it will decrease quality a little bit. Also reduce of chunk size can help (--chunk_size 200000).
  • In current revision code requires less GPU memory, but it process multiple files slower. If you want old fast method use argument --large_gpu. It will require > 11 GB of GPU memory, but will work faster.
  • There is Google.Collab version of this code.

Quality comparison

Quality comparison with best separation models performed on MultiSong Dataset.

Algorithm SDR bass SDR drums SDR other SDR vocals SDR instrumental
MVSEP MDX23 12.5034 11.6870 6.5378 9.5138 15.8213
Demucs HT 4 12.1006 11.3037 5.7728 8.3555 13.9902
Demucs 3 10.6947 10.2744 5.3580 8.1335 14.4409
MDX B ---- 8.5118 14.8192
  • Note: SDR - signal to distortion ratio. Larger is better.

Changes

v1.0.1

  • Settings in GUI updated, now you can control all possible options
  • Kim vocal model updated from version 1 to version 2, you still can use version 1 using parameter --use_kim_model_1
  • Added possibility to generate only vocals/instrumental pair if you don’t need bass, drums and other stems. Use parameter --only_vocals
  • Standalone program was updated. It has less size now. GUI will download torch/cuda on the first run.

Citation

@misc{solovyev2023benchmarks,
      title={Benchmarks and leaderboards for sound demixing tasks}, 
      author={Roman Solovyev and Alexander Stempkovskiy and Tatiana Habruseva},
      year={2023},
      eprint={2305.07489},
      archivePrefix={arXiv},
      primaryClass={cs.SD}
}