MusicGen: Simple and Controllable Music Generation

Описание: Conditional music generation is challening. MusicGen is a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns, which eliminates the need for cascading several models, e.g., hierarchically or upsampling. MusicGen can generate high-quality samples, while being conditioned on textual description or melodic features, allowing better controls over the generated output. Extensive empirical evaluation, considering both automatic and human studies, shows that MusicGen is superior to the evaluated baselines on a standard text-to-music benchmark.

In this video, I will talk about the following: Why is music generation challenging? How is audio encoded? What is the architecture of MusicGen? How does MusicGen perform?

For more details, please look at https://github.com/facebookresearch/a... and https://arxiv.org/pdf/2306.05284.pdf

Copet, Jade, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. "Simple and Controllable Music Generation." arXiv preprint arXiv:2306.05284 (2023).