meta / musicgen

Generate music from a prompt or melody

  • Public
  • 2.4M runs
  • A100 (80GB)
  • GitHub
  • Paper
  • License
Run with an API
Playground API Examples README Versions

Input

string

Model to use for generation

Default: "stereo-melody-large"

string
Shift + Return to add a new line

A description of the music you want to generate.

file

An audio file that will influence the generated music. If `continuation` is `True`, the generated music will be a continuation of the audio file. Otherwise, the generated music will mimic the audio file's melody.

integer

Duration of the generated audio in seconds.

Default: 8

boolean

If `True`, generated music will continue from `input_audio`. Otherwise, generated music will mimic `input_audio`'s melody.

Default: false

integer
(minimum: 0)

Start time of the audio file to use for continuation.

Default: 0

integer
(minimum: 0)

End time of the audio file to use for continuation. If -1 or None, will default to the end of the audio clip.

boolean

If `True`, the EnCodec tokens will be decoded with MultiBand Diffusion. Only works with non-stereo models.

Default: false

string

Strategy for normalizing audio.

Default: "loudness"

integer

Reduces sampling to the k most likely tokens.

Default: 250

number

Reduces sampling to tokens with cumulative probability of p. When set to `0` (default), top_k sampling is used.

Default: 0

number

Controls the 'conservativeness' of the sampling process. Higher temperature means more diversity.

Default: 1

integer

Increases the influence of inputs on the output. Higher values produce lower-varience outputs that adhere more closely to inputs.

Default: 3

string

Output format for generated audio.

Default: "wav"

integer

Seed for random number generator. If None or -1, a random seed will be used.

Add a payment method to run this model.

Sign in with GitHub

By signing in, you agree to our
terms of service and privacy policy

Output

Current Time 00:00:000
Duration 00:00:000
Loaded: 0%
Stream Type LIVE
Remaining Time 00:00:000
 
1x
Generated in
Tweak itReport

Run time and cost

This model costs approximately $0.077 to run on Replicate, or 12 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 56 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Model Description

MusicGen is a simple and controllable model for music generation. This deployment exposes two versions of MusicGen:

  • Melody. A 1.5 billion parameter model that you can prompt with both text and audio
  • Large. A 3.5 billion parameter model that you can prompt with text

You can specify the model you want to use via the model_version parameter, which is set to 'melody' by default.

To learn more about this model, check out the repository and paper

Model Architecture and Development

MusicGen is single stage auto-regressive Transformer model trained over a 32kHz EnCodec tokenizer with 4 codebooks sampled at 50 Hz. Unlike existing methods like MusicLM, MusicGen doesn’t require a self-supervised semantic representation, and it generates all 4 codebooks in one pass. By introducing a small delay between the codebooks, the authors show they can predict them in parallel, thus having only 50 auto-regressive steps per second of audio. They used 20K hours of licensed music to train MusicGen. Specifically, they relied on an internal dataset of 10K high-quality music tracks, and on the ShutterStock and Pond5 music data.

Licenses

  • All code in this repository is licensed under the Apache License 2.0 license.
  • The code in the Audiocraft repository is released under the MIT license as found in the LICENSE file.
  • The weights in the Audiocraft repository are released under the CC-BY-NC 4.0 license as found in the LICENSE_weights file.