amaai-lab / video2music

Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model

  • Public
  • 981 runs
  • L40S
  • GitHub
  • Paper
  • License
Iterate in playground

Input

*file

Input video file

string
Shift + Return to add a new line

Primer chords (e.g., C Am F G)

Default: "C Am F G"

string

Key

Default: "C major"

Output

Generated in

This output was created using a different version of the model, amaai-lab/video2music:c0c1f046.

Run time and cost

This model costs approximately $0.025 to run on Replicate, or 40 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 26 seconds.

Readme

We propose a novel AI-powered multimodal music generation framework called Video2Music. This framework uniquely uses video features as conditioning input to generate matching music using a Transformer architecture. By employing cutting-edge technology, our system aims to provide video creators with a seamless and efficient solution for generating tailor-made background music.