mtg/music-arousal-valence – Run with an API on Replicate

mtg / music-arousal-valence

Regression of musical arousal and valence values

Cold

Public
8.6K runs
CPU
GitHub
License

Run with an API

Playground API Examples README Versions

Input

Video Player is loading.

Current Time 00:00:000

Duration 00:00:000

Loaded: 0%

Stream Type LIVE

Remaining Time 00:00:000

audio

file

Audio file to process

url

string

Shift + Return to add a new line

YouTube URL to process (overrides audio input)

embedding_type

string

Embedding type to use: vggish, or musicnn

Default: "msd-musicnn"

dataset

string

Arousal/Valence training dataset

Default: "emomusic"

output_format

string

Output either a bar chart visualization or a JSON blob

Default: "Visualization"

Run this model in Node.js with one line of code:

npx create-replicate --model=mtg/music-arousal-valence

or set up a project from scratch

Install Replicate’s Node.js client library

npm install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run mtg/music-arousal-valence using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "mtg/music-arousal-valence:9ae3915e0727ccad372d2608fe076cad87437306da79876df695c82a63149175",
  {
    input: {
      audio: "https://replicate.delivery/mgxm/907f9b45-185c-41b1-96af-f2742bada25b/rock.mp3",
      dataset: "emomusic",
      output_format: "Visualization",
      embedding_type: "msd-musicnn"
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library

pip install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client

import replicate

Run mtg/music-arousal-valence using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "mtg/music-arousal-valence:9ae3915e0727ccad372d2608fe076cad87437306da79876df695c82a63149175",
    input={
        "audio": "https://replicate.delivery/mgxm/907f9b45-185c-41b1-96af-f2742bada25b/rock.mp3",
        "dataset": "emomusic",
        "output_format": "Visualization",
        "embedding_type": "msd-musicnn"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run mtg/music-arousal-valence using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "9ae3915e0727ccad372d2608fe076cad87437306da79876df695c82a63149175",
    "input": {
      "audio": "https://replicate.delivery/mgxm/907f9b45-185c-41b1-96af-f2742bada25b/rock.mp3",
      "dataset": "emomusic",
      "output_format": "Visualization",
      "embedding_type": "msd-musicnn"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Install Cog

brew install cog

If you don’t have Homebrew, there are other installation options available.

Pull and run mtg/music-arousal-valence using Cog (this will download the full model and run it in your local environment):

cog predict r8.im/mtg/music-arousal-valence@sha256:9ae3915e0727ccad372d2608fe076cad87437306da79876df695c82a63149175 \
  -i 'audio="https://replicate.delivery/mgxm/907f9b45-185c-41b1-96af-f2742bada25b/rock.mp3"' \
  -i 'dataset="emomusic"' \
  -i 'output_format="Visualization"' \
  -i 'embedding_type="msd-musicnn"'

To learn more, take a look at the Cog documentation.

Pull and run mtg/music-arousal-valence using Docker (this will download the full model and run it in your local environment):

docker run -d -p 5000:5000 r8.im/mtg/music-arousal-valence@sha256:9ae3915e0727ccad372d2608fe076cad87437306da79876df695c82a63149175
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "audio": "https://replicate.delivery/mgxm/907f9b45-185c-41b1-96af-f2742bada25b/rock.mp3",
      "dataset": "emomusic",
      "output_format": "Visualization",
      "embedding_type": "msd-musicnn"
    }
  }' \
  http://localhost:5000/predictions

Output

Generated in

7.9 seconds

Tweak itReport

This example was created by a different version, mtg/music-arousal-valence:1064850e.

Examples

View more examples

Run time and cost

This model costs approximately $0.00046 to run on Replicate, or 2173 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on CPU hardware. Predictions typically complete within 5 seconds.

Readme

This demo runs a series of transfer learning regression models trained to predict musical arousal and valence values. These classifiers were trained on a mixture of public and in-house MTG datasets.

Source models

MusiCNN. A musically motivated CNN with two variants trained on the Million Song Dataset and the MagnaTagATune.
VGGish. A large VGG variant trained on a preliminary version of the AudioSet Dataset.

Transfer learning classifiers

Our models consist of single-hidden-layer MLPs trained on the considered embeddings.

License

These models are part of Essentia Models made by MTG-UPF and are publicly available under CC by-nc-sa and commercial license.