zsxkib / mmaudio-t4

Cost-optimized MMAudio V2 (T4 GPU): Add sound to video using this version running on T4 hardware for lower cost. Synthesizes high-quality audio from video content.

Cold

Public
373 runs
T4
GitHub
Weights
Paper
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Prediction

zsxkib/mmaudio-t4:330393ae234d739f3261ae389a5506a73a1bae8c77dc6c6faebc5bde78b6e972

Model

zsxkib/mmaudio-t4:330393ae

vb33xx6ry1rg80cnysn9h9ztf4

Status

Succeeded

Source

Web

Hardware

Total duration

5m 40s

Created

2 months ago

Input

video
prompt: waves, storm
duration: 10
num_steps: 25
cfg_strength: 4.5
negative_prompt: music

{
  "video": "https://huggingface.co/hkchengrex/MMAudio/resolve/main/examples/sora_kraken.mp4",
  "prompt": "waves, storm",
  "duration": 10,
  "num_steps": 25,
  "cfg_strength": 4.5,
  "negative_prompt": "music"
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/mmaudio-t4 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/mmaudio-t4:330393ae234d739f3261ae389a5506a73a1bae8c77dc6c6faebc5bde78b6e972",
  {
    input: {
      video: "https://huggingface.co/hkchengrex/MMAudio/resolve/main/examples/sora_kraken.mp4",
      prompt: "waves, storm",
      duration: 10,
      num_steps: 25,
      cfg_strength: 4.5,
      negative_prompt: "music"
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run zsxkib/mmaudio-t4 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/mmaudio-t4:330393ae234d739f3261ae389a5506a73a1bae8c77dc6c6faebc5bde78b6e972",
    input={
        "video": "https://huggingface.co/hkchengrex/MMAudio/resolve/main/examples/sora_kraken.mp4",
        "prompt": "waves, storm",
        "duration": 10,
        "num_steps": 25,
        "cfg_strength": 4.5,
        "negative_prompt": "music"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run zsxkib/mmaudio-t4 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zsxkib/mmaudio-t4:330393ae234d739f3261ae389a5506a73a1bae8c77dc6c6faebc5bde78b6e972",
    "input": {
      "video": "https://huggingface.co/hkchengrex/MMAudio/resolve/main/examples/sora_kraken.mp4",
      "prompt": "waves, storm",
      "duration": 10,
      "num_steps": 25,
      "cfg_strength": 4.5,
      "negative_prompt": "music"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Generated in

37.2 seconds

Tweak it Share Report

Want to make some of these yourself?

Run this model