Official

kwaivgi / kling-lip-sync

Add lip-sync to any video with an audio file or text

Warm

Public
1.8K runs
$0.014 per second of output video

Iterate in playground

Run with an API

Playground API Examples README

Input

Video Player is loading.

Current Time 00:00:000

Duration 00:00:000

Loaded: 0%

Stream Type LIVE

Remaining Time 00:00:000

video_url

string

Shift + Return to add a new line

https://replicate.delivery/xezq/ipjGAn65es3cfkPhe7g3IvYQqDfbHeCfBof1ujrrMvb3rQAXKA/tmptucl_ok3.mp4https://replicate.delivery/xezq/ipjGAn65es3cfkPhe7g3IvYQqDfbHeCfBof1ujrrMvb3rQAXKA/tmptucl_ok3.mp4

URL of a video for lip syncing. It can be an .mp4 or .mov file, should be less than 100MB, with a duration of 2-10 seconds, and a resolution of 720p-1080p (720-1920px dimensions). Cannot be used with video_id.

audio_file

file

Audio file for lip sync. Must be .mp3, .wav, .m4a, or .aac and less than 5MB.

text

string

Shift + Return to add a new line

Text content for lip sync (if not using audio)

voice_id

string

Voice ID for speech synthesis (if using text and not audio)

Default: "en_AOT"

voice_speed

number

(minimum: 0.8, maximum: 2)

Speech rate (only used if using text and not audio)

Default: 1

video_id

string

Shift + Return to add a new line

ID of a video generated by Kling. Cannot be used with video_url.

Run this model in Node.js with one line of code:

npx create-replicate --model=kwaivgi/kling-lip-sync

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run kwaivgi/kling-lip-sync using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const input = {
  voice_id: "en_AOT",
  video_url: "https://replicate.delivery/xezq/ipjGAn65es3cfkPhe7g3IvYQqDfbHeCfBof1ujrrMvb3rQAXKA/tmptucl_ok3.mp4",
  audio_file: "https://replicate.delivery/pbxt/N245edsFrGTRuk6v5OWFet0nsqiahHTlSF8yRfZEbbZCxSzY/replicate-prediction-sz4ehr9vanrme0cpwnp9wr4g8c.mp3",
  voice_speed: 1
};

const output = await replicate.run("kwaivgi/kling-lip-sync", { input });

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run kwaivgi/kling-lip-sync using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "kwaivgi/kling-lip-sync",
    input={
        "voice_id": "en_AOT",
        "video_url": "https://replicate.delivery/xezq/ipjGAn65es3cfkPhe7g3IvYQqDfbHeCfBof1ujrrMvb3rQAXKA/tmptucl_ok3.mp4",
        "audio_file": "https://replicate.delivery/pbxt/N245edsFrGTRuk6v5OWFet0nsqiahHTlSF8yRfZEbbZCxSzY/replicate-prediction-sz4ehr9vanrme0cpwnp9wr4g8c.mp3",
        "voice_speed": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run kwaivgi/kling-lip-sync using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "input": {
      "voice_id": "en_AOT",
      "video_url": "https://replicate.delivery/xezq/ipjGAn65es3cfkPhe7g3IvYQqDfbHeCfBof1ujrrMvb3rQAXKA/tmptucl_ok3.mp4",
      "audio_file": "https://replicate.delivery/pbxt/N245edsFrGTRuk6v5OWFet0nsqiahHTlSF8yRfZEbbZCxSzY/replicate-prediction-sz4ehr9vanrme0cpwnp9wr4g8c.mp3",
      "voice_speed": 1
    }
  }' \
  https://api.replicate.com/v1/models/kwaivgi/kling-lip-sync/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.