zsxkib/v-express:e0122658 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

zsxkib /v-express:e0122658

Playground API

Input

reference_image

file

Path to the reference image that will be used as the base for the generated video.

driving_audio

file

Path to the audio file that will be used to drive the motion in the generated video.

use_video_audio

boolean

If True and driving_video is provided, use the audio from the driving video instead of the driving_audio.

Default: false

driving_video

file

Path to the video file that will be used to extract the head motion. If not provided, the generated video will use the motion based on the selected motion_mode.

motion_mode

string

Mode for generating the head motion in the output video.

Default: "fast"

reference_attention_weight

number

(minimum: 0, maximum: 1)

Amount of attention to pay to the reference image vs. the driving motion. Higher values will make the generated video adhere more closely to the reference image. Range: 0.0 to 1.0

Default: 0.95

audio_attention_weight

number

(minimum: 0, maximum: 10)

Amount of attention to pay to the driving audio vs. the reference image. Higher values will make the generated video's motion more closely match the driving audio. Range: 0.0 to 10.0

Default: 3

num_inference_steps

integer

(minimum: 1, maximum: 100)

Number of diffusion steps to perform during generation. More steps will generally produce better quality results but will take longer to run. Range: 1 to 100

Default: 25

image_width

integer

(minimum: 64, maximum: 2048)

Width of the generated video frames.

Default: 512

image_height

integer

(minimum: 64, maximum: 2048)

Height of the generated video frames.

Default: 512

frames_per_second

number

(minimum: 1, maximum: 60)

Frame rate of the generated video.

Default: 30

guidance_scale

number

(minimum: 1, maximum: 20)

Guidance scale for the diffusion model. Higher values will result in the generated video following the driving motion and audio more closely.

Default: 3.5

num_context_frames

integer

(minimum: 1, maximum: 24)

Number of context frames to use for motion estimation.

Default: 12

context_stride

integer

(minimum: 1, maximum: 10)

Stride of the context frames.

Default: 1

context_overlap

integer

(minimum: 0, maximum: 24)

Number of overlapping frames between context windows.

Default: 4

num_audio_padding_frames

integer

(minimum: 0, maximum: 10)

Number of audio frames to pad on each side of the driving audio.

Default: 2

seed

integer

Random seed. Leave blank to randomize the seed

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/v-express

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/v-express using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/v-express:e01226580ffa003adb191086383ae9c8d687ba1f37ddedd758351a28cf3488b2",
  {
    input: {
      image_width: 512,
      motion_mode: "fast",
      image_height: 512,
      context_stride: 1,
      guidance_scale: 3.5,
      context_overlap: 4,
      use_video_audio: false,
      frames_per_second: 30,
      num_context_frames: 12,
      num_inference_steps: 25,
      audio_attention_weight: 3,
      num_audio_padding_frames: 2,
      reference_attention_weight: 0.95
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/v-express using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/v-express:e01226580ffa003adb191086383ae9c8d687ba1f37ddedd758351a28cf3488b2",
    input={
        "image_width": 512,
        "motion_mode": "fast",
        "image_height": 512,
        "context_stride": 1,
        "guidance_scale": 3.5,
        "context_overlap": 4,
        "use_video_audio": False,
        "frames_per_second": 30,
        "num_context_frames": 12,
        "num_inference_steps": 25,
        "audio_attention_weight": 3,
        "num_audio_padding_frames": 2,
        "reference_attention_weight": 0.95
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/v-express using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zsxkib/v-express:e01226580ffa003adb191086383ae9c8d687ba1f37ddedd758351a28cf3488b2",
    "input": {
      "image_width": 512,
      "motion_mode": "fast",
      "image_height": 512,
      "context_stride": 1,
      "guidance_scale": 3.5,
      "context_overlap": 4,
      "use_video_audio": false,
      "frames_per_second": 30,
      "num_context_frames": 12,
      "num_inference_steps": 25,
      "audio_attention_weight": 3,
      "num_audio_padding_frames": 2,
      "reference_attention_weight": 0.95
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.