chenxwh / diffsynth-exvideo

Extended video synthesis model that generates 128 frames

Cold

Public
203 runs
A100 (80GB)
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

Input prompt

Default: "bonfire, on the stone"

negative_prompt

string

Shift + Return to add a new line

错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，

Specify things to not see in the output

Default: "错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，"

num_frames

integer

(maximum: 128)

Number of the output frames

Default: 128

num_inference_steps

integer

(minimum: 1, maximum: 500)

Number of denoising steps for image and video generation

Default: 25

num_inference_steps_upscale_video

integer

(minimum: 1, maximum: 500)

Number of denoising steps for upscaling the video

Default: 25

seed

integer

Random seed. Leave blank to randomize the seed

Run this model in Node.js with one line of code:

npx create-replicate --model=chenxwh/diffsynth-exvideo

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/diffsynth-exvideo using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/diffsynth-exvideo:b3b0e929bf918153fbc0c5444fbe215f5cdbdbdf610910cf4dfcb6f6006e4783",
  {
    input: {
      prompt: "bonfire, on the stone",
      num_frames: 128,
      negative_prompt: "错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，",
      num_inference_steps: 25,
      num_inference_steps_upscale_video: 25
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run chenxwh/diffsynth-exvideo using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/diffsynth-exvideo:b3b0e929bf918153fbc0c5444fbe215f5cdbdbdf610910cf4dfcb6f6006e4783",
    input={
        "prompt": "bonfire, on the stone",
        "num_frames": 128,
        "negative_prompt": "错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，",
        "num_inference_steps": 25,
        "num_inference_steps_upscale_video": 25
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run chenxwh/diffsynth-exvideo using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "b3b0e929bf918153fbc0c5444fbe215f5cdbdbdf610910cf4dfcb6f6006e4783",
    "input": {
      "prompt": "bonfire, on the stone",
      "num_frames": 128,
      "negative_prompt": "错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，",
      "num_inference_steps": 25,
      "num_inference_steps_upscale_video": 25
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

image

video

upscale_video

{
  "completed_at": "2024-07-01T00:40:40.508249Z",
  "created_at": "2024-07-01T00:23:02.435000Z",
  "data_removed": false,
  "error": null,
  "id": "v6t7hkbm4drgg0cgdep8m63fkw",
  "input": {
    "prompt": "bonfire, on the stone",
    "num_frames": 128,
    "negative_prompt": "错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，",
    "num_inference_steps": 25,
    "num_inference_steps_upscale_video": 25
  },
  "logs": "Using seed: 14650\n  0%|          | 0/25 [00:00<?, ?it/s]\n  4%|▍         | 1/25 [00:00<00:09,  2.41it/s]\n  8%|▊         | 2/25 [00:00<00:08,  2.67it/s]\n 12%|█▏        | 3/25 [00:01<00:07,  2.77it/s]\n 16%|█▌        | 4/25 [00:01<00:07,  2.82it/s]\n 20%|██        | 5/25 [00:01<00:07,  2.84it/s]\n 24%|██▍       | 6/25 [00:02<00:06,  2.86it/s]\n 28%|██▊       | 7/25 [00:02<00:06,  2.87it/s]\n 32%|███▏      | 8/25 [00:02<00:05,  2.88it/s]\n 36%|███▌      | 9/25 [00:03<00:05,  2.88it/s]\n 40%|████      | 10/25 [00:03<00:05,  2.88it/s]\n 44%|████▍     | 11/25 [00:03<00:04,  2.89it/s]\n 48%|████▊     | 12/25 [00:04<00:04,  2.89it/s]\n 52%|█████▏    | 13/25 [00:04<00:04,  2.89it/s]\n 56%|█████▌    | 14/25 [00:04<00:03,  2.89it/s]\n 60%|██████    | 15/25 [00:05<00:03,  2.89it/s]\n 64%|██████▍   | 16/25 [00:05<00:03,  2.89it/s]\n 68%|██████▊   | 17/25 [00:05<00:02,  2.89it/s]\n 72%|███████▏  | 18/25 [00:06<00:02,  2.89it/s]\n 76%|███████▌  | 19/25 [00:06<00:02,  2.89it/s]\n 80%|████████  | 20/25 [00:06<00:01,  2.89it/s]\n 84%|████████▍ | 21/25 [00:07<00:01,  2.89it/s]\n 88%|████████▊ | 22/25 [00:07<00:01,  2.89it/s]\n 92%|█████████▏| 23/25 [00:08<00:00,  2.89it/s]\n 96%|█████████▌| 24/25 [00:08<00:00,  2.89it/s]\n100%|██████████| 25/25 [00:08<00:00,  2.89it/s]\n100%|██████████| 25/25 [00:08<00:00,  2.87it/s]\n  0%|          | 0/25 [00:00<?, ?it/s]\n  4%|▍         | 1/25 [00:03<01:32,  3.84s/it]\n  8%|▊         | 2/25 [00:07<01:29,  3.89s/it]\n 12%|█▏        | 3/25 [00:11<01:25,  3.91s/it]\n 16%|█▌        | 4/25 [00:15<01:22,  3.92s/it]\n 20%|██        | 5/25 [00:19<01:18,  3.92s/it]\n 24%|██▍       | 6/25 [00:23<01:14,  3.93s/it]\n 28%|██▊       | 7/25 [00:27<01:10,  3.93s/it]\n 32%|███▏      | 8/25 [00:31<01:06,  3.94s/it]\n 36%|███▌      | 9/25 [00:35<01:03,  3.94s/it]\n 40%|████      | 10/25 [00:39<00:59,  3.94s/it]\n 44%|████▍     | 11/25 [00:43<00:55,  3.95s/it]\n 48%|████▊     | 12/25 [00:47<00:51,  3.95s/it]\n 52%|█████▏    | 13/25 [00:51<00:47,  3.95s/it]\n 56%|█████▌    | 14/25 [00:55<00:43,  3.95s/it]\n 60%|██████    | 15/25 [00:59<00:39,  3.95s/it]\n 64%|██████▍   | 16/25 [01:03<00:35,  3.95s/it]\n 68%|██████▊   | 17/25 [01:06<00:31,  3.95s/it]\n 72%|███████▏  | 18/25 [01:10<00:27,  3.95s/it]\n 76%|███████▌  | 19/25 [01:14<00:23,  3.96s/it]\n 80%|████████  | 20/25 [01:18<00:19,  3.96s/it]\n 84%|████████▍ | 21/25 [01:22<00:15,  3.96s/it]\n 88%|████████▊ | 22/25 [01:26<00:11,  3.96s/it]\n 92%|█████████▏| 23/25 [01:30<00:07,  3.96s/it]\n 96%|█████████▌| 24/25 [01:34<00:03,  3.95s/it]\n100%|██████████| 25/25 [01:38<00:00,  3.95s/it]\n100%|██████████| 25/25 [01:38<00:00,  3.94s/it]\n  0%|          | 0/31 [00:00<?, ?it/s]\n  3%|▎         | 1/31 [00:00<00:15,  1.95it/s]\n  6%|▋         | 2/31 [00:00<00:12,  2.28it/s]\n 10%|▉         | 3/31 [00:01<00:11,  2.41it/s]\n 13%|█▎        | 4/31 [00:01<00:10,  2.47it/s]\n 16%|█▌        | 5/31 [00:02<00:10,  2.51it/s]\n 19%|█▉        | 6/31 [00:02<00:09,  2.53it/s]\n 23%|██▎       | 7/31 [00:02<00:09,  2.55it/s]\n 26%|██▌       | 8/31 [00:03<00:08,  2.56it/s]\n 29%|██▉       | 9/31 [00:03<00:08,  2.57it/s]\n 32%|███▏      | 10/31 [00:03<00:08,  2.57it/s]\n 35%|███▌      | 11/31 [00:04<00:07,  2.58it/s]\n 39%|███▊      | 12/31 [00:04<00:07,  2.58it/s]\n 42%|████▏     | 13/31 [00:05<00:06,  2.58it/s]\n 45%|████▌     | 14/31 [00:05<00:06,  2.58it/s]\n 48%|████▊     | 15/31 [00:05<00:06,  2.58it/s]\n 52%|█████▏    | 16/31 [00:06<00:05,  2.58it/s]\n 55%|█████▍    | 17/31 [00:06<00:05,  2.58it/s]\n 58%|█████▊    | 18/31 [00:07<00:05,  2.58it/s]\n 61%|██████▏   | 19/31 [00:07<00:04,  2.58it/s]\n 65%|██████▍   | 20/31 [00:07<00:04,  2.58it/s]\n 68%|██████▊   | 21/31 [00:08<00:03,  2.58it/s]\n 71%|███████   | 22/31 [00:08<00:03,  2.58it/s]\n 74%|███████▍  | 23/31 [00:09<00:03,  2.58it/s]\n 77%|███████▋  | 24/31 [00:09<00:02,  2.58it/s]\n 81%|████████  | 25/31 [00:09<00:02,  2.59it/s]\n 84%|████████▍ | 26/31 [00:10<00:01,  2.59it/s]\n 87%|████████▋ | 27/31 [00:10<00:01,  2.59it/s]\n 90%|█████████ | 28/31 [00:10<00:01,  2.59it/s]\n 94%|█████████▎| 29/31 [00:11<00:00,  2.58it/s]\n 97%|█████████▋| 30/31 [00:11<00:00,  2.58it/s]\n100%|██████████| 31/31 [00:12<00:00,  2.59it/s]\n100%|██████████| 31/31 [00:12<00:00,  2.56it/s]\n  0%|          | 0/25 [00:00<?, ?it/s]\n  4%|▍         | 1/25 [00:18<07:28, 18.70s/it]\n  8%|▊         | 2/25 [00:37<07:15, 18.94s/it]\n 12%|█▏        | 3/25 [00:56<06:58, 19.02s/it]\n 16%|█▌        | 4/25 [01:16<06:39, 19.04s/it]\n 20%|██        | 5/25 [01:35<06:21, 19.06s/it]\n 24%|██▍       | 6/25 [01:54<06:02, 19.08s/it]\n 28%|██▊       | 7/25 [02:13<05:43, 19.09s/it]\n 32%|███▏      | 8/25 [02:32<05:24, 19.09s/it]\n 36%|███▌      | 9/25 [02:51<05:05, 19.09s/it]\n 40%|████      | 10/25 [03:10<04:46, 19.09s/it]\n 44%|████▍     | 11/25 [03:29<04:27, 19.10s/it]\n 48%|████▊     | 12/25 [03:48<04:08, 19.10s/it]\n 52%|█████▏    | 13/25 [04:07<03:49, 19.10s/it]\n 56%|█████▌    | 14/25 [04:27<03:30, 19.10s/it]\n 60%|██████    | 15/25 [04:46<03:10, 19.10s/it]\n 64%|██████▍   | 16/25 [05:05<02:51, 19.10s/it]\n 68%|██████▊   | 17/25 [05:24<02:32, 19.10s/it]\n 72%|███████▏  | 18/25 [05:43<02:13, 19.10s/it]\n 76%|███████▌  | 19/25 [06:02<01:54, 19.10s/it]\n 80%|████████  | 20/25 [06:21<01:35, 19.10s/it]\n 84%|████████▍ | 21/25 [06:40<01:16, 19.10s/it]\n 88%|████████▊ | 22/25 [06:59<00:57, 19.11s/it]\n 92%|█████████▏| 23/25 [07:18<00:38, 19.11s/it]\n 96%|█████████▌| 24/25 [07:38<00:19, 19.11s/it]\n100%|██████████| 25/25 [07:57<00:00, 19.11s/it]\n100%|██████████| 25/25 [07:57<00:00, 19.09s/it]\n  0%|          | 0/31 [00:00<?, ?it/s]\n  3%|▎         | 1/31 [00:02<01:00,  2.03s/it]\n  6%|▋         | 2/31 [00:03<00:51,  1.79s/it]\n 10%|▉         | 3/31 [00:05<00:47,  1.71s/it]\n 13%|█▎        | 4/31 [00:06<00:45,  1.68s/it]\n 16%|█▌        | 5/31 [00:08<00:43,  1.66s/it]\n 19%|█▉        | 6/31 [00:10<00:41,  1.65s/it]\n 23%|██▎       | 7/31 [00:11<00:39,  1.64s/it]\n 26%|██▌       | 8/31 [00:13<00:37,  1.63s/it]\n 29%|██▉       | 9/31 [00:15<00:35,  1.63s/it]\n 32%|███▏      | 10/31 [00:16<00:34,  1.63s/it]\n 35%|███▌      | 11/31 [00:18<00:32,  1.63s/it]\n 39%|███▊      | 12/31 [00:19<00:30,  1.63s/it]\n 42%|████▏     | 13/31 [00:21<00:29,  1.63s/it]\n 45%|████▌     | 14/31 [00:23<00:27,  1.63s/it]\n 48%|████▊     | 15/31 [00:24<00:26,  1.63s/it]\n 52%|█████▏    | 16/31 [00:26<00:24,  1.62s/it]\n 55%|█████▍    | 17/31 [00:28<00:22,  1.62s/it]\n 58%|█████▊    | 18/31 [00:29<00:21,  1.62s/it]\n 61%|██████▏   | 19/31 [00:31<00:19,  1.62s/it]\n 65%|██████▍   | 20/31 [00:32<00:17,  1.62s/it]\n 68%|██████▊   | 21/31 [00:34<00:16,  1.62s/it]\n 71%|███████   | 22/31 [00:36<00:14,  1.62s/it]\n 74%|███████▍  | 23/31 [00:37<00:12,  1.62s/it]\n 77%|███████▋  | 24/31 [00:39<00:11,  1.62s/it]\n 81%|████████  | 25/31 [00:41<00:09,  1.62s/it]\n 84%|████████▍ | 26/31 [00:42<00:08,  1.62s/it]\n 87%|████████▋ | 27/31 [00:44<00:06,  1.62s/it]\n 90%|█████████ | 28/31 [00:45<00:04,  1.62s/it]\n 94%|█████████▎| 29/31 [00:47<00:03,  1.62s/it]\n 97%|█████████▋| 30/31 [00:49<00:01,  1.62s/it]\n100%|██████████| 31/31 [00:50<00:00,  1.62s/it]\n100%|██████████| 31/31 [00:50<00:00,  1.64s/it]\nSaving video:   0%|          | 0/128 [00:00<?, ?it/s]\nSaving video:   1%|          | 1/128 [00:00<00:38,  3.30it/s]\nSaving video:  38%|███▊      | 48/128 [00:00<00:00, 152.46it/s]\nSaving video:  59%|█████▉    | 76/128 [00:00<00:00, 179.40it/s]\nSaving video:  80%|███████▉  | 102/128 [00:00<00:00, 181.31it/s]\nSaving video:  98%|█████████▊| 125/128 [00:00<00:00, 175.95it/s]\nSaving video: 100%|██████████| 128/128 [00:00<00:00, 154.66it/s]\nSaving video:   0%|          | 0/128 [00:00<?, ?it/s]\nSaving video:   1%|          | 1/128 [00:00<00:22,  5.67it/s]\nSaving video:  12%|█▎        | 16/128 [00:00<00:01, 69.54it/s]\nSaving video:  27%|██▋       | 34/128 [00:00<00:00, 111.81it/s]\nSaving video:  37%|███▋      | 47/128 [00:00<00:00, 98.22it/s] \nSaving video:  46%|████▌     | 59/128 [00:00<00:00, 101.97it/s]\nSaving video:  55%|█████▌    | 71/128 [00:00<00:00, 68.95it/s] \nSaving video:  62%|██████▎   | 80/128 [00:01<00:00, 61.61it/s]\nSaving video:  69%|██████▉   | 88/128 [00:01<00:00, 57.51it/s]\nSaving video:  74%|███████▍  | 95/128 [00:01<00:00, 50.42it/s]\nSaving video:  79%|███████▉  | 101/128 [00:01<00:00, 51.70it/s]\nSaving video:  84%|████████▍ | 108/128 [00:01<00:00, 55.35it/s]\nSaving video:  90%|████████▉ | 115/128 [00:01<00:00, 49.30it/s]\nSaving video:  95%|█████████▍| 121/128 [00:02<00:00, 44.93it/s]\nSaving video: 100%|██████████| 128/128 [00:02<00:00, 60.15it/s]",
  "metrics": {
    "predict_time": 670.432782891,
    "total_time": 1058.073249
  },
  "output": {
    "image": "https://replicate.delivery/pbxt/ld3K2jKSlx5IBNGDrbeH9EgHfWQ58OdfcNCZefeeql6gCa9hJA/image.png",
    "video": "https://replicate.delivery/pbxt/KVeBfYNizfIv8ofRqEaGWFQVJ0pTVhjVtEuPDpnmqOdaQrPMB/video_512.mp4",
    "upscale_video": "https://replicate.delivery/pbxt/f8DBSfGG9Qsh8kXWjbHjw5fYJimHfclgioFQenm9yTD4gWfwE/video_1024.mp4"
  },
  "started_at": "2024-07-01T00:29:30.075466Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/v6t7hkbm4drgg0cgdep8m63fkw",
    "cancel": "https://api.replicate.com/v1/predictions/v6t7hkbm4drgg0cgdep8m63fkw/cancel"
  },
  "version": "b3b0e929bf918153fbc0c5444fbe215f5cdbdbdf610910cf4dfcb6f6006e4783"
}

Generated in

11 minutes 10 seconds

Tweak it ShareReport

Using seed: 14650
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:00<00:09,  2.41it/s]
  8%|▊         | 2/25 [00:00<00:08,  2.67it/s]
 12%|█▏        | 3/25 [00:01<00:07,  2.77it/s]
 16%|█▌        | 4/25 [00:01<00:07,  2.82it/s]
 20%|██        | 5/25 [00:01<00:07,  2.84it/s]
 24%|██▍       | 6/25 [00:02<00:06,  2.86it/s]
 28%|██▊       | 7/25 [00:02<00:06,  2.87it/s]
 32%|███▏      | 8/25 [00:02<00:05,  2.88it/s]
 36%|███▌      | 9/25 [00:03<00:05,  2.88it/s]
 40%|████      | 10/25 [00:03<00:05,  2.88it/s]
 44%|████▍     | 11/25 [00:03<00:04,  2.89it/s]
 48%|████▊     | 12/25 [00:04<00:04,  2.89it/s]
 52%|█████▏    | 13/25 [00:04<00:04,  2.89it/s]
 56%|█████▌    | 14/25 [00:04<00:03,  2.89it/s]
 60%|██████    | 15/25 [00:05<00:03,  2.89it/s]
 64%|██████▍   | 16/25 [00:05<00:03,  2.89it/s]
 68%|██████▊   | 17/25 [00:05<00:02,  2.89it/s]
 72%|███████▏  | 18/25 [00:06<00:02,  2.89it/s]
 76%|███████▌  | 19/25 [00:06<00:02,  2.89it/s]
 80%|████████  | 20/25 [00:06<00:01,  2.89it/s]
 84%|████████▍ | 21/25 [00:07<00:01,  2.89it/s]
 88%|████████▊ | 22/25 [00:07<00:01,  2.89it/s]
 92%|█████████▏| 23/25 [00:08<00:00,  2.89it/s]
 96%|█████████▌| 24/25 [00:08<00:00,  2.89it/s]
100%|██████████| 25/25 [00:08<00:00,  2.89it/s]
100%|██████████| 25/25 [00:08<00:00,  2.87it/s]
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:03<01:32,  3.84s/it]
  8%|▊         | 2/25 [00:07<01:29,  3.89s/it]
 12%|█▏        | 3/25 [00:11<01:25,  3.91s/it]
 16%|█▌        | 4/25 [00:15<01:22,  3.92s/it]
 20%|██        | 5/25 [00:19<01:18,  3.92s/it]
 24%|██▍       | 6/25 [00:23<01:14,  3.93s/it]
 28%|██▊       | 7/25 [00:27<01:10,  3.93s/it]
 32%|███▏      | 8/25 [00:31<01:06,  3.94s/it]
 36%|███▌      | 9/25 [00:35<01:03,  3.94s/it]
 40%|████      | 10/25 [00:39<00:59,  3.94s/it]
 44%|████▍     | 11/25 [00:43<00:55,  3.95s/it]
 48%|████▊     | 12/25 [00:47<00:51,  3.95s/it]
 52%|█████▏    | 13/25 [00:51<00:47,  3.95s/it]
 56%|█████▌    | 14/25 [00:55<00:43,  3.95s/it]
 60%|██████    | 15/25 [00:59<00:39,  3.95s/it]
 64%|██████▍   | 16/25 [01:03<00:35,  3.95s/it]
 68%|██████▊   | 17/25 [01:06<00:31,  3.95s/it]
 72%|███████▏  | 18/25 [01:10<00:27,  3.95s/it]
 76%|███████▌  | 19/25 [01:14<00:23,  3.96s/it]
 80%|████████  | 20/25 [01:18<00:19,  3.96s/it]
 84%|████████▍ | 21/25 [01:22<00:15,  3.96s/it]
 88%|████████▊ | 22/25 [01:26<00:11,  3.96s/it]
 92%|█████████▏| 23/25 [01:30<00:07,  3.96s/it]
 96%|█████████▌| 24/25 [01:34<00:03,  3.95s/it]
100%|██████████| 25/25 [01:38<00:00,  3.95s/it]
100%|██████████| 25/25 [01:38<00:00,  3.94s/it]
  0%|          | 0/31 [00:00<?, ?it/s]
  3%|▎         | 1/31 [00:00<00:15,  1.95it/s]
  6%|▋         | 2/31 [00:00<00:12,  2.28it/s]
 10%|▉         | 3/31 [00:01<00:11,  2.41it/s]
 13%|█▎        | 4/31 [00:01<00:10,  2.47it/s]
 16%|█▌        | 5/31 [00:02<00:10,  2.51it/s]
 19%|█▉        | 6/31 [00:02<00:09,  2.53it/s]
 23%|██▎       | 7/31 [00:02<00:09,  2.55it/s]
 26%|██▌       | 8/31 [00:03<00:08,  2.56it/s]
 29%|██▉       | 9/31 [00:03<00:08,  2.57it/s]
 32%|███▏      | 10/31 [00:03<00:08,  2.57it/s]
 35%|███▌      | 11/31 [00:04<00:07,  2.58it/s]
 39%|███▊      | 12/31 [00:04<00:07,  2.58it/s]
 42%|████▏     | 13/31 [00:05<00:06,  2.58it/s]
 45%|████▌     | 14/31 [00:05<00:06,  2.58it/s]
 48%|████▊     | 15/31 [00:05<00:06,  2.58it/s]
 52%|█████▏    | 16/31 [00:06<00:05,  2.58it/s]
 55%|█████▍    | 17/31 [00:06<00:05,  2.58it/s]
 58%|█████▊    | 18/31 [00:07<00:05,  2.58it/s]
 61%|██████▏   | 19/31 [00:07<00:04,  2.58it/s]
 65%|██████▍   | 20/31 [00:07<00:04,  2.58it/s]
 68%|██████▊   | 21/31 [00:08<00:03,  2.58it/s]
 71%|███████   | 22/31 [00:08<00:03,  2.58it/s]
 74%|███████▍  | 23/31 [00:09<00:03,  2.58it/s]
 77%|███████▋  | 24/31 [00:09<00:02,  2.58it/s]
 81%|████████  | 25/31 [00:09<00:02,  2.59it/s]
 84%|████████▍ | 26/31 [00:10<00:01,  2.59it/s]
 87%|████████▋ | 27/31 [00:10<00:01,  2.59it/s]
 90%|█████████ | 28/31 [00:10<00:01,  2.59it/s]
 94%|█████████▎| 29/31 [00:11<00:00,  2.58it/s]
 97%|█████████▋| 30/31 [00:11<00:00,  2.58it/s]
100%|██████████| 31/31 [00:12<00:00,  2.59it/s]
100%|██████████| 31/31 [00:12<00:00,  2.56it/s]
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:18<07:28, 18.70s/it]
  8%|▊         | 2/25 [00:37<07:15, 18.94s/it]
 12%|█▏        | 3/25 [00:56<06:58, 19.02s/it]
 16%|█▌        | 4/25 [01:16<06:39, 19.04s/it]
 20%|██        | 5/25 [01:35<06:21, 19.06s/it]
 24%|██▍       | 6/25 [01:54<06:02, 19.08s/it]
 28%|██▊       | 7/25 [02:13<05:43, 19.09s/it]
 32%|███▏      | 8/25 [02:32<05:24, 19.09s/it]
 36%|███▌      | 9/25 [02:51<05:05, 19.09s/it]
 40%|████      | 10/25 [03:10<04:46, 19.09s/it]
 44%|████▍     | 11/25 [03:29<04:27, 19.10s/it]
 48%|████▊     | 12/25 [03:48<04:08, 19.10s/it]
 52%|█████▏    | 13/25 [04:07<03:49, 19.10s/it]
 56%|█████▌    | 14/25 [04:27<03:30, 19.10s/it]
 60%|██████    | 15/25 [04:46<03:10, 19.10s/it]
 64%|██████▍   | 16/25 [05:05<02:51, 19.10s/it]
 68%|██████▊   | 17/25 [05:24<02:32, 19.10s/it]
 72%|███████▏  | 18/25 [05:43<02:13, 19.10s/it]
 76%|███████▌  | 19/25 [06:02<01:54, 19.10s/it]
 80%|████████  | 20/25 [06:21<01:35, 19.10s/it]
 84%|████████▍ | 21/25 [06:40<01:16, 19.10s/it]
 88%|████████▊ | 22/25 [06:59<00:57, 19.11s/it]
 92%|█████████▏| 23/25 [07:18<00:38, 19.11s/it]
 96%|█████████▌| 24/25 [07:38<00:19, 19.11s/it]
100%|██████████| 25/25 [07:57<00:00, 19.11s/it]
100%|██████████| 25/25 [07:57<00:00, 19.09s/it]
  0%|          | 0/31 [00:00<?, ?it/s]
  3%|▎         | 1/31 [00:02<01:00,  2.03s/it]
  6%|▋         | 2/31 [00:03<00:51,  1.79s/it]
 10%|▉         | 3/31 [00:05<00:47,  1.71s/it]
 13%|█▎        | 4/31 [00:06<00:45,  1.68s/it]
 16%|█▌        | 5/31 [00:08<00:43,  1.66s/it]
 19%|█▉        | 6/31 [00:10<00:41,  1.65s/it]
 23%|██▎       | 7/31 [00:11<00:39,  1.64s/it]
 26%|██▌       | 8/31 [00:13<00:37,  1.63s/it]
 29%|██▉       | 9/31 [00:15<00:35,  1.63s/it]
 32%|███▏      | 10/31 [00:16<00:34,  1.63s/it]
 35%|███▌      | 11/31 [00:18<00:32,  1.63s/it]
 39%|███▊      | 12/31 [00:19<00:30,  1.63s/it]
 42%|████▏     | 13/31 [00:21<00:29,  1.63s/it]
 45%|████▌     | 14/31 [00:23<00:27,  1.63s/it]
 48%|████▊     | 15/31 [00:24<00:26,  1.63s/it]
 52%|█████▏    | 16/31 [00:26<00:24,  1.62s/it]
 55%|█████▍    | 17/31 [00:28<00:22,  1.62s/it]
 58%|█████▊    | 18/31 [00:29<00:21,  1.62s/it]
 61%|██████▏   | 19/31 [00:31<00:19,  1.62s/it]
 65%|██████▍   | 20/31 [00:32<00:17,  1.62s/it]
 68%|██████▊   | 21/31 [00:34<00:16,  1.62s/it]
 71%|███████   | 22/31 [00:36<00:14,  1.62s/it]
 74%|███████▍  | 23/31 [00:37<00:12,  1.62s/it]
 77%|███████▋  | 24/31 [00:39<00:11,  1.62s/it]
 81%|████████  | 25/31 [00:41<00:09,  1.62s/it]
 84%|████████▍ | 26/31 [00:42<00:08,  1.62s/it]
 87%|████████▋ | 27/31 [00:44<00:06,  1.62s/it]
 90%|█████████ | 28/31 [00:45<00:04,  1.62s/it]
 94%|█████████▎| 29/31 [00:47<00:03,  1.62s/it]
 97%|█████████▋| 30/31 [00:49<00:01,  1.62s/it]
100%|██████████| 31/31 [00:50<00:00,  1.62s/it]
100%|██████████| 31/31 [00:50<00:00,  1.64s/it]
Saving video:   0%|          | 0/128 [00:00<?, ?it/s]
Saving video:   1%|          | 1/128 [00:00<00:38,  3.30it/s]
Saving video:  38%|███▊      | 48/128 [00:00<00:00, 152.46it/s]
Saving video:  59%|█████▉    | 76/128 [00:00<00:00, 179.40it/s]
Saving video:  80%|███████▉  | 102/128 [00:00<00:00, 181.31it/s]
Saving video:  98%|█████████▊| 125/128 [00:00<00:00, 175.95it/s]
Saving video: 100%|██████████| 128/128 [00:00<00:00, 154.66it/s]
Saving video:   0%|          | 0/128 [00:00<?, ?it/s]
Saving video:   1%|          | 1/128 [00:00<00:22,  5.67it/s]
Saving video:  12%|█▎        | 16/128 [00:00<00:01, 69.54it/s]
Saving video:  27%|██▋       | 34/128 [00:00<00:00, 111.81it/s]
Saving video:  37%|███▋      | 47/128 [00:00<00:00, 98.22it/s] 
Saving video:  46%|████▌     | 59/128 [00:00<00:00, 101.97it/s]
Saving video:  55%|█████▌    | 71/128 [00:00<00:00, 68.95it/s] 
Saving video:  62%|██████▎   | 80/128 [00:01<00:00, 61.61it/s]
Saving video:  69%|██████▉   | 88/128 [00:01<00:00, 57.51it/s]
Saving video:  74%|███████▍  | 95/128 [00:01<00:00, 50.42it/s]
Saving video:  79%|███████▉  | 101/128 [00:01<00:00, 51.70it/s]
Saving video:  84%|████████▍ | 108/128 [00:01<00:00, 55.35it/s]
Saving video:  90%|████████▉ | 115/128 [00:01<00:00, 49.30it/s]
Saving video:  95%|█████████▍| 121/128 [00:02<00:00, 44.93it/s]
Saving video: 100%|██████████| 128/128 [00:02<00:00, 60.15it/s]

Examples

View more examples

Run time and cost

This model costs approximately $1.11 to run on Replicate, or 0 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 14 minutes.

Readme

DiffSynth Studio

Introduction

DiffSynth Studio is a Diffusion engine. We have restructured architectures including Text Encoder, UNet, VAE, among others, maintaining compatibility with models from the open-source community while enhancing computational performance. We provide many interesting features. Enjoy the magic of Diffusion models!

This demo supports ExVideo

Long Video Synthesis

We trained an extended video synthesis model, which can generate 128 frames

https://github.com/modelscope/DiffSynth-Studio/assets/35051019/d97f6aa9-8064-4b5b-9d49-ed6001bb9acc