cjwbw/sadtalker – Run with an API on Replicate

cjwbw / sadtalker

Stylized Audio-Driven Single Image Talking Face Animation

Cold

Public
126.8K runs
A100 (80GB)
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

source_image

*file

Preview

Upload the source image, it can be video.mp4 or picture.png

driven_audio

*file

Preview

Video Player is loading.

Current Time 00:00:000

Duration 00:00:000

Loaded: 0%

Stream Type LIVE

Remaining Time 00:00:000

Upload the driven audio, accepts .wav and .mp4 file

use_enhancer

boolean

Use GFPGAN as Face enhancer

Default: false

pose_style

integer

(minimum: 0, maximum: 45)

Pose style

Default: 0

expression_scale

number

a larger value will make the expression motion stronger

Default: 1

use_eyeblink

boolean

Use eye blink

Default: true

preprocess

string

Choose how to preprocess the images

Default: "crop"

size_of_image

integer

Face model resolution

Default: 256

facerender

string

Choose face render

Default: "facevid2vid"

still_mode

boolean

Still Mode (fewer head motion, works with preprocess 'full')

Default: true

Run this model in Node.js with one line of code:

npx create-replicate --model=cjwbw/sadtalker

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run cjwbw/sadtalker using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "cjwbw/sadtalker:a519cc0cfebaaeade068b23899165a11ec76aaa1d2b313d40d214f204ec957a3",
  {
    input: {
      facerender: "facevid2vid",
      pose_style: 0,
      preprocess: "crop",
      still_mode: true,
      driven_audio: "https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav",
      source_image: "https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png",
      use_enhancer: true,
      use_eyeblink: true,
      size_of_image: 256,
      expression_scale: 1
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run cjwbw/sadtalker using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "cjwbw/sadtalker:a519cc0cfebaaeade068b23899165a11ec76aaa1d2b313d40d214f204ec957a3",
    input={
        "facerender": "facevid2vid",
        "pose_style": 0,
        "preprocess": "crop",
        "still_mode": True,
        "driven_audio": "https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav",
        "source_image": "https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png",
        "use_enhancer": True,
        "use_eyeblink": True,
        "size_of_image": 256,
        "expression_scale": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run cjwbw/sadtalker using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "a519cc0cfebaaeade068b23899165a11ec76aaa1d2b313d40d214f204ec957a3",
    "input": {
      "facerender": "facevid2vid",
      "pose_style": 0,
      "preprocess": "crop",
      "still_mode": true,
      "driven_audio": "https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav",
      "source_image": "https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png",
      "use_enhancer": true,
      "use_eyeblink": true,
      "size_of_image": 256,
      "expression_scale": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/cjwbw/sadtalker@sha256:a519cc0cfebaaeade068b23899165a11ec76aaa1d2b313d40d214f204ec957a3 \
  -i 'facerender="facevid2vid"' \
  -i 'pose_style=0' \
  -i 'preprocess="crop"' \
  -i 'still_mode=true' \
  -i 'driven_audio="https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav"' \
  -i 'source_image="https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png"' \
  -i 'use_enhancer=true' \
  -i 'use_eyeblink=true' \
  -i 'size_of_image=256' \
  -i 'expression_scale=1'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/cjwbw/sadtalker@sha256:a519cc0cfebaaeade068b23899165a11ec76aaa1d2b313d40d214f204ec957a3
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "facerender": "facevid2vid",
      "pose_style": 0,
      "preprocess": "crop",
      "still_mode": true,
      "driven_audio": "https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav",
      "source_image": "https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png",
      "use_enhancer": true,
      "use_eyeblink": true,
      "size_of_image": 256,
      "expression_scale": 1
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2024-06-01T15:03:53.073327Z",
  "created_at": "2024-06-01T15:00:19.765000Z",
  "data_removed": false,
  "error": null,
  "id": "tewrn6ptpnrgm0cfth885r399c",
  "input": {
    "facerender": "facevid2vid",
    "pose_style": 0,
    "preprocess": "crop",
    "still_mode": true,
    "driven_audio": "https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav",
    "source_image": "https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png",
    "use_enhancer": true,
    "use_eyeblink": true,
    "size_of_image": 256,
    "expression_scale": 1
  },
  "logs": "WARNING: The new version of the model will be updated by safetensor, you may need to download it mannully. We run the old version of the checkpoint this time!\nlandmark Det::   0%|          | 0/1 [00:00<?, ?it/s]\nlandmark Det:: 100%|██████████| 1/1 [00:00<00:00, 17.04it/s]\n3DMM Extraction In Video::   0%|          | 0/1 [00:00<?, ?it/s]\n3DMM Extraction In Video:: 100%|██████████| 1/1 [00:00<00:00, 70.24it/s]\nmel::   0%|          | 0/227 [00:00<?, ?it/s]\nmel:: 100%|██████████| 227/227 [00:00<00:00, 34730.69it/s]\naudio2exp::   0%|          | 0/23 [00:00<?, ?it/s]\naudio2exp:: 100%|██████████| 23/23 [00:00<00:00, 332.65it/s]\nFace Renderer::   0%|          | 0/227 [00:00<?, ?it/s]\nFace Renderer::   1%|▏         | 3/227 [00:00<00:08, 26.44it/s]\nFace Renderer::   3%|▎         | 6/227 [00:00<00:07, 27.75it/s]\nFace Renderer::   4%|▍         | 10/227 [00:00<00:07, 29.33it/s]\nFace Renderer::   6%|▌         | 14/227 [00:00<00:07, 29.90it/s]\nFace Renderer::   8%|▊         | 18/227 [00:00<00:06, 30.24it/s]\nFace Renderer::  10%|▉         | 22/227 [00:00<00:06, 30.44it/s]\nFace Renderer::  11%|█▏        | 26/227 [00:00<00:06, 30.57it/s]\nFace Renderer::  13%|█▎        | 30/227 [00:00<00:06, 30.55it/s]\nFace Renderer::  15%|█▍        | 34/227 [00:01<00:06, 30.63it/s]\nFace Renderer::  17%|█▋        | 38/227 [00:01<00:06, 30.23it/s]\nFace Renderer::  19%|█▊        | 42/227 [00:01<00:06, 30.18it/s]\nFace Renderer::  20%|██        | 46/227 [00:01<00:05, 30.34it/s]\nFace Renderer::  22%|██▏       | 50/227 [00:01<00:05, 29.92it/s]\nFace Renderer::  24%|██▍       | 54/227 [00:01<00:05, 30.20it/s]\nFace Renderer::  26%|██▌       | 58/227 [00:01<00:05, 30.37it/s]\nFace Renderer::  27%|██▋       | 62/227 [00:02<00:05, 30.49it/s]\nFace Renderer::  29%|██▉       | 66/227 [00:02<00:05, 30.60it/s]\nFace Renderer::  31%|███       | 70/227 [00:02<00:05, 30.63it/s]\nFace Renderer::  33%|███▎      | 74/227 [00:02<00:04, 30.66it/s]\nFace Renderer::  34%|███▍      | 78/227 [00:02<00:04, 30.63it/s]\nFace Renderer::  36%|███▌      | 82/227 [00:02<00:04, 30.71it/s]\nFace Renderer::  38%|███▊      | 86/227 [00:02<00:04, 30.75it/s]\nFace Renderer::  40%|███▉      | 90/227 [00:02<00:04, 30.68it/s]\nFace Renderer::  41%|████▏     | 94/227 [00:03<00:04, 30.72it/s]\nFace Renderer::  43%|████▎     | 98/227 [00:03<00:04, 29.85it/s]\nFace Renderer::  45%|████▍     | 102/227 [00:03<00:04, 30.10it/s]\nFace Renderer::  47%|████▋     | 106/227 [00:03<00:03, 30.31it/s]\nFace Renderer::  48%|████▊     | 110/227 [00:03<00:03, 30.45it/s]\nFace Renderer::  50%|█████     | 114/227 [00:03<00:03, 30.51it/s]\nFace Renderer::  52%|█████▏    | 118/227 [00:03<00:03, 30.62it/s]\nFace Renderer::  54%|█████▎    | 122/227 [00:04<00:03, 30.66it/s]\nFace Renderer::  56%|█████▌    | 126/227 [00:04<00:03, 30.40it/s]\nFace Renderer::  57%|█████▋    | 130/227 [00:04<00:03, 29.88it/s]\nFace Renderer::  59%|█████▉    | 134/227 [00:04<00:03, 30.10it/s]\nFace Renderer::  61%|██████    | 138/227 [00:04<00:02, 30.16it/s]\nFace Renderer::  63%|██████▎   | 142/227 [00:04<00:02, 30.30it/s]\nFace Renderer::  64%|██████▍   | 146/227 [00:04<00:02, 30.46it/s]\nFace Renderer::  66%|██████▌   | 150/227 [00:04<00:02, 30.58it/s]\nFace Renderer::  68%|██████▊   | 154/227 [00:05<00:02, 30.59it/s]\nFace Renderer::  70%|██████▉   | 158/227 [00:05<00:02, 30.54it/s]\nFace Renderer::  71%|███████▏  | 162/227 [00:05<00:02, 30.68it/s]\nFace Renderer::  73%|███████▎  | 166/227 [00:05<00:01, 30.62it/s]\nFace Renderer::  75%|███████▍  | 170/227 [00:05<00:01, 30.70it/s]\nFace Renderer::  77%|███████▋  | 174/227 [00:05<00:01, 30.72it/s]\nFace Renderer::  78%|███████▊  | 178/227 [00:05<00:01, 30.69it/s]\nFace Renderer::  80%|████████  | 182/227 [00:05<00:01, 30.75it/s]\nFace Renderer::  82%|████████▏ | 186/227 [00:06<00:01, 30.77it/s]\nFace Renderer::  84%|████████▎ | 190/227 [00:06<00:01, 30.67it/s]\nFace Renderer::  85%|████████▌ | 194/227 [00:06<00:01, 30.76it/s]\nFace Renderer::  87%|████████▋ | 198/227 [00:06<00:00, 30.76it/s]\nFace Renderer::  89%|████████▉ | 202/227 [00:06<00:00, 30.72it/s]\nFace Renderer::  91%|█████████ | 206/227 [00:06<00:00, 30.69it/s]\nFace Renderer::  93%|█████████▎| 210/227 [00:06<00:00, 30.74it/s]\nFace Renderer::  94%|█████████▍| 214/227 [00:07<00:00, 30.44it/s]\nFace Renderer::  96%|█████████▌| 218/227 [00:07<00:00, 30.75it/s]\nFace Renderer::  98%|█████████▊| 222/227 [00:07<00:00, 30.56it/s]\nFace Renderer:: 100%|█████████▉| 226/227 [00:07<00:00, 30.62it/s]\nFace Renderer:: 100%|██████████| 227/227 [00:07<00:00, 30.45it/s]\nIMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (256, 226) to (256, 240) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).\nThe generated video is named exp_dir/tmpc3kuukfvart_1##tmpvz4viuhbjapanese.mp4\nface enhancer....\nDownloading: \"https://github.com/xinntao/facexlib/releases/download/v0.2.2/parsing_parsenet.pth\" to /src/gfpgan/weights/parsing_parsenet.pth\n  0%|          | 0.00/81.4M [00:00<?, ?B/s]\n 42%|████▏     | 34.3M/81.4M [00:00<00:00, 360MB/s]\n 90%|█████████ | 73.3M/81.4M [00:00<00:00, 388MB/s]\n100%|██████████| 81.4M/81.4M [00:00<00:00, 385MB/s]\nDownloading: \"https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth\" to /root/.pyenv/versions/3.8.19/lib/python3.8/site-packages/gfpgan/weights/GFPGANv1.4.pth\n  0%|          | 0.00/332M [00:00<?, ?B/s]\n 11%|█         | 36.8M/332M [00:00<00:00, 385MB/s]\n 23%|██▎       | 77.3M/332M [00:00<00:00, 409MB/s]\n 35%|███▌      | 117M/332M [00:00<00:00, 413MB/s] \n 47%|████▋     | 157M/332M [00:00<00:00, 416MB/s]\n 59%|█████▉    | 198M/332M [00:00<00:00, 419MB/s]\n 72%|███████▏  | 238M/332M [00:00<00:00, 420MB/s]\n 84%|████████▎ | 278M/332M [00:00<00:00, 419MB/s]\n 96%|█████████▌| 318M/332M [00:00<00:00, 418MB/s]\n100%|██████████| 332M/332M [00:00<00:00, 416MB/s]\nFace Enhancer::   0%|          | 0/227 [00:00<?, ?it/s]\nFace Enhancer::   0%|          | 1/227 [00:00<01:08,  3.32it/s]\nFace Enhancer::   1%|          | 2/227 [00:00<00:51,  4.38it/s]\nFace Enhancer::   1%|▏         | 3/227 [00:00<00:44,  5.06it/s]\nFace Enhancer::   2%|▏         | 4/227 [00:00<00:41,  5.41it/s]\nFace Enhancer::   2%|▏         | 5/227 [00:01<00:44,  5.04it/s]\nFace Enhancer::   3%|▎         | 6/227 [00:01<00:41,  5.34it/s]\nFace Enhancer::   3%|▎         | 7/227 [00:01<00:39,  5.59it/s]\nFace Enhancer::   4%|▎         | 8/227 [00:01<00:39,  5.58it/s]\nFace Enhancer::   4%|▍         | 9/227 [00:01<00:37,  5.82it/s]\nFace Enhancer::   4%|▍         | 10/227 [00:01<00:37,  5.77it/s]\nFace Enhancer::   5%|▍         | 11/227 [00:02<00:37,  5.73it/s]\nFace Enhancer::   5%|▌         | 12/227 [00:02<00:36,  5.90it/s]\nFace Enhancer::   6%|▌         | 13/227 [00:02<00:35,  5.99it/s]\nFace Enhancer::   6%|▌         | 14/227 [00:02<00:35,  5.96it/s]\nFace Enhancer::   7%|▋         | 15/227 [00:02<00:34,  6.11it/s]\nFace Enhancer::   7%|▋         | 16/227 [00:02<00:34,  6.17it/s]\nFace Enhancer::   7%|▋         | 17/227 [00:03<00:34,  6.09it/s]\nFace Enhancer::   8%|▊         | 18/227 [00:03<00:33,  6.19it/s]\nFace Enhancer::   8%|▊         | 19/227 [00:03<00:33,  6.26it/s]\nFace Enhancer::   9%|▉         | 20/227 [00:03<00:33,  6.10it/s]\nFace Enhancer::   9%|▉         | 21/227 [00:03<00:33,  6.20it/s]\nFace Enhancer::  10%|▉         | 22/227 [00:03<00:32,  6.27it/s]\nFace Enhancer::  10%|█         | 23/227 [00:03<00:33,  6.16it/s]\nFace Enhancer::  11%|█         | 24/227 [00:04<00:32,  6.21it/s]\nFace Enhancer::  11%|█         | 25/227 [00:04<00:32,  6.25it/s]\nFace Enhancer::  11%|█▏        | 26/227 [00:04<00:32,  6.11it/s]\nFace Enhancer::  12%|█▏        | 27/227 [00:04<00:32,  6.18it/s]\nFace Enhancer::  12%|█▏        | 28/227 [00:04<00:33,  6.01it/s]\nFace Enhancer::  13%|█▎        | 29/227 [00:04<00:33,  5.96it/s]\nFace Enhancer::  13%|█▎        | 30/227 [00:05<00:32,  6.09it/s]\nFace Enhancer::  14%|█▎        | 31/227 [00:05<00:31,  6.18it/s]\nFace Enhancer::  14%|█▍        | 32/227 [00:05<00:31,  6.10it/s]\nFace Enhancer::  15%|█▍        | 33/227 [00:05<00:31,  6.19it/s]\nFace Enhancer::  15%|█▍        | 34/227 [00:05<00:31,  6.07it/s]\nFace Enhancer::  15%|█▌        | 35/227 [00:05<00:32,  5.82it/s]\nFace Enhancer::  16%|█▌        | 36/227 [00:06<00:32,  5.95it/s]\nFace Enhancer::  16%|█▋        | 37/227 [00:06<00:31,  6.09it/s]\nFace Enhancer::  17%|█▋        | 38/227 [00:06<00:31,  6.02it/s]\nFace Enhancer::  17%|█▋        | 39/227 [00:06<00:30,  6.11it/s]\nFace Enhancer::  18%|█▊        | 40/227 [00:06<00:30,  6.04it/s]\nFace Enhancer::  18%|█▊        | 41/227 [00:06<00:30,  6.01it/s]\nFace Enhancer::  19%|█▊        | 42/227 [00:07<00:30,  6.10it/s]\nFace Enhancer::  19%|█▉        | 43/227 [00:07<00:34,  5.38it/s]\nFace Enhancer::  19%|█▉        | 44/227 [00:07<00:33,  5.49it/s]\nFace Enhancer::  20%|█▉        | 45/227 [00:07<00:33,  5.46it/s]\nFace Enhancer::  20%|██        | 46/227 [00:07<00:32,  5.56it/s]\nFace Enhancer::  21%|██        | 47/227 [00:08<00:31,  5.66it/s]\nFace Enhancer::  21%|██        | 48/227 [00:08<00:30,  5.85it/s]\nFace Enhancer::  22%|██▏       | 49/227 [00:08<00:29,  6.00it/s]\nFace Enhancer::  22%|██▏       | 50/227 [00:08<00:29,  5.97it/s]\nFace Enhancer::  22%|██▏       | 51/227 [00:08<00:29,  5.90it/s]\nFace Enhancer::  23%|██▎       | 52/227 [00:08<00:30,  5.81it/s]\nFace Enhancer::  23%|██▎       | 53/227 [00:09<00:30,  5.66it/s]\nFace Enhancer::  24%|██▍       | 54/227 [00:09<00:32,  5.38it/s]\nFace Enhancer::  24%|██▍       | 55/227 [00:09<00:30,  5.67it/s]\nFace Enhancer::  25%|██▍       | 56/227 [00:09<00:29,  5.76it/s]\nFace Enhancer::  25%|██▌       | 57/227 [00:09<00:29,  5.74it/s]\nFace Enhancer::  26%|██▌       | 58/227 [00:09<00:28,  5.91it/s]\nFace Enhancer::  26%|██▌       | 59/227 [00:10<00:28,  5.91it/s]\nFace Enhancer::  26%|██▋       | 60/227 [00:10<00:28,  5.94it/s]\nFace Enhancer::  27%|██▋       | 61/227 [00:10<00:27,  6.09it/s]\nFace Enhancer::  27%|██▋       | 62/227 [00:10<00:27,  6.01it/s]\nFace Enhancer::  28%|██▊       | 63/227 [00:10<00:27,  5.93it/s]\nFace Enhancer::  28%|██▊       | 64/227 [00:10<00:27,  5.86it/s]\nFace Enhancer::  29%|██▊       | 65/227 [00:11<00:28,  5.74it/s]\nFace Enhancer::  29%|██▉       | 66/227 [00:11<00:28,  5.70it/s]\nFace Enhancer::  30%|██▉       | 67/227 [00:11<00:28,  5.57it/s]\nFace Enhancer::  30%|██▉       | 68/227 [00:11<00:29,  5.46it/s]\nFace Enhancer::  30%|███       | 69/227 [00:11<00:29,  5.32it/s]\nFace Enhancer::  31%|███       | 70/227 [00:12<00:28,  5.42it/s]\nFace Enhancer::  31%|███▏      | 71/227 [00:12<00:28,  5.55it/s]\nFace Enhancer::  32%|███▏      | 72/227 [00:12<00:27,  5.67it/s]\nFace Enhancer::  32%|███▏      | 73/227 [00:12<00:26,  5.80it/s]\nFace Enhancer::  33%|███▎      | 74/227 [00:12<00:27,  5.61it/s]\nFace Enhancer::  33%|███▎      | 75/227 [00:12<00:28,  5.42it/s]\nFace Enhancer::  33%|███▎      | 76/227 [00:13<00:28,  5.27it/s]\nFace Enhancer::  34%|███▍      | 77/227 [00:13<00:27,  5.40it/s]\nFace Enhancer::  34%|███▍      | 78/227 [00:13<00:26,  5.62it/s]\nFace Enhancer::  35%|███▍      | 79/227 [00:13<00:25,  5.84it/s]\nFace Enhancer::  35%|███▌      | 80/227 [00:13<00:25,  5.87it/s]\nFace Enhancer::  36%|███▌      | 81/227 [00:13<00:24,  6.03it/s]\nFace Enhancer::  36%|███▌      | 82/227 [00:14<00:24,  6.01it/s]\nFace Enhancer::  37%|███▋      | 83/227 [00:14<00:24,  5.79it/s]\nFace Enhancer::  37%|███▋      | 84/227 [00:14<00:24,  5.74it/s]\nFace Enhancer::  37%|███▋      | 85/227 [00:14<00:23,  5.94it/s]\nFace Enhancer::  38%|███▊      | 86/227 [00:14<00:23,  5.93it/s]\nFace Enhancer::  38%|███▊      | 87/227 [00:14<00:23,  6.08it/s]\nFace Enhancer::  39%|███▉      | 88/227 [00:15<00:22,  6.20it/s]\nFace Enhancer::  39%|███▉      | 89/227 [00:15<00:22,  6.11it/s]\nFace Enhancer::  40%|███▉      | 90/227 [00:15<00:22,  6.13it/s]\nFace Enhancer::  40%|████      | 91/227 [00:15<00:21,  6.22it/s]\nFace Enhancer::  41%|████      | 92/227 [00:15<00:23,  5.80it/s]\nFace Enhancer::  41%|████      | 93/227 [00:15<00:22,  5.99it/s]\nFace Enhancer::  41%|████▏     | 94/227 [00:16<00:21,  6.13it/s]\nFace Enhancer::  42%|████▏     | 95/227 [00:16<00:21,  6.00it/s]\nFace Enhancer::  42%|████▏     | 96/227 [00:16<00:21,  6.13it/s]\nFace Enhancer::  43%|████▎     | 97/227 [00:16<00:20,  6.23it/s]\nFace Enhancer::  43%|████▎     | 98/227 [00:16<00:21,  5.94it/s]\nFace Enhancer::  44%|████▎     | 99/227 [00:16<00:21,  6.04it/s]\nFace Enhancer::  44%|████▍     | 100/227 [00:17<00:20,  6.13it/s]\nFace Enhancer::  44%|████▍     | 101/227 [00:17<00:20,  6.07it/s]\nFace Enhancer::  45%|████▍     | 102/227 [00:17<00:20,  6.04it/s]\nFace Enhancer::  45%|████▌     | 103/227 [00:17<00:20,  6.17it/s]\nFace Enhancer::  46%|████▌     | 104/227 [00:17<00:20,  5.91it/s]\nFace Enhancer::  46%|████▋     | 105/227 [00:17<00:20,  6.07it/s]\nFace Enhancer::  47%|████▋     | 106/227 [00:18<00:20,  6.04it/s]\nFace Enhancer::  47%|████▋     | 107/227 [00:18<00:19,  6.01it/s]\nFace Enhancer::  48%|████▊     | 108/227 [00:18<00:20,  5.91it/s]\nFace Enhancer::  48%|████▊     | 109/227 [00:18<00:20,  5.85it/s]\nFace Enhancer::  48%|████▊     | 110/227 [00:18<00:20,  5.70it/s]\nFace Enhancer::  49%|████▉     | 111/227 [00:18<00:19,  5.92it/s]\nFace Enhancer::  49%|████▉     | 112/227 [00:19<00:19,  6.05it/s]\nFace Enhancer::  50%|████▉     | 113/227 [00:19<00:18,  6.01it/s]\nFace Enhancer::  50%|█████     | 114/227 [00:19<00:18,  6.13it/s]\nFace Enhancer::  51%|█████     | 115/227 [00:19<00:17,  6.23it/s]\nFace Enhancer::  51%|█████     | 116/227 [00:19<00:18,  6.15it/s]\nFace Enhancer::  52%|█████▏    | 117/227 [00:19<00:17,  6.25it/s]\nFace Enhancer::  52%|█████▏    | 118/227 [00:20<00:17,  6.22it/s]\nFace Enhancer::  52%|█████▏    | 119/227 [00:20<00:17,  6.13it/s]\nFace Enhancer::  53%|█████▎    | 120/227 [00:20<00:17,  6.20it/s]\nFace Enhancer::  53%|█████▎    | 121/227 [00:20<00:16,  6.27it/s]\nFace Enhancer::  54%|█████▎    | 122/227 [00:20<00:17,  6.15it/s]\nFace Enhancer::  54%|█████▍    | 123/227 [00:20<00:16,  6.24it/s]\nFace Enhancer::  55%|█████▍    | 124/227 [00:21<00:16,  6.26it/s]\nFace Enhancer::  55%|█████▌    | 125/227 [00:21<00:16,  6.15it/s]\nFace Enhancer::  56%|█████▌    | 126/227 [00:21<00:16,  6.23it/s]\nFace Enhancer::  56%|█████▌    | 127/227 [00:21<00:15,  6.29it/s]\nFace Enhancer::  56%|█████▋    | 128/227 [00:21<00:16,  6.16it/s]\nFace Enhancer::  57%|█████▋    | 129/227 [00:21<00:15,  6.24it/s]\nFace Enhancer::  57%|█████▋    | 130/227 [00:22<00:15,  6.25it/s]\nFace Enhancer::  58%|█████▊    | 131/227 [00:22<00:15,  6.09it/s]\nFace Enhancer::  58%|█████▊    | 132/227 [00:22<00:15,  6.20it/s]\nFace Enhancer::  59%|█████▊    | 133/227 [00:22<00:14,  6.28it/s]\nFace Enhancer::  59%|█████▉    | 134/227 [00:22<00:15,  6.18it/s]\nFace Enhancer::  59%|█████▉    | 135/227 [00:22<00:14,  6.26it/s]\nFace Enhancer::  60%|█████▉    | 136/227 [00:22<00:14,  6.29it/s]\nFace Enhancer::  60%|██████    | 137/227 [00:23<00:14,  6.16it/s]\nFace Enhancer::  61%|██████    | 138/227 [00:23<00:14,  6.08it/s]\nFace Enhancer::  61%|██████    | 139/227 [00:23<00:14,  6.17it/s]\nFace Enhancer::  62%|██████▏   | 140/227 [00:23<00:14,  6.08it/s]\nFace Enhancer::  62%|██████▏   | 141/227 [00:23<00:14,  5.98it/s]\nFace Enhancer::  63%|██████▎   | 142/227 [00:23<00:13,  6.11it/s]\nFace Enhancer::  63%|██████▎   | 143/227 [00:24<00:14,  5.84it/s]\nFace Enhancer::  63%|██████▎   | 144/227 [00:24<00:13,  5.99it/s]\nFace Enhancer::  64%|██████▍   | 145/227 [00:24<00:13,  6.10it/s]\nFace Enhancer::  64%|██████▍   | 146/227 [00:24<00:13,  6.03it/s]\nFace Enhancer::  65%|██████▍   | 147/227 [00:24<00:13,  5.94it/s]\nFace Enhancer::  65%|██████▌   | 148/227 [00:24<00:13,  6.06it/s]\nFace Enhancer::  66%|██████▌   | 149/227 [00:25<00:13,  5.99it/s]\nFace Enhancer::  66%|██████▌   | 150/227 [00:25<00:12,  6.10it/s]\nFace Enhancer::  67%|██████▋   | 151/227 [00:25<00:13,  5.79it/s]\nFace Enhancer::  67%|██████▋   | 152/227 [00:25<00:12,  5.81it/s]\nFace Enhancer::  67%|██████▋   | 153/227 [00:25<00:12,  5.98it/s]\nFace Enhancer::  68%|██████▊   | 154/227 [00:25<00:12,  6.06it/s]\nFace Enhancer::  68%|██████▊   | 155/227 [00:26<00:11,  6.00it/s]\nFace Enhancer::  69%|██████▊   | 156/227 [00:26<00:11,  6.11it/s]\nFace Enhancer::  69%|██████▉   | 157/227 [00:26<00:11,  6.21it/s]\nFace Enhancer::  70%|██████▉   | 158/227 [00:26<00:11,  6.12it/s]\nFace Enhancer::  70%|███████   | 159/227 [00:26<00:11,  6.15it/s]\nFace Enhancer::  70%|███████   | 160/227 [00:26<00:10,  6.23it/s]\nFace Enhancer::  71%|███████   | 161/227 [00:27<00:10,  6.03it/s]\nFace Enhancer::  71%|███████▏  | 162/227 [00:27<00:10,  6.12it/s]\nFace Enhancer::  72%|███████▏  | 163/227 [00:27<00:10,  6.09it/s]\nFace Enhancer::  72%|███████▏  | 164/227 [00:27<00:10,  5.95it/s]\nFace Enhancer::  73%|███████▎  | 165/227 [00:27<00:10,  5.82it/s]\nFace Enhancer::  73%|███████▎  | 166/227 [00:28<00:10,  5.61it/s]\nFace Enhancer::  74%|███████▎  | 167/227 [00:28<00:11,  5.25it/s]\nFace Enhancer::  74%|███████▍  | 168/227 [00:28<00:10,  5.52it/s]\nFace Enhancer::  74%|███████▍  | 169/227 [00:28<00:10,  5.76it/s]\nFace Enhancer::  75%|███████▍  | 170/227 [00:28<00:09,  5.81it/s]\nFace Enhancer::  75%|███████▌  | 171/227 [00:28<00:09,  6.00it/s]\nFace Enhancer::  76%|███████▌  | 172/227 [00:29<00:09,  6.08it/s]\nFace Enhancer::  76%|███████▌  | 173/227 [00:29<00:08,  6.02it/s]\nFace Enhancer::  77%|███████▋  | 174/227 [00:29<00:08,  6.12it/s]\nFace Enhancer::  77%|███████▋  | 175/227 [00:29<00:08,  6.21it/s]\nFace Enhancer::  78%|███████▊  | 176/227 [00:29<00:08,  6.12it/s]\nFace Enhancer::  78%|███████▊  | 177/227 [00:29<00:08,  6.07it/s]\nFace Enhancer::  78%|███████▊  | 178/227 [00:30<00:08,  6.03it/s]\nFace Enhancer::  79%|███████▉  | 179/227 [00:30<00:08,  5.79it/s]\nFace Enhancer::  79%|███████▉  | 180/227 [00:30<00:07,  5.92it/s]\nFace Enhancer::  80%|███████▉  | 181/227 [00:30<00:07,  6.05it/s]\nFace Enhancer::  80%|████████  | 182/227 [00:30<00:07,  5.92it/s]\nFace Enhancer::  81%|████████  | 183/227 [00:30<00:07,  6.07it/s]\nFace Enhancer::  81%|████████  | 184/227 [00:31<00:06,  6.18it/s]\nFace Enhancer::  81%|████████▏ | 185/227 [00:31<00:06,  6.10it/s]\nFace Enhancer::  82%|████████▏ | 186/227 [00:31<00:06,  6.16it/s]\nFace Enhancer::  82%|████████▏ | 187/227 [00:31<00:06,  6.26it/s]\nFace Enhancer::  83%|████████▎ | 188/227 [00:31<00:06,  6.16it/s]\nFace Enhancer::  83%|████████▎ | 189/227 [00:31<00:06,  6.26it/s]\nFace Enhancer::  84%|████████▎ | 190/227 [00:31<00:05,  6.32it/s]\nFace Enhancer::  84%|████████▍ | 191/227 [00:32<00:05,  6.17it/s]\nFace Enhancer::  85%|████████▍ | 192/227 [00:32<00:05,  6.24it/s]\nFace Enhancer::  85%|████████▌ | 193/227 [00:32<00:05,  6.15it/s]\nFace Enhancer::  85%|████████▌ | 194/227 [00:32<00:05,  6.08it/s]\nFace Enhancer::  86%|████████▌ | 195/227 [00:32<00:05,  6.12it/s]\nFace Enhancer::  86%|████████▋ | 196/227 [00:32<00:05,  5.92it/s]\nFace Enhancer::  87%|████████▋ | 197/227 [00:33<00:05,  5.86it/s]\nFace Enhancer::  87%|████████▋ | 198/227 [00:33<00:04,  5.94it/s]\nFace Enhancer::  88%|████████▊ | 199/227 [00:33<00:04,  6.01it/s]\nFace Enhancer::  88%|████████▊ | 200/227 [00:33<00:04,  5.95it/s]\nFace Enhancer::  89%|████████▊ | 201/227 [00:33<00:04,  5.73it/s]\nFace Enhancer::  89%|████████▉ | 202/227 [00:34<00:04,  5.71it/s]\nFace Enhancer::  89%|████████▉ | 203/227 [00:34<00:04,  5.35it/s]\nFace Enhancer::  90%|████████▉ | 204/227 [00:34<00:04,  5.45it/s]\nFace Enhancer::  90%|█████████ | 205/227 [00:34<00:03,  5.68it/s]\nFace Enhancer::  91%|█████████ | 206/227 [00:34<00:03,  5.58it/s]\nFace Enhancer::  91%|█████████ | 207/227 [00:34<00:03,  5.35it/s]\nFace Enhancer::  92%|█████████▏| 208/227 [00:35<00:03,  5.46it/s]\nFace Enhancer::  92%|█████████▏| 209/227 [00:35<00:03,  5.37it/s]\nFace Enhancer::  93%|█████████▎| 210/227 [00:35<00:03,  5.64it/s]\nFace Enhancer::  93%|█████████▎| 211/227 [00:35<00:02,  5.66it/s]\nFace Enhancer::  93%|█████████▎| 212/227 [00:35<00:02,  5.39it/s]\nFace Enhancer::  94%|█████████▍| 213/227 [00:36<00:02,  5.64it/s]\nFace Enhancer::  94%|█████████▍| 214/227 [00:36<00:02,  5.84it/s]\nFace Enhancer::  95%|█████████▍| 215/227 [00:36<00:02,  5.81it/s]\nFace Enhancer::  95%|█████████▌| 216/227 [00:36<00:01,  5.86it/s]\nFace Enhancer::  96%|█████████▌| 217/227 [00:36<00:01,  5.98it/s]\nFace Enhancer::  96%|█████████▌| 218/227 [00:36<00:01,  5.78it/s]\nFace Enhancer::  96%|█████████▋| 219/227 [00:37<00:01,  5.66it/s]\nFace Enhancer::  97%|█████████▋| 220/227 [00:37<00:01,  5.70it/s]\nFace Enhancer::  97%|█████████▋| 221/227 [00:37<00:01,  5.57it/s]\nFace Enhancer::  98%|█████████▊| 222/227 [00:37<00:00,  5.20it/s]\nFace Enhancer::  98%|█████████▊| 223/227 [00:37<00:00,  5.51it/s]\nFace Enhancer::  99%|█████████▊| 224/227 [00:37<00:00,  5.55it/s]\nFace Enhancer::  99%|█████████▉| 225/227 [00:38<00:00,  5.66it/s]\nFace Enhancer:: 100%|█████████▉| 226/227 [00:38<00:00,  5.85it/s]\nFace Enhancer:: 100%|██████████| 227/227 [00:38<00:00,  5.77it/s]\nFace Enhancer:: 100%|██████████| 227/227 [00:38<00:00,  5.90it/s]\nThe generated video is named exp_dir/tmpc3kuukfvart_1##tmpvz4viuhbjapanese_enhanced.mp4\nThe generated video is named tmpc3kuukfvart_1##tmpvz4viuhbjapanese in exp_dir",
  "metrics": {
    "predict_time": 60.372041,
    "total_time": 213.308327
  },
  "output": "https://replicate.delivery/pbxt/SrAFzxkTBL6fCiTKMRMnqtIL7FYZCh321JYkTt8MLGWsUHdJA/out.mp4",
  "started_at": "2024-06-01T15:02:52.701286Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/tewrn6ptpnrgm0cfth885r399c",
    "cancel": "https://api.replicate.com/v1/predictions/tewrn6ptpnrgm0cfth885r399c/cancel"
  },
  "version": "a519cc0cfebaaeade068b23899165a11ec76aaa1d2b313d40d214f204ec957a3"
}

Generated in

60.4 seconds

Tweak it ShareReport

WARNING: The new version of the model will be updated by safetensor, you may need to download it mannully. We run the old version of the checkpoint this time!
landmark Det::   0%|          | 0/1 [00:00<?, ?it/s]
landmark Det:: 100%|██████████| 1/1 [00:00<00:00, 17.04it/s]
3DMM Extraction In Video::   0%|          | 0/1 [00:00<?, ?it/s]
3DMM Extraction In Video:: 100%|██████████| 1/1 [00:00<00:00, 70.24it/s]
mel::   0%|          | 0/227 [00:00<?, ?it/s]
mel:: 100%|██████████| 227/227 [00:00<00:00, 34730.69it/s]
audio2exp::   0%|          | 0/23 [00:00<?, ?it/s]
audio2exp:: 100%|██████████| 23/23 [00:00<00:00, 332.65it/s]
Face Renderer::   0%|          | 0/227 [00:00<?, ?it/s]
Face Renderer::   1%|▏         | 3/227 [00:00<00:08, 26.44it/s]
Face Renderer::   3%|▎         | 6/227 [00:00<00:07, 27.75it/s]
Face Renderer::   4%|▍         | 10/227 [00:00<00:07, 29.33it/s]
Face Renderer::   6%|▌         | 14/227 [00:00<00:07, 29.90it/s]
Face Renderer::   8%|▊         | 18/227 [00:00<00:06, 30.24it/s]
Face Renderer::  10%|▉         | 22/227 [00:00<00:06, 30.44it/s]
Face Renderer::  11%|█▏        | 26/227 [00:00<00:06, 30.57it/s]
Face Renderer::  13%|█▎        | 30/227 [00:00<00:06, 30.55it/s]
Face Renderer::  15%|█▍        | 34/227 [00:01<00:06, 30.63it/s]
Face Renderer::  17%|█▋        | 38/227 [00:01<00:06, 30.23it/s]
Face Renderer::  19%|█▊        | 42/227 [00:01<00:06, 30.18it/s]
Face Renderer::  20%|██        | 46/227 [00:01<00:05, 30.34it/s]
Face Renderer::  22%|██▏       | 50/227 [00:01<00:05, 29.92it/s]
Face Renderer::  24%|██▍       | 54/227 [00:01<00:05, 30.20it/s]
Face Renderer::  26%|██▌       | 58/227 [00:01<00:05, 30.37it/s]
Face Renderer::  27%|██▋       | 62/227 [00:02<00:05, 30.49it/s]
Face Renderer::  29%|██▉       | 66/227 [00:02<00:05, 30.60it/s]
Face Renderer::  31%|███       | 70/227 [00:02<00:05, 30.63it/s]
Face Renderer::  33%|███▎      | 74/227 [00:02<00:04, 30.66it/s]
Face Renderer::  34%|███▍      | 78/227 [00:02<00:04, 30.63it/s]
Face Renderer::  36%|███▌      | 82/227 [00:02<00:04, 30.71it/s]
Face Renderer::  38%|███▊      | 86/227 [00:02<00:04, 30.75it/s]
Face Renderer::  40%|███▉      | 90/227 [00:02<00:04, 30.68it/s]
Face Renderer::  41%|████▏     | 94/227 [00:03<00:04, 30.72it/s]
Face Renderer::  43%|████▎     | 98/227 [00:03<00:04, 29.85it/s]
Face Renderer::  45%|████▍     | 102/227 [00:03<00:04, 30.10it/s]
Face Renderer::  47%|████▋     | 106/227 [00:03<00:03, 30.31it/s]
Face Renderer::  48%|████▊     | 110/227 [00:03<00:03, 30.45it/s]
Face Renderer::  50%|█████     | 114/227 [00:03<00:03, 30.51it/s]
Face Renderer::  52%|█████▏    | 118/227 [00:03<00:03, 30.62it/s]
Face Renderer::  54%|█████▎    | 122/227 [00:04<00:03, 30.66it/s]
Face Renderer::  56%|█████▌    | 126/227 [00:04<00:03, 30.40it/s]
Face Renderer::  57%|█████▋    | 130/227 [00:04<00:03, 29.88it/s]
Face Renderer::  59%|█████▉    | 134/227 [00:04<00:03, 30.10it/s]
Face Renderer::  61%|██████    | 138/227 [00:04<00:02, 30.16it/s]
Face Renderer::  63%|██████▎   | 142/227 [00:04<00:02, 30.30it/s]
Face Renderer::  64%|██████▍   | 146/227 [00:04<00:02, 30.46it/s]
Face Renderer::  66%|██████▌   | 150/227 [00:04<00:02, 30.58it/s]
Face Renderer::  68%|██████▊   | 154/227 [00:05<00:02, 30.59it/s]
Face Renderer::  70%|██████▉   | 158/227 [00:05<00:02, 30.54it/s]
Face Renderer::  71%|███████▏  | 162/227 [00:05<00:02, 30.68it/s]
Face Renderer::  73%|███████▎  | 166/227 [00:05<00:01, 30.62it/s]
Face Renderer::  75%|███████▍  | 170/227 [00:05<00:01, 30.70it/s]
Face Renderer::  77%|███████▋  | 174/227 [00:05<00:01, 30.72it/s]
Face Renderer::  78%|███████▊  | 178/227 [00:05<00:01, 30.69it/s]
Face Renderer::  80%|████████  | 182/227 [00:05<00:01, 30.75it/s]
Face Renderer::  82%|████████▏ | 186/227 [00:06<00:01, 30.77it/s]
Face Renderer::  84%|████████▎ | 190/227 [00:06<00:01, 30.67it/s]
Face Renderer::  85%|████████▌ | 194/227 [00:06<00:01, 30.76it/s]
Face Renderer::  87%|████████▋ | 198/227 [00:06<00:00, 30.76it/s]
Face Renderer::  89%|████████▉ | 202/227 [00:06<00:00, 30.72it/s]
Face Renderer::  91%|█████████ | 206/227 [00:06<00:00, 30.69it/s]
Face Renderer::  93%|█████████▎| 210/227 [00:06<00:00, 30.74it/s]
Face Renderer::  94%|█████████▍| 214/227 [00:07<00:00, 30.44it/s]
Face Renderer::  96%|█████████▌| 218/227 [00:07<00:00, 30.75it/s]
Face Renderer::  98%|█████████▊| 222/227 [00:07<00:00, 30.56it/s]
Face Renderer:: 100%|█████████▉| 226/227 [00:07<00:00, 30.62it/s]
Face Renderer:: 100%|██████████| 227/227 [00:07<00:00, 30.45it/s]
IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (256, 226) to (256, 240) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
The generated video is named exp_dir/tmpc3kuukfvart_1##tmpvz4viuhbjapanese.mp4
face enhancer....
Downloading: "https://github.com/xinntao/facexlib/releases/download/v0.2.2/parsing_parsenet.pth" to /src/gfpgan/weights/parsing_parsenet.pth
  0%|          | 0.00/81.4M [00:00<?, ?B/s]
 42%|████▏     | 34.3M/81.4M [00:00<00:00, 360MB/s]
 90%|█████████ | 73.3M/81.4M [00:00<00:00, 388MB/s]
100%|██████████| 81.4M/81.4M [00:00<00:00, 385MB/s]
Downloading: "https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth" to /root/.pyenv/versions/3.8.19/lib/python3.8/site-packages/gfpgan/weights/GFPGANv1.4.pth
  0%|          | 0.00/332M [00:00<?, ?B/s]
 11%|█         | 36.8M/332M [00:00<00:00, 385MB/s]
 23%|██▎       | 77.3M/332M [00:00<00:00, 409MB/s]
 35%|███▌      | 117M/332M [00:00<00:00, 413MB/s] 
 47%|████▋     | 157M/332M [00:00<00:00, 416MB/s]
 59%|█████▉    | 198M/332M [00:00<00:00, 419MB/s]
 72%|███████▏  | 238M/332M [00:00<00:00, 420MB/s]
 84%|████████▎ | 278M/332M [00:00<00:00, 419MB/s]
 96%|█████████▌| 318M/332M [00:00<00:00, 418MB/s]
100%|██████████| 332M/332M [00:00<00:00, 416MB/s]
Face Enhancer::   0%|          | 0/227 [00:00<?, ?it/s]
Face Enhancer::   0%|          | 1/227 [00:00<01:08,  3.32it/s]
Face Enhancer::   1%|          | 2/227 [00:00<00:51,  4.38it/s]
Face Enhancer::   1%|▏         | 3/227 [00:00<00:44,  5.06it/s]
Face Enhancer::   2%|▏         | 4/227 [00:00<00:41,  5.41it/s]
Face Enhancer::   2%|▏         | 5/227 [00:01<00:44,  5.04it/s]
Face Enhancer::   3%|▎         | 6/227 [00:01<00:41,  5.34it/s]
Face Enhancer::   3%|▎         | 7/227 [00:01<00:39,  5.59it/s]
Face Enhancer::   4%|▎         | 8/227 [00:01<00:39,  5.58it/s]
Face Enhancer::   4%|▍         | 9/227 [00:01<00:37,  5.82it/s]
Face Enhancer::   4%|▍         | 10/227 [00:01<00:37,  5.77it/s]
Face Enhancer::   5%|▍         | 11/227 [00:02<00:37,  5.73it/s]
Face Enhancer::   5%|▌         | 12/227 [00:02<00:36,  5.90it/s]
Face Enhancer::   6%|▌         | 13/227 [00:02<00:35,  5.99it/s]
Face Enhancer::   6%|▌         | 14/227 [00:02<00:35,  5.96it/s]
Face Enhancer::   7%|▋         | 15/227 [00:02<00:34,  6.11it/s]
Face Enhancer::   7%|▋         | 16/227 [00:02<00:34,  6.17it/s]
Face Enhancer::   7%|▋         | 17/227 [00:03<00:34,  6.09it/s]
Face Enhancer::   8%|▊         | 18/227 [00:03<00:33,  6.19it/s]
Face Enhancer::   8%|▊         | 19/227 [00:03<00:33,  6.26it/s]
Face Enhancer::   9%|▉         | 20/227 [00:03<00:33,  6.10it/s]
Face Enhancer::   9%|▉         | 21/227 [00:03<00:33,  6.20it/s]
Face Enhancer::  10%|▉         | 22/227 [00:03<00:32,  6.27it/s]
Face Enhancer::  10%|█         | 23/227 [00:03<00:33,  6.16it/s]
Face Enhancer::  11%|█         | 24/227 [00:04<00:32,  6.21it/s]
Face Enhancer::  11%|█         | 25/227 [00:04<00:32,  6.25it/s]
Face Enhancer::  11%|█▏        | 26/227 [00:04<00:32,  6.11it/s]
Face Enhancer::  12%|█▏        | 27/227 [00:04<00:32,  6.18it/s]
Face Enhancer::  12%|█▏        | 28/227 [00:04<00:33,  6.01it/s]
Face Enhancer::  13%|█▎        | 29/227 [00:04<00:33,  5.96it/s]
Face Enhancer::  13%|█▎        | 30/227 [00:05<00:32,  6.09it/s]
Face Enhancer::  14%|█▎        | 31/227 [00:05<00:31,  6.18it/s]
Face Enhancer::  14%|█▍        | 32/227 [00:05<00:31,  6.10it/s]
Face Enhancer::  15%|█▍        | 33/227 [00:05<00:31,  6.19it/s]
Face Enhancer::  15%|█▍        | 34/227 [00:05<00:31,  6.07it/s]
Face Enhancer::  15%|█▌        | 35/227 [00:05<00:32,  5.82it/s]
Face Enhancer::  16%|█▌        | 36/227 [00:06<00:32,  5.95it/s]
Face Enhancer::  16%|█▋        | 37/227 [00:06<00:31,  6.09it/s]
Face Enhancer::  17%|█▋        | 38/227 [00:06<00:31,  6.02it/s]
Face Enhancer::  17%|█▋        | 39/227 [00:06<00:30,  6.11it/s]
Face Enhancer::  18%|█▊        | 40/227 [00:06<00:30,  6.04it/s]
Face Enhancer::  18%|█▊        | 41/227 [00:06<00:30,  6.01it/s]
Face Enhancer::  19%|█▊        | 42/227 [00:07<00:30,  6.10it/s]
Face Enhancer::  19%|█▉        | 43/227 [00:07<00:34,  5.38it/s]
Face Enhancer::  19%|█▉        | 44/227 [00:07<00:33,  5.49it/s]
Face Enhancer::  20%|█▉        | 45/227 [00:07<00:33,  5.46it/s]
Face Enhancer::  20%|██        | 46/227 [00:07<00:32,  5.56it/s]
Face Enhancer::  21%|██        | 47/227 [00:08<00:31,  5.66it/s]
Face Enhancer::  21%|██        | 48/227 [00:08<00:30,  5.85it/s]
Face Enhancer::  22%|██▏       | 49/227 [00:08<00:29,  6.00it/s]
Face Enhancer::  22%|██▏       | 50/227 [00:08<00:29,  5.97it/s]
Face Enhancer::  22%|██▏       | 51/227 [00:08<00:29,  5.90it/s]
Face Enhancer::  23%|██▎       | 52/227 [00:08<00:30,  5.81it/s]
Face Enhancer::  23%|██▎       | 53/227 [00:09<00:30,  5.66it/s]
Face Enhancer::  24%|██▍       | 54/227 [00:09<00:32,  5.38it/s]
Face Enhancer::  24%|██▍       | 55/227 [00:09<00:30,  5.67it/s]
Face Enhancer::  25%|██▍       | 56/227 [00:09<00:29,  5.76it/s]
Face Enhancer::  25%|██▌       | 57/227 [00:09<00:29,  5.74it/s]
Face Enhancer::  26%|██▌       | 58/227 [00:09<00:28,  5.91it/s]
Face Enhancer::  26%|██▌       | 59/227 [00:10<00:28,  5.91it/s]
Face Enhancer::  26%|██▋       | 60/227 [00:10<00:28,  5.94it/s]
Face Enhancer::  27%|██▋       | 61/227 [00:10<00:27,  6.09it/s]
Face Enhancer::  27%|██▋       | 62/227 [00:10<00:27,  6.01it/s]
Face Enhancer::  28%|██▊       | 63/227 [00:10<00:27,  5.93it/s]
Face Enhancer::  28%|██▊       | 64/227 [00:10<00:27,  5.86it/s]
Face Enhancer::  29%|██▊       | 65/227 [00:11<00:28,  5.74it/s]
Face Enhancer::  29%|██▉       | 66/227 [00:11<00:28,  5.70it/s]
Face Enhancer::  30%|██▉       | 67/227 [00:11<00:28,  5.57it/s]
Face Enhancer::  30%|██▉       | 68/227 [00:11<00:29,  5.46it/s]
Face Enhancer::  30%|███       | 69/227 [00:11<00:29,  5.32it/s]
Face Enhancer::  31%|███       | 70/227 [00:12<00:28,  5.42it/s]
Face Enhancer::  31%|███▏      | 71/227 [00:12<00:28,  5.55it/s]
Face Enhancer::  32%|███▏      | 72/227 [00:12<00:27,  5.67it/s]
Face Enhancer::  32%|███▏      | 73/227 [00:12<00:26,  5.80it/s]
Face Enhancer::  33%|███▎      | 74/227 [00:12<00:27,  5.61it/s]
Face Enhancer::  33%|███▎      | 75/227 [00:12<00:28,  5.42it/s]
Face Enhancer::  33%|███▎      | 76/227 [00:13<00:28,  5.27it/s]
Face Enhancer::  34%|███▍      | 77/227 [00:13<00:27,  5.40it/s]
Face Enhancer::  34%|███▍      | 78/227 [00:13<00:26,  5.62it/s]
Face Enhancer::  35%|███▍      | 79/227 [00:13<00:25,  5.84it/s]
Face Enhancer::  35%|███▌      | 80/227 [00:13<00:25,  5.87it/s]
Face Enhancer::  36%|███▌      | 81/227 [00:13<00:24,  6.03it/s]
Face Enhancer::  36%|███▌      | 82/227 [00:14<00:24,  6.01it/s]
Face Enhancer::  37%|███▋      | 83/227 [00:14<00:24,  5.79it/s]
Face Enhancer::  37%|███▋      | 84/227 [00:14<00:24,  5.74it/s]
Face Enhancer::  37%|███▋      | 85/227 [00:14<00:23,  5.94it/s]
Face Enhancer::  38%|███▊      | 86/227 [00:14<00:23,  5.93it/s]
Face Enhancer::  38%|███▊      | 87/227 [00:14<00:23,  6.08it/s]
Face Enhancer::  39%|███▉      | 88/227 [00:15<00:22,  6.20it/s]
Face Enhancer::  39%|███▉      | 89/227 [00:15<00:22,  6.11it/s]
Face Enhancer::  40%|███▉      | 90/227 [00:15<00:22,  6.13it/s]
Face Enhancer::  40%|████      | 91/227 [00:15<00:21,  6.22it/s]
Face Enhancer::  41%|████      | 92/227 [00:15<00:23,  5.80it/s]
Face Enhancer::  41%|████      | 93/227 [00:15<00:22,  5.99it/s]
Face Enhancer::  41%|████▏     | 94/227 [00:16<00:21,  6.13it/s]
Face Enhancer::  42%|████▏     | 95/227 [00:16<00:21,  6.00it/s]
Face Enhancer::  42%|████▏     | 96/227 [00:16<00:21,  6.13it/s]
Face Enhancer::  43%|████▎     | 97/227 [00:16<00:20,  6.23it/s]
Face Enhancer::  43%|████▎     | 98/227 [00:16<00:21,  5.94it/s]
Face Enhancer::  44%|████▎     | 99/227 [00:16<00:21,  6.04it/s]
Face Enhancer::  44%|████▍     | 100/227 [00:17<00:20,  6.13it/s]
Face Enhancer::  44%|████▍     | 101/227 [00:17<00:20,  6.07it/s]
Face Enhancer::  45%|████▍     | 102/227 [00:17<00:20,  6.04it/s]
Face Enhancer::  45%|████▌     | 103/227 [00:17<00:20,  6.17it/s]
Face Enhancer::  46%|████▌     | 104/227 [00:17<00:20,  5.91it/s]
Face Enhancer::  46%|████▋     | 105/227 [00:17<00:20,  6.07it/s]
Face Enhancer::  47%|████▋     | 106/227 [00:18<00:20,  6.04it/s]
Face Enhancer::  47%|████▋     | 107/227 [00:18<00:19,  6.01it/s]
Face Enhancer::  48%|████▊     | 108/227 [00:18<00:20,  5.91it/s]
Face Enhancer::  48%|████▊     | 109/227 [00:18<00:20,  5.85it/s]
Face Enhancer::  48%|████▊     | 110/227 [00:18<00:20,  5.70it/s]
Face Enhancer::  49%|████▉     | 111/227 [00:18<00:19,  5.92it/s]
Face Enhancer::  49%|████▉     | 112/227 [00:19<00:19,  6.05it/s]
Face Enhancer::  50%|████▉     | 113/227 [00:19<00:18,  6.01it/s]
Face Enhancer::  50%|█████     | 114/227 [00:19<00:18,  6.13it/s]
Face Enhancer::  51%|█████     | 115/227 [00:19<00:17,  6.23it/s]
Face Enhancer::  51%|█████     | 116/227 [00:19<00:18,  6.15it/s]
Face Enhancer::  52%|█████▏    | 117/227 [00:19<00:17,  6.25it/s]
Face Enhancer::  52%|█████▏    | 118/227 [00:20<00:17,  6.22it/s]
Face Enhancer::  52%|█████▏    | 119/227 [00:20<00:17,  6.13it/s]
Face Enhancer::  53%|█████▎    | 120/227 [00:20<00:17,  6.20it/s]
Face Enhancer::  53%|█████▎    | 121/227 [00:20<00:16,  6.27it/s]
Face Enhancer::  54%|█████▎    | 122/227 [00:20<00:17,  6.15it/s]
Face Enhancer::  54%|█████▍    | 123/227 [00:20<00:16,  6.24it/s]
Face Enhancer::  55%|█████▍    | 124/227 [00:21<00:16,  6.26it/s]
Face Enhancer::  55%|█████▌    | 125/227 [00:21<00:16,  6.15it/s]
Face Enhancer::  56%|█████▌    | 126/227 [00:21<00:16,  6.23it/s]
Face Enhancer::  56%|█████▌    | 127/227 [00:21<00:15,  6.29it/s]
Face Enhancer::  56%|█████▋    | 128/227 [00:21<00:16,  6.16it/s]
Face Enhancer::  57%|█████▋    | 129/227 [00:21<00:15,  6.24it/s]
Face Enhancer::  57%|█████▋    | 130/227 [00:22<00:15,  6.25it/s]
Face Enhancer::  58%|█████▊    | 131/227 [00:22<00:15,  6.09it/s]
Face Enhancer::  58%|█████▊    | 132/227 [00:22<00:15,  6.20it/s]
Face Enhancer::  59%|█████▊    | 133/227 [00:22<00:14,  6.28it/s]
Face Enhancer::  59%|█████▉    | 134/227 [00:22<00:15,  6.18it/s]
Face Enhancer::  59%|█████▉    | 135/227 [00:22<00:14,  6.26it/s]
Face Enhancer::  60%|█████▉    | 136/227 [00:22<00:14,  6.29it/s]
Face Enhancer::  60%|██████    | 137/227 [00:23<00:14,  6.16it/s]
Face Enhancer::  61%|██████    | 138/227 [00:23<00:14,  6.08it/s]
Face Enhancer::  61%|██████    | 139/227 [00:23<00:14,  6.17it/s]
Face Enhancer::  62%|██████▏   | 140/227 [00:23<00:14,  6.08it/s]
Face Enhancer::  62%|██████▏   | 141/227 [00:23<00:14,  5.98it/s]
Face Enhancer::  63%|██████▎   | 142/227 [00:23<00:13,  6.11it/s]
Face Enhancer::  63%|██████▎   | 143/227 [00:24<00:14,  5.84it/s]
Face Enhancer::  63%|██████▎   | 144/227 [00:24<00:13,  5.99it/s]
Face Enhancer::  64%|██████▍   | 145/227 [00:24<00:13,  6.10it/s]
Face Enhancer::  64%|██████▍   | 146/227 [00:24<00:13,  6.03it/s]
Face Enhancer::  65%|██████▍   | 147/227 [00:24<00:13,  5.94it/s]
Face Enhancer::  65%|██████▌   | 148/227 [00:24<00:13,  6.06it/s]
Face Enhancer::  66%|██████▌   | 149/227 [00:25<00:13,  5.99it/s]
Face Enhancer::  66%|██████▌   | 150/227 [00:25<00:12,  6.10it/s]
Face Enhancer::  67%|██████▋   | 151/227 [00:25<00:13,  5.79it/s]
Face Enhancer::  67%|██████▋   | 152/227 [00:25<00:12,  5.81it/s]
Face Enhancer::  67%|██████▋   | 153/227 [00:25<00:12,  5.98it/s]
Face Enhancer::  68%|██████▊   | 154/227 [00:25<00:12,  6.06it/s]
Face Enhancer::  68%|██████▊   | 155/227 [00:26<00:11,  6.00it/s]
Face Enhancer::  69%|██████▊   | 156/227 [00:26<00:11,  6.11it/s]
Face Enhancer::  69%|██████▉   | 157/227 [00:26<00:11,  6.21it/s]
Face Enhancer::  70%|██████▉   | 158/227 [00:26<00:11,  6.12it/s]
Face Enhancer::  70%|███████   | 159/227 [00:26<00:11,  6.15it/s]
Face Enhancer::  70%|███████   | 160/227 [00:26<00:10,  6.23it/s]
Face Enhancer::  71%|███████   | 161/227 [00:27<00:10,  6.03it/s]
Face Enhancer::  71%|███████▏  | 162/227 [00:27<00:10,  6.12it/s]
Face Enhancer::  72%|███████▏  | 163/227 [00:27<00:10,  6.09it/s]
Face Enhancer::  72%|███████▏  | 164/227 [00:27<00:10,  5.95it/s]
Face Enhancer::  73%|███████▎  | 165/227 [00:27<00:10,  5.82it/s]
Face Enhancer::  73%|███████▎  | 166/227 [00:28<00:10,  5.61it/s]
Face Enhancer::  74%|███████▎  | 167/227 [00:28<00:11,  5.25it/s]
Face Enhancer::  74%|███████▍  | 168/227 [00:28<00:10,  5.52it/s]
Face Enhancer::  74%|███████▍  | 169/227 [00:28<00:10,  5.76it/s]
Face Enhancer::  75%|███████▍  | 170/227 [00:28<00:09,  5.81it/s]
Face Enhancer::  75%|███████▌  | 171/227 [00:28<00:09,  6.00it/s]
Face Enhancer::  76%|███████▌  | 172/227 [00:29<00:09,  6.08it/s]
Face Enhancer::  76%|███████▌  | 173/227 [00:29<00:08,  6.02it/s]
Face Enhancer::  77%|███████▋  | 174/227 [00:29<00:08,  6.12it/s]
Face Enhancer::  77%|███████▋  | 175/227 [00:29<00:08,  6.21it/s]
Face Enhancer::  78%|███████▊  | 176/227 [00:29<00:08,  6.12it/s]
Face Enhancer::  78%|███████▊  | 177/227 [00:29<00:08,  6.07it/s]
Face Enhancer::  78%|███████▊  | 178/227 [00:30<00:08,  6.03it/s]
Face Enhancer::  79%|███████▉  | 179/227 [00:30<00:08,  5.79it/s]
Face Enhancer::  79%|███████▉  | 180/227 [00:30<00:07,  5.92it/s]
Face Enhancer::  80%|███████▉  | 181/227 [00:30<00:07,  6.05it/s]
Face Enhancer::  80%|████████  | 182/227 [00:30<00:07,  5.92it/s]
Face Enhancer::  81%|████████  | 183/227 [00:30<00:07,  6.07it/s]
Face Enhancer::  81%|████████  | 184/227 [00:31<00:06,  6.18it/s]
Face Enhancer::  81%|████████▏ | 185/227 [00:31<00:06,  6.10it/s]
Face Enhancer::  82%|████████▏ | 186/227 [00:31<00:06,  6.16it/s]
Face Enhancer::  82%|████████▏ | 187/227 [00:31<00:06,  6.26it/s]
Face Enhancer::  83%|████████▎ | 188/227 [00:31<00:06,  6.16it/s]
Face Enhancer::  83%|████████▎ | 189/227 [00:31<00:06,  6.26it/s]
Face Enhancer::  84%|████████▎ | 190/227 [00:31<00:05,  6.32it/s]
Face Enhancer::  84%|████████▍ | 191/227 [00:32<00:05,  6.17it/s]
Face Enhancer::  85%|████████▍ | 192/227 [00:32<00:05,  6.24it/s]
Face Enhancer::  85%|████████▌ | 193/227 [00:32<00:05,  6.15it/s]
Face Enhancer::  85%|████████▌ | 194/227 [00:32<00:05,  6.08it/s]
Face Enhancer::  86%|████████▌ | 195/227 [00:32<00:05,  6.12it/s]
Face Enhancer::  86%|████████▋ | 196/227 [00:32<00:05,  5.92it/s]
Face Enhancer::  87%|████████▋ | 197/227 [00:33<00:05,  5.86it/s]
Face Enhancer::  87%|████████▋ | 198/227 [00:33<00:04,  5.94it/s]
Face Enhancer::  88%|████████▊ | 199/227 [00:33<00:04,  6.01it/s]
Face Enhancer::  88%|████████▊ | 200/227 [00:33<00:04,  5.95it/s]
Face Enhancer::  89%|████████▊ | 201/227 [00:33<00:04,  5.73it/s]
Face Enhancer::  89%|████████▉ | 202/227 [00:34<00:04,  5.71it/s]
Face Enhancer::  89%|████████▉ | 203/227 [00:34<00:04,  5.35it/s]
Face Enhancer::  90%|████████▉ | 204/227 [00:34<00:04,  5.45it/s]
Face Enhancer::  90%|█████████ | 205/227 [00:34<00:03,  5.68it/s]
Face Enhancer::  91%|█████████ | 206/227 [00:34<00:03,  5.58it/s]
Face Enhancer::  91%|█████████ | 207/227 [00:34<00:03,  5.35it/s]
Face Enhancer::  92%|█████████▏| 208/227 [00:35<00:03,  5.46it/s]
Face Enhancer::  92%|█████████▏| 209/227 [00:35<00:03,  5.37it/s]
Face Enhancer::  93%|█████████▎| 210/227 [00:35<00:03,  5.64it/s]
Face Enhancer::  93%|█████████▎| 211/227 [00:35<00:02,  5.66it/s]
Face Enhancer::  93%|█████████▎| 212/227 [00:35<00:02,  5.39it/s]
Face Enhancer::  94%|█████████▍| 213/227 [00:36<00:02,  5.64it/s]
Face Enhancer::  94%|█████████▍| 214/227 [00:36<00:02,  5.84it/s]
Face Enhancer::  95%|█████████▍| 215/227 [00:36<00:02,  5.81it/s]
Face Enhancer::  95%|█████████▌| 216/227 [00:36<00:01,  5.86it/s]
Face Enhancer::  96%|█████████▌| 217/227 [00:36<00:01,  5.98it/s]
Face Enhancer::  96%|█████████▌| 218/227 [00:36<00:01,  5.78it/s]
Face Enhancer::  96%|█████████▋| 219/227 [00:37<00:01,  5.66it/s]
Face Enhancer::  97%|█████████▋| 220/227 [00:37<00:01,  5.70it/s]
Face Enhancer::  97%|█████████▋| 221/227 [00:37<00:01,  5.57it/s]
Face Enhancer::  98%|█████████▊| 222/227 [00:37<00:00,  5.20it/s]
Face Enhancer::  98%|█████████▊| 223/227 [00:37<00:00,  5.51it/s]
Face Enhancer::  99%|█████████▊| 224/227 [00:37<00:00,  5.55it/s]
Face Enhancer::  99%|█████████▉| 225/227 [00:38<00:00,  5.66it/s]
Face Enhancer:: 100%|█████████▉| 226/227 [00:38<00:00,  5.85it/s]
Face Enhancer:: 100%|██████████| 227/227 [00:38<00:00,  5.77it/s]
Face Enhancer:: 100%|██████████| 227/227 [00:38<00:00,  5.90it/s]
The generated video is named exp_dir/tmpc3kuukfvart_1##tmpvz4viuhbjapanese_enhanced.mp4
The generated video is named tmpc3kuukfvart_1##tmpvz4viuhbjapanese in exp_dir

Examples

View more examples

Run time and cost

This model costs approximately $0.18 to run on Replicate, or 5 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 130 seconds. The predict time for this model varies significantly based on the inputs.

Readme

original repo: https://github.com/OpenTalker/SadTalker

CVPR 2023

sadtalker

TL;DR: single portrait image 🙎‍♂️ + audio 🎤 = talking head video 🎞.

</div>

🛎 Citation

If you find our work useful in your research, please consider citing:

@article{zhang2022sadtalker,
  title={SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation},
  author={Zhang, Wenxuan and Cun, Xiaodong and Wang, Xuan and Zhang, Yong and Shen, Xi and Guo, Yu and Shan, Ying and Wang, Fei},
  journal={arXiv preprint arXiv:2211.12194},
  year={2022}
}

💗 Acknowledgements

Facerender code borrows heavily from zhanglonghao’s reproduction of face-vid2vid and PIRender. We thank the authors for sharing their wonderful code. In training process, We also use the model from Deep3DFaceReconstruction and Wav2lip. We thank for their wonderful work.

See also these wonderful 3rd libraries we use:

Face Utils: https://github.com/xinntao/facexlib
Face Enhancement: https://github.com/TencentARC/GFPGAN
Image/Video Enhancement:https://github.com/xinntao/Real-ESRGAN

🥂 Extensions:

SadTalker-Video-Lip-Sync from @Zz-ww: SadTalker for Video Lip Editing

📢 Disclaimer

This is not an official product of Tencent. This repository can only be used for personal/research/non-commercial purposes.

LOGO: color and font suggestion: ChatGPT, logo font：Montserrat Alternates .

All the copyright of the demo images and audio are from communities users or the geneartion from stable diffusion. Free free to contact us if you feel uncomfortable.

Input

Output

Examples

Run time and cost

Readme

🛎 Citation

💗 Acknowledgements

🥂 Extensions:

🥂 Related Works

📢 Disclaimer