zsxkib / stable-video-face-restoration

SVFR: A Unified Framework for Generalized Video Face Restoration

Cold

Public
467 runs
L40S
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

video

*file

Preview

Input video file (e.g. MP4).

tasks

string

Which restoration tasks to apply.

Default: "face-restoration"

mask

file

An inpainting mask image (white areas will be restored). Only required when tasks includes inpainting.

num_inference_steps

integer

Number of diffusion steps.

Default: 30

decode_chunk_size

integer

Chunk size for decoding long videos.

Default: 16

overlap

integer

Number of overlapping frames between segments.

Default: 3

noise_aug_strength

number

Noise augmentation strength.

Default: 0

min_appearance_guidance_scale

number

Minimum guidance scale for restoration.

Default: 2

max_appearance_guidance_scale

number

Maximum guidance scale for restoration.

Default: 2

i2i_noise_strength

number

Image-to-image noise strength.

Default: 1

seed

integer

Random seed. Leave blank to randomize.

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/stable-video-face-restoration

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/stable-video-face-restoration using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/stable-video-face-restoration:63512c77555a80ca5c84c590641036ba9f938d38b9a1841ea369780072561373",
  {
    input: {
      seed: -1,
      tasks: "face-restoration-and-colorization",
      video: "https://replicate.delivery/pbxt/MIbOfHGBUvbjrd6qxFBb1ujR4FsyJfspsrN6EFbZfOM6CtZp/lq3.mp4",
      overlap: 3,
      decode_chunk_size: 16,
      i2i_noise_strength: 1,
      noise_aug_strength: 0,
      num_inference_steps: 30,
      max_appearance_guidance_scale: 2,
      min_appearance_guidance_scale: 2
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/stable-video-face-restoration using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/stable-video-face-restoration:63512c77555a80ca5c84c590641036ba9f938d38b9a1841ea369780072561373",
    input={
        "seed": -1,
        "tasks": "face-restoration-and-colorization",
        "video": "https://replicate.delivery/pbxt/MIbOfHGBUvbjrd6qxFBb1ujR4FsyJfspsrN6EFbZfOM6CtZp/lq3.mp4",
        "overlap": 3,
        "decode_chunk_size": 16,
        "i2i_noise_strength": 1,
        "noise_aug_strength": 0,
        "num_inference_steps": 30,
        "max_appearance_guidance_scale": 2,
        "min_appearance_guidance_scale": 2
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/stable-video-face-restoration using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zsxkib/stable-video-face-restoration:63512c77555a80ca5c84c590641036ba9f938d38b9a1841ea369780072561373",
    "input": {
      "seed": -1,
      "tasks": "face-restoration-and-colorization",
      "video": "https://replicate.delivery/pbxt/MIbOfHGBUvbjrd6qxFBb1ujR4FsyJfspsrN6EFbZfOM6CtZp/lq3.mp4",
      "overlap": 3,
      "decode_chunk_size": 16,
      "i2i_noise_strength": 1,
      "noise_aug_strength": 0,
      "num_inference_steps": 30,
      "max_appearance_guidance_scale": 2,
      "min_appearance_guidance_scale": 2
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2025-01-13T16:45:12.428374Z",
  "created_at": "2025-01-13T16:39:41.729000Z",
  "data_removed": false,
  "error": null,
  "id": "zmbbax8bm5rm80cmc24tptr4x0",
  "input": {
    "seed": -1,
    "tasks": "face-restoration-and-colorization",
    "video": "https://replicate.delivery/pbxt/MIbOfHGBUvbjrd6qxFBb1ujR4FsyJfspsrN6EFbZfOM6CtZp/lq3.mp4"
  },
  "logs": "Using seed: 32680\nCUDA_VISIBLE_DEVICES is not set.\ntask_ids: [0, 1]\ntask_prompt: [1, 1, 0]\ntotal_frames: 54\nGenerating the first clip...\n  0%|          | 0/50 [00:00<?, ?it/s]\n  2%|▏         | 1/50 [00:00<00:39,  1.24it/s]\n  4%|▍         | 2/50 [00:01<00:44,  1.09it/s]\n  6%|▌         | 3/50 [00:02<00:45,  1.04it/s]\n  8%|▊         | 4/50 [00:03<00:35,  1.28it/s]\n 10%|█         | 5/50 [00:03<00:30,  1.47it/s]\n 12%|█▏        | 6/50 [00:04<00:27,  1.62it/s]\n 14%|█▍        | 7/50 [00:04<00:24,  1.72it/s]\n 16%|█▌        | 8/50 [00:05<00:23,  1.80it/s]\n 18%|█▊        | 9/50 [00:05<00:22,  1.85it/s]\n 20%|██        | 10/50 [00:06<00:21,  1.89it/s]\n 22%|██▏       | 11/50 [00:06<00:20,  1.92it/s]\n 24%|██▍       | 12/50 [00:07<00:19,  1.94it/s]\n 26%|██▌       | 13/50 [00:07<00:18,  1.95it/s]\n 28%|██▊       | 14/50 [00:08<00:18,  1.96it/s]\n 30%|███       | 15/50 [00:08<00:17,  1.97it/s]\n 32%|███▏      | 16/50 [00:09<00:17,  1.97it/s]\n 34%|███▍      | 17/50 [00:09<00:16,  1.98it/s]\n 36%|███▌      | 18/50 [00:10<00:16,  1.98it/s]\n 38%|███▊      | 19/50 [00:10<00:15,  1.98it/s]\n 40%|████      | 20/50 [00:11<00:15,  1.98it/s]\n 42%|████▏     | 21/50 [00:11<00:14,  1.98it/s]\n 44%|████▍     | 22/50 [00:12<00:14,  1.98it/s]\n 46%|████▌     | 23/50 [00:12<00:13,  1.98it/s]\n 48%|████▊     | 24/50 [00:13<00:13,  1.98it/s]\n 50%|█████     | 25/50 [00:13<00:12,  1.98it/s]\n 52%|█████▏    | 26/50 [00:14<00:12,  1.98it/s]\n 54%|█████▍    | 27/50 [00:14<00:11,  1.98it/s]\n 56%|█████▌    | 28/50 [00:15<00:11,  1.98it/s]\n 58%|█████▊    | 29/50 [00:15<00:10,  1.98it/s]\n 60%|██████    | 30/50 [00:16<00:10,  1.98it/s]\n 62%|██████▏   | 31/50 [00:16<00:09,  1.98it/s]\n 64%|██████▍   | 32/50 [00:17<00:09,  1.98it/s]\n 66%|██████▌   | 33/50 [00:17<00:08,  1.97it/s]\n 68%|██████▊   | 34/50 [00:18<00:08,  1.97it/s]\n 70%|███████   | 35/50 [00:18<00:07,  1.97it/s]\n 72%|███████▏  | 36/50 [00:19<00:07,  1.97it/s]\n 74%|███████▍  | 37/50 [00:19<00:06,  1.97it/s]\n 76%|███████▌  | 38/50 [00:20<00:06,  1.97it/s]\n 78%|███████▊  | 39/50 [00:20<00:05,  1.97it/s]\n 80%|████████  | 40/50 [00:21<00:05,  1.97it/s]\n 82%|████████▏ | 41/50 [00:22<00:04,  1.97it/s]\n 84%|████████▍ | 42/50 [00:22<00:04,  1.97it/s]\n 86%|████████▌ | 43/50 [00:23<00:03,  1.97it/s]\n 88%|████████▊ | 44/50 [00:23<00:03,  1.97it/s]\n 90%|█████████ | 45/50 [00:24<00:02,  1.97it/s]\n 92%|█████████▏| 46/50 [00:24<00:02,  1.97it/s]\n 94%|█████████▍| 47/50 [00:25<00:01,  1.97it/s]\n 96%|█████████▌| 48/50 [00:25<00:01,  1.97it/s]\n 98%|█████████▊| 49/50 [00:26<00:00,  1.96it/s]\n100%|██████████| 50/50 [00:26<00:00,  1.96it/s]\n100%|██████████| 50/50 [00:26<00:00,  1.88it/s]\nGenerating all video clips...\n  0%|          | 0/30 [00:00<?, ?it/s]\n  3%|▎         | 1/30 [00:02<01:07,  2.31s/it]\n  7%|▋         | 2/30 [00:04<01:09,  2.47s/it]\n 10%|█         | 3/30 [00:07<01:07,  2.52s/it]\n 13%|█▎        | 4/30 [00:09<01:00,  2.34s/it]\n 17%|█▋        | 5/30 [00:11<00:56,  2.24s/it]\n 20%|██        | 6/30 [00:13<00:52,  2.19s/it]\n 23%|██▎       | 7/30 [00:15<00:49,  2.15s/it]\n 27%|██▋       | 8/30 [00:17<00:46,  2.13s/it]\n 30%|███       | 9/30 [00:19<00:44,  2.11s/it]\n 33%|███▎      | 10/30 [00:21<00:41,  2.10s/it]\n 37%|███▋      | 11/30 [00:24<00:39,  2.09s/it]\n 40%|████      | 12/30 [00:26<00:37,  2.08s/it]\n 43%|████▎     | 13/30 [00:28<00:35,  2.08s/it]\n 47%|████▋     | 14/30 [00:30<00:33,  2.07s/it]\n 50%|█████     | 15/30 [00:32<00:31,  2.07s/it]\n 53%|█████▎    | 16/30 [00:34<00:29,  2.07s/it]\n 57%|█████▋    | 17/30 [00:36<00:26,  2.07s/it]\n 60%|██████    | 18/30 [00:38<00:24,  2.07s/it]\n 63%|██████▎   | 19/30 [00:40<00:22,  2.07s/it]\n 67%|██████▋   | 20/30 [00:42<00:20,  2.07s/it]\n 70%|███████   | 21/30 [00:44<00:18,  2.07s/it]\n 73%|███████▎  | 22/30 [00:46<00:16,  2.07s/it]\n 77%|███████▋  | 23/30 [00:48<00:14,  2.07s/it]\n 80%|████████  | 24/30 [00:50<00:12,  2.07s/it]\n 83%|████████▎ | 25/30 [00:53<00:10,  2.08s/it]\n 87%|████████▋ | 26/30 [00:55<00:08,  2.08s/it]\n 90%|█████████ | 27/30 [00:57<00:06,  2.08s/it]\n 93%|█████████▎| 28/30 [00:59<00:04,  2.08s/it]\n 97%|█████████▋| 29/30 [01:01<00:02,  2.08s/it]\n100%|██████████| 30/30 [01:03<00:00,  2.08s/it]\n100%|██████████| 30/30 [01:03<00:00,  2.12s/it]",
  "metrics": {
    "predict_time": 107.24592646,
    "total_time": 330.699374
  },
  "output": "https://replicate.delivery/xezq/8mxHD4QdNWItP5JVlJfr0bbBeDnbXi2ZRhDpCsoxATDYUvEUA/tmpo46rt9aflq3_32680.mp4",
  "started_at": "2025-01-13T16:43:25.182448Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://stream.replicate.com/v1/files/bcwr-7bipm3aa6ataeu43qxzskgwslmqaieyxtf77bzhysuvahzcwhibq",
    "get": "https://api.replicate.com/v1/predictions/zmbbax8bm5rm80cmc24tptr4x0",
    "cancel": "https://api.replicate.com/v1/predictions/zmbbax8bm5rm80cmc24tptr4x0/cancel"
  },
  "version": "773f313ce0ee71a7d2103adad6ce1e53ac0e66d26e20a90d85bfe5bad73b0740"
}

Generated in

1 minute 47 seconds

Tweak it ShareReport

Using seed: 32680
CUDA_VISIBLE_DEVICES is not set.
task_ids: [0, 1]
task_prompt: [1, 1, 0]
total_frames: 54
Generating the first clip...
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:00<00:39,  1.24it/s]
  4%|▍         | 2/50 [00:01<00:44,  1.09it/s]
  6%|▌         | 3/50 [00:02<00:45,  1.04it/s]
  8%|▊         | 4/50 [00:03<00:35,  1.28it/s]
 10%|█         | 5/50 [00:03<00:30,  1.47it/s]
 12%|█▏        | 6/50 [00:04<00:27,  1.62it/s]
 14%|█▍        | 7/50 [00:04<00:24,  1.72it/s]
 16%|█▌        | 8/50 [00:05<00:23,  1.80it/s]
 18%|█▊        | 9/50 [00:05<00:22,  1.85it/s]
 20%|██        | 10/50 [00:06<00:21,  1.89it/s]
 22%|██▏       | 11/50 [00:06<00:20,  1.92it/s]
 24%|██▍       | 12/50 [00:07<00:19,  1.94it/s]
 26%|██▌       | 13/50 [00:07<00:18,  1.95it/s]
 28%|██▊       | 14/50 [00:08<00:18,  1.96it/s]
 30%|███       | 15/50 [00:08<00:17,  1.97it/s]
 32%|███▏      | 16/50 [00:09<00:17,  1.97it/s]
 34%|███▍      | 17/50 [00:09<00:16,  1.98it/s]
 36%|███▌      | 18/50 [00:10<00:16,  1.98it/s]
 38%|███▊      | 19/50 [00:10<00:15,  1.98it/s]
 40%|████      | 20/50 [00:11<00:15,  1.98it/s]
 42%|████▏     | 21/50 [00:11<00:14,  1.98it/s]
 44%|████▍     | 22/50 [00:12<00:14,  1.98it/s]
 46%|████▌     | 23/50 [00:12<00:13,  1.98it/s]
 48%|████▊     | 24/50 [00:13<00:13,  1.98it/s]
 50%|█████     | 25/50 [00:13<00:12,  1.98it/s]
 52%|█████▏    | 26/50 [00:14<00:12,  1.98it/s]
 54%|█████▍    | 27/50 [00:14<00:11,  1.98it/s]
 56%|█████▌    | 28/50 [00:15<00:11,  1.98it/s]
 58%|█████▊    | 29/50 [00:15<00:10,  1.98it/s]
 60%|██████    | 30/50 [00:16<00:10,  1.98it/s]
 62%|██████▏   | 31/50 [00:16<00:09,  1.98it/s]
 64%|██████▍   | 32/50 [00:17<00:09,  1.98it/s]
 66%|██████▌   | 33/50 [00:17<00:08,  1.97it/s]
 68%|██████▊   | 34/50 [00:18<00:08,  1.97it/s]
 70%|███████   | 35/50 [00:18<00:07,  1.97it/s]
 72%|███████▏  | 36/50 [00:19<00:07,  1.97it/s]
 74%|███████▍  | 37/50 [00:19<00:06,  1.97it/s]
 76%|███████▌  | 38/50 [00:20<00:06,  1.97it/s]
 78%|███████▊  | 39/50 [00:20<00:05,  1.97it/s]
 80%|████████  | 40/50 [00:21<00:05,  1.97it/s]
 82%|████████▏ | 41/50 [00:22<00:04,  1.97it/s]
 84%|████████▍ | 42/50 [00:22<00:04,  1.97it/s]
 86%|████████▌ | 43/50 [00:23<00:03,  1.97it/s]
 88%|████████▊ | 44/50 [00:23<00:03,  1.97it/s]
 90%|█████████ | 45/50 [00:24<00:02,  1.97it/s]
 92%|█████████▏| 46/50 [00:24<00:02,  1.97it/s]
 94%|█████████▍| 47/50 [00:25<00:01,  1.97it/s]
 96%|█████████▌| 48/50 [00:25<00:01,  1.97it/s]
 98%|█████████▊| 49/50 [00:26<00:00,  1.96it/s]
100%|██████████| 50/50 [00:26<00:00,  1.96it/s]
100%|██████████| 50/50 [00:26<00:00,  1.88it/s]
Generating all video clips...
  0%|          | 0/30 [00:00<?, ?it/s]
  3%|▎         | 1/30 [00:02<01:07,  2.31s/it]
  7%|▋         | 2/30 [00:04<01:09,  2.47s/it]
 10%|█         | 3/30 [00:07<01:07,  2.52s/it]
 13%|█▎        | 4/30 [00:09<01:00,  2.34s/it]
 17%|█▋        | 5/30 [00:11<00:56,  2.24s/it]
 20%|██        | 6/30 [00:13<00:52,  2.19s/it]
 23%|██▎       | 7/30 [00:15<00:49,  2.15s/it]
 27%|██▋       | 8/30 [00:17<00:46,  2.13s/it]
 30%|███       | 9/30 [00:19<00:44,  2.11s/it]
 33%|███▎      | 10/30 [00:21<00:41,  2.10s/it]
 37%|███▋      | 11/30 [00:24<00:39,  2.09s/it]
 40%|████      | 12/30 [00:26<00:37,  2.08s/it]
 43%|████▎     | 13/30 [00:28<00:35,  2.08s/it]
 47%|████▋     | 14/30 [00:30<00:33,  2.07s/it]
 50%|█████     | 15/30 [00:32<00:31,  2.07s/it]
 53%|█████▎    | 16/30 [00:34<00:29,  2.07s/it]
 57%|█████▋    | 17/30 [00:36<00:26,  2.07s/it]
 60%|██████    | 18/30 [00:38<00:24,  2.07s/it]
 63%|██████▎   | 19/30 [00:40<00:22,  2.07s/it]
 67%|██████▋   | 20/30 [00:42<00:20,  2.07s/it]
 70%|███████   | 21/30 [00:44<00:18,  2.07s/it]
 73%|███████▎  | 22/30 [00:46<00:16,  2.07s/it]
 77%|███████▋  | 23/30 [00:48<00:14,  2.07s/it]
 80%|████████  | 24/30 [00:50<00:12,  2.07s/it]
 83%|████████▎ | 25/30 [00:53<00:10,  2.08s/it]
 87%|████████▋ | 26/30 [00:55<00:08,  2.08s/it]
 90%|█████████ | 27/30 [00:57<00:06,  2.08s/it]
 93%|█████████▎| 28/30 [00:59<00:04,  2.08s/it]
 97%|█████████▋| 29/30 [01:01<00:02,  2.08s/it]
100%|██████████| 30/30 [01:03<00:00,  2.08s/it]
100%|██████████| 30/30 [01:03<00:00,  2.12s/it]

This output was created using a different version of the model, zsxkib/stable-video-face-restoration:773f313c.

Examples

View more examples

Run time and cost

This model runs on Nvidia L40S GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

SVFR - Video Face Restoration

SVFR (Stable Video Face Restoration) brings old and degraded videos back to life by restoring faces with remarkable detail. This model offers three powerful restoration options:

Face restoration - Enhances facial details and sharpness
Face restoration + colorization - Brings both enhanced details and vibrant colors
Full restoration pipeline - Complete enhancement including masked inpainting

Inputs Required

A video file (MP4 or other common formats)
Choice of restoration type
Optional mask image for inpainting (required only for full pipeline)
Random seed for reproducible results

Notes

The model works best with clear, front-facing facial footage
For inpainting, provide a black & white mask where white areas indicate regions to restore
Processing time depends on video length and chosen restoration options

License

Available for non-commercial research purposes only. See SVFR paper for details.

BibTex

@misc{wang2025svfrunifiedframeworkgeneralized,
      title={SVFR: A Unified Framework for Generalized Video Face Restoration}, 
      author={Zhiyao Wang and Xu Chen and Chengming Xu and Junwei Zhu and Xiaobin Hu and Jiangning Zhang and Chengjie Wang and Yuqi Liu and Yiyi Zhou and Rongrong Ji},
      year={2025},
      eprint={2501.01235},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2501.01235}, 
}

Follow me on Twitter/X