jd7h/propainter:b3e1ec85 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

jd7h /propainter:b3e1ec85

Playground API

Input

video

*file

Preview

Input video

mask

*file

Preview

Mask

return_input_video

boolean

Return the input video in the output.

Default: false

resize_ratio

number

Resize scale for processing video.

Default: 1

height

integer

Height of the processing video.

Default: -1

width

integer

Width of the processing video.

Default: -1

mask_dilation

integer

Mask dilation for video and flow masking.

Default: 4

ref_stride

integer

Stride of global reference frames.

Default: 10

neighbor_length

integer

Length of local neighboring frames.

Default: 10

subvideo_length

integer

Length of sub-video for long video inference.

Default: 80

raft_iter

integer

Iterations for RAFT inference.

Default: 20

mode

string

Modes: video_inpainting / video_outpainting

Default: "video_inpainting"

scale_h

number

Outpainting scale of height for video_outpainting mode.

Default: 1

scale_w

number

Outpainting scale of width for video_outpainting mode.

Default: 1.2

save_fps

integer

Frames per second.

Default: 24

fp16

boolean

Use fp16 (half precision) during inference. Default: fp32 (single precision).

Default: false

Run this model in Node.js with one line of code:

npx create-replicate --model=jd7h/propainter

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run jd7h/propainter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "jd7h/propainter:b3e1ec853a25dbbd5178128139eb773612924c7f032c9919c20c254e6813a5ad",
  {
    input: {
      fp16: true,
      mask: "https://replicate.delivery/pbxt/JgGgeIePTonFUOOZsJReOpIWuhDz2BSVL6v5z2vMsESQRjUI/watermark-mask.png",
      mode: "video_inpainting",
      video: "https://replicate.delivery/pbxt/JgGgeACVH84IkobKLPrqDvYgO4SH2f049BPdBHqfCI9RlOhf/running_car_watermark.mp4",
      width: -1,
      height: -1,
      scale_h: 1,
      scale_w: 1,
      save_fps: 24,
      raft_iter: 20,
      ref_stride: 10,
      resize_ratio: 1,
      mask_dilation: 4,
      neighbor_length: 10,
      subvideo_length: 80,
      return_input_video: true
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run jd7h/propainter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "jd7h/propainter:b3e1ec853a25dbbd5178128139eb773612924c7f032c9919c20c254e6813a5ad",
    input={
        "fp16": True,
        "mask": "https://replicate.delivery/pbxt/JgGgeIePTonFUOOZsJReOpIWuhDz2BSVL6v5z2vMsESQRjUI/watermark-mask.png",
        "mode": "video_inpainting",
        "video": "https://replicate.delivery/pbxt/JgGgeACVH84IkobKLPrqDvYgO4SH2f049BPdBHqfCI9RlOhf/running_car_watermark.mp4",
        "width": -1,
        "height": -1,
        "scale_h": 1,
        "scale_w": 1,
        "save_fps": 24,
        "raft_iter": 20,
        "ref_stride": 10,
        "resize_ratio": 1,
        "mask_dilation": 4,
        "neighbor_length": 10,
        "subvideo_length": 80,
        "return_input_video": True
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run jd7h/propainter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "jd7h/propainter:b3e1ec853a25dbbd5178128139eb773612924c7f032c9919c20c254e6813a5ad",
    "input": {
      "fp16": true,
      "mask": "https://replicate.delivery/pbxt/JgGgeIePTonFUOOZsJReOpIWuhDz2BSVL6v5z2vMsESQRjUI/watermark-mask.png",
      "mode": "video_inpainting",
      "video": "https://replicate.delivery/pbxt/JgGgeACVH84IkobKLPrqDvYgO4SH2f049BPdBHqfCI9RlOhf/running_car_watermark.mp4",
      "width": -1,
      "height": -1,
      "scale_h": 1,
      "scale_w": 1,
      "save_fps": 24,
      "raft_iter": 20,
      "ref_stride": 10,
      "resize_ratio": 1,
      "mask_dilation": 4,
      "neighbor_length": 10,
      "subvideo_length": 80,
      "return_input_video": true
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/jd7h/propainter@sha256:b3e1ec853a25dbbd5178128139eb773612924c7f032c9919c20c254e6813a5ad \
  -i 'fp16=true' \
  -i 'mask="https://replicate.delivery/pbxt/JgGgeIePTonFUOOZsJReOpIWuhDz2BSVL6v5z2vMsESQRjUI/watermark-mask.png"' \
  -i 'mode="video_inpainting"' \
  -i 'video="https://replicate.delivery/pbxt/JgGgeACVH84IkobKLPrqDvYgO4SH2f049BPdBHqfCI9RlOhf/running_car_watermark.mp4"' \
  -i 'width=-1' \
  -i 'height=-1' \
  -i 'scale_h=1' \
  -i 'scale_w=1' \
  -i 'save_fps=24' \
  -i 'raft_iter=20' \
  -i 'ref_stride=10' \
  -i 'resize_ratio=1' \
  -i 'mask_dilation=4' \
  -i 'neighbor_length=10' \
  -i 'subvideo_length=80' \
  -i 'return_input_video=true'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/jd7h/propainter@sha256:b3e1ec853a25dbbd5178128139eb773612924c7f032c9919c20c254e6813a5ad
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "fp16": true,
      "mask": "https://replicate.delivery/pbxt/JgGgeIePTonFUOOZsJReOpIWuhDz2BSVL6v5z2vMsESQRjUI/watermark-mask.png",
      "mode": "video_inpainting",
      "video": "https://replicate.delivery/pbxt/JgGgeACVH84IkobKLPrqDvYgO4SH2f049BPdBHqfCI9RlOhf/running_car_watermark.mp4",
      "width": -1,
      "height": -1,
      "scale_h": 1,
      "scale_w": 1,
      "save_fps": 24,
      "raft_iter": 20,
      "ref_stride": 10,
      "resize_ratio": 1,
      "mask_dilation": 4,
      "neighbor_length": 10,
      "subvideo_length": 80,
      "return_input_video": true
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2023-10-11T12:10:18.182608Z",
  "created_at": "2023-10-11T12:06:40.537902Z",
  "data_removed": false,
  "error": null,
  "id": "tvpllqlbkwhzmsdwalvsurohri",
  "input": {
    "fp16": true,
    "mask": "https://replicate.delivery/pbxt/JgGgeIePTonFUOOZsJReOpIWuhDz2BSVL6v5z2vMsESQRjUI/watermark-mask.png",
    "mode": "video_inpainting",
    "video": "https://replicate.delivery/pbxt/JgGgeACVH84IkobKLPrqDvYgO4SH2f049BPdBHqfCI9RlOhf/running_car_watermark.mp4",
    "width": -1,
    "height": -1,
    "scale_h": 1,
    "scale_w": 1,
    "save_fps": 24,
    "raft_iter": 20,
    "ref_stride": 10,
    "resize_ratio": 1,
    "mask_dilation": 4,
    "neighbor_length": 10,
    "subvideo_length": 80,
    "return_input_video": true
  },
  "logs": "Interpreting mask as a single static file...\nProcessing: tmp80zu13oyrunning_car_watermark [292 frames]...\n  0%|          | 0/59 [00:00<?, ?it/s]\n  2%|▏         | 1/59 [00:00<00:43,  1.32it/s]\n  3%|▎         | 2/59 [00:01<00:57,  1.01s/it]\n  5%|▌         | 3/59 [00:03<00:58,  1.05s/it]\n  7%|▋         | 4/59 [00:04<00:57,  1.04s/it]\n  8%|▊         | 5/59 [00:05<00:56,  1.05s/it]\n 10%|█         | 6/59 [00:06<00:55,  1.05s/it]\n 12%|█▏        | 7/59 [00:07<00:55,  1.07s/it]\n 14%|█▎        | 8/59 [00:08<00:54,  1.07s/it]\n 15%|█▌        | 9/59 [00:09<00:54,  1.09s/it]\n 17%|█▋        | 10/59 [00:10<00:53,  1.09s/it]\n 19%|█▊        | 11/59 [00:11<00:52,  1.10s/it]\n 20%|██        | 12/59 [00:12<00:51,  1.11s/it]\n 22%|██▏       | 13/59 [00:13<00:51,  1.11s/it]\n 24%|██▎       | 14/59 [00:15<00:50,  1.11s/it]\n 25%|██▌       | 15/59 [00:16<00:49,  1.12s/it]\n 27%|██▋       | 16/59 [00:17<00:48,  1.12s/it]\n 29%|██▉       | 17/59 [00:18<00:47,  1.12s/it]\n 31%|███       | 18/59 [00:19<00:45,  1.12s/it]\n 32%|███▏      | 19/59 [00:20<00:44,  1.12s/it]\n 34%|███▍      | 20/59 [00:21<00:43,  1.12s/it]\n 36%|███▌      | 21/59 [00:22<00:42,  1.12s/it]\n 37%|███▋      | 22/59 [00:24<00:41,  1.12s/it]\n 39%|███▉      | 23/59 [00:25<00:40,  1.13s/it]\n 41%|████      | 24/59 [00:26<00:39,  1.13s/it]\n 42%|████▏     | 25/59 [00:27<00:38,  1.13s/it]\n 44%|████▍     | 26/59 [00:28<00:37,  1.13s/it]\n 46%|████▌     | 27/59 [00:29<00:36,  1.13s/it]\n 47%|████▋     | 28/59 [00:30<00:34,  1.13s/it]\n 49%|████▉     | 29/59 [00:31<00:33,  1.13s/it]\n 51%|█████     | 30/59 [00:33<00:32,  1.13s/it]\n 53%|█████▎    | 31/59 [00:34<00:31,  1.13s/it]\n 54%|█████▍    | 32/59 [00:35<00:30,  1.13s/it]\n 56%|█████▌    | 33/59 [00:36<00:29,  1.13s/it]\n 58%|█████▊    | 34/59 [00:37<00:28,  1.12s/it]\n 59%|█████▉    | 35/59 [00:38<00:27,  1.13s/it]\n 61%|██████    | 36/59 [00:39<00:25,  1.13s/it]\n 63%|██████▎   | 37/59 [00:40<00:24,  1.12s/it]\n 64%|██████▍   | 38/59 [00:42<00:23,  1.12s/it]\n 66%|██████▌   | 39/59 [00:43<00:22,  1.12s/it]\n 68%|██████▊   | 40/59 [00:44<00:21,  1.13s/it]\n 69%|██████▉   | 41/59 [00:45<00:20,  1.13s/it]\n 71%|███████   | 42/59 [00:46<00:19,  1.16s/it]\n 73%|███████▎  | 43/59 [00:48<00:19,  1.23s/it]\n 75%|███████▍  | 44/59 [00:49<00:18,  1.20s/it]\n 76%|███████▋  | 45/59 [00:50<00:16,  1.18s/it]\n 78%|███████▊  | 46/59 [00:51<00:15,  1.16s/it]\n 80%|███████▉  | 47/59 [00:52<00:13,  1.15s/it]\n 81%|████████▏ | 48/59 [00:53<00:12,  1.14s/it]\n 83%|████████▎ | 49/59 [00:54<00:11,  1.14s/it]\n 85%|████████▍ | 50/59 [00:56<00:10,  1.14s/it]\n 86%|████████▋ | 51/59 [00:57<00:09,  1.14s/it]\n 88%|████████▊ | 52/59 [00:58<00:07,  1.13s/it]\n 90%|████████▉ | 53/59 [00:59<00:06,  1.13s/it]\n 92%|█████████▏| 54/59 [01:00<00:05,  1.12s/it]\n 93%|█████████▎| 55/59 [01:01<00:04,  1.11s/it]\n 95%|█████████▍| 56/59 [01:02<00:03,  1.09s/it]\n 97%|█████████▋| 57/59 [01:03<00:02,  1.08s/it]\n 98%|█████████▊| 58/59 [01:04<00:01,  1.07s/it]\n100%|██████████| 59/59 [01:05<00:00,  1.05it/s]\n100%|██████████| 59/59 [01:05<00:00,  1.11s/it]\nIMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (972, 540) to (976, 544) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).\n[swscaler @ 0x7075ec0] Warning: data is not aligned! This can lead to a speed loss\nIMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (972, 540) to (976, 544) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).\n[swscaler @ 0x6249ec0] Warning: data is not aligned! This can lead to a speed loss\nAll results are saved in results/tmp80zu13oyrunning_car_watermark",
  "metrics": {
    "predict_time": 151.436898,
    "total_time": 217.644706
  },
  "output": [
    "https://pbxt.replicate.delivery/4pdTO5qjJtpsKRUzpJHbXkv3HHKRNg49O2uydO30pwFqCRbE/masked_in.mp4",
    "https://pbxt.replicate.delivery/UDeBarXTIPXCGCGlmeUrktwhkwUG9vGDFjhTEvuAngXpKEtRA/inpaint_out.mp4"
  ],
  "started_at": "2023-10-11T12:07:46.745710Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/tvpllqlbkwhzmsdwalvsurohri",
    "cancel": "https://api.replicate.com/v1/predictions/tvpllqlbkwhzmsdwalvsurohri/cancel"
  },
  "version": "b3e1ec853a25dbbd5178128139eb773612924c7f032c9919c20c254e6813a5ad"
}

Generated in

2 minutes 31 seconds

Tweak itReport

Interpreting mask as a single static file...
Processing: tmp80zu13oyrunning_car_watermark [292 frames]...
  0%|          | 0/59 [00:00<?, ?it/s]
  2%|▏         | 1/59 [00:00<00:43,  1.32it/s]
  3%|▎         | 2/59 [00:01<00:57,  1.01s/it]
  5%|▌         | 3/59 [00:03<00:58,  1.05s/it]
  7%|▋         | 4/59 [00:04<00:57,  1.04s/it]
  8%|▊         | 5/59 [00:05<00:56,  1.05s/it]
 10%|█         | 6/59 [00:06<00:55,  1.05s/it]
 12%|█▏        | 7/59 [00:07<00:55,  1.07s/it]
 14%|█▎        | 8/59 [00:08<00:54,  1.07s/it]
 15%|█▌        | 9/59 [00:09<00:54,  1.09s/it]
 17%|█▋        | 10/59 [00:10<00:53,  1.09s/it]
 19%|█▊        | 11/59 [00:11<00:52,  1.10s/it]
 20%|██        | 12/59 [00:12<00:51,  1.11s/it]
 22%|██▏       | 13/59 [00:13<00:51,  1.11s/it]
 24%|██▎       | 14/59 [00:15<00:50,  1.11s/it]
 25%|██▌       | 15/59 [00:16<00:49,  1.12s/it]
 27%|██▋       | 16/59 [00:17<00:48,  1.12s/it]
 29%|██▉       | 17/59 [00:18<00:47,  1.12s/it]
 31%|███       | 18/59 [00:19<00:45,  1.12s/it]
 32%|███▏      | 19/59 [00:20<00:44,  1.12s/it]
 34%|███▍      | 20/59 [00:21<00:43,  1.12s/it]
 36%|███▌      | 21/59 [00:22<00:42,  1.12s/it]
 37%|███▋      | 22/59 [00:24<00:41,  1.12s/it]
 39%|███▉      | 23/59 [00:25<00:40,  1.13s/it]
 41%|████      | 24/59 [00:26<00:39,  1.13s/it]
 42%|████▏     | 25/59 [00:27<00:38,  1.13s/it]
 44%|████▍     | 26/59 [00:28<00:37,  1.13s/it]
 46%|████▌     | 27/59 [00:29<00:36,  1.13s/it]
 47%|████▋     | 28/59 [00:30<00:34,  1.13s/it]
 49%|████▉     | 29/59 [00:31<00:33,  1.13s/it]
 51%|█████     | 30/59 [00:33<00:32,  1.13s/it]
 53%|█████▎    | 31/59 [00:34<00:31,  1.13s/it]
 54%|█████▍    | 32/59 [00:35<00:30,  1.13s/it]
 56%|█████▌    | 33/59 [00:36<00:29,  1.13s/it]
 58%|█████▊    | 34/59 [00:37<00:28,  1.12s/it]
 59%|█████▉    | 35/59 [00:38<00:27,  1.13s/it]
 61%|██████    | 36/59 [00:39<00:25,  1.13s/it]
 63%|██████▎   | 37/59 [00:40<00:24,  1.12s/it]
 64%|██████▍   | 38/59 [00:42<00:23,  1.12s/it]
 66%|██████▌   | 39/59 [00:43<00:22,  1.12s/it]
 68%|██████▊   | 40/59 [00:44<00:21,  1.13s/it]
 69%|██████▉   | 41/59 [00:45<00:20,  1.13s/it]
 71%|███████   | 42/59 [00:46<00:19,  1.16s/it]
 73%|███████▎  | 43/59 [00:48<00:19,  1.23s/it]
 75%|███████▍  | 44/59 [00:49<00:18,  1.20s/it]
 76%|███████▋  | 45/59 [00:50<00:16,  1.18s/it]
 78%|███████▊  | 46/59 [00:51<00:15,  1.16s/it]
 80%|███████▉  | 47/59 [00:52<00:13,  1.15s/it]
 81%|████████▏ | 48/59 [00:53<00:12,  1.14s/it]
 83%|████████▎ | 49/59 [00:54<00:11,  1.14s/it]
 85%|████████▍ | 50/59 [00:56<00:10,  1.14s/it]
 86%|████████▋ | 51/59 [00:57<00:09,  1.14s/it]
 88%|████████▊ | 52/59 [00:58<00:07,  1.13s/it]
 90%|████████▉ | 53/59 [00:59<00:06,  1.13s/it]
 92%|█████████▏| 54/59 [01:00<00:05,  1.12s/it]
 93%|█████████▎| 55/59 [01:01<00:04,  1.11s/it]
 95%|█████████▍| 56/59 [01:02<00:03,  1.09s/it]
 97%|█████████▋| 57/59 [01:03<00:02,  1.08s/it]
 98%|█████████▊| 58/59 [01:04<00:01,  1.07s/it]
100%|██████████| 59/59 [01:05<00:00,  1.05it/s]
100%|██████████| 59/59 [01:05<00:00,  1.11s/it]
IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (972, 540) to (976, 544) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[swscaler @ 0x7075ec0] Warning: data is not aligned! This can lead to a speed loss
IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (972, 540) to (976, 544) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[swscaler @ 0x6249ec0] Warning: data is not aligned! This can lead to a speed loss
All results are saved in results/tmp80zu13oyrunning_car_watermark