okaris/live-portrait – Run with an API on Replicate

okaris / live-portrait

Cold

Public
98.5K runs
L40S

Run with an API

Playground API Examples README Versions

Input

source_image

*file

Preview

Source image for animation

source_video

file

Source video for animation

driving_video

*file

Preview

Driving video for animation

flag_relative

boolean

Use relative motion

Default: true

flag_do_crop

boolean

Crop source image/video

Default: true

flag_remap

boolean

Use paste-back

Default: true

flag_crop_driving_video

boolean

Crop driving video

Default: false

flag_video_editing_head_rotation

boolean

Use relative head rotation (v2v)

Default: false

scale

number

(minimum: 1.8, maximum: 3.2)

Source crop scale

Default: 2.3

vx_ratio

number

(minimum: -0.5, maximum: 0.5)

Source crop x

Default: 0

vy_ratio

number

(minimum: -0.5, maximum: 0.5)

Source crop y

Default: -0.125

scale_crop_driving_video

number

(minimum: 1.8, maximum: 3.2)

Driving crop scale

Default: 2.2

vx_ratio_crop_driving_video

number

(minimum: -0.5, maximum: 0.5)

Driving crop x

Default: 0

vy_ratio_crop_driving_video

number

(minimum: -0.5, maximum: 0.5)

Driving crop y

Default: -0.1

driving_smooth_observation_variance

number

(minimum: 1e-11, maximum: 0.01)

Motion smooth strength (v2v)

Default: 3e-7

Run this model in Node.js with one line of code:

npx create-replicate --model=okaris/live-portrait

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run okaris/live-portrait using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "okaris/live-portrait:8be2edeab144ba0865f9fa84168f621ee417a2003db947802f900519f7c43300",
  {
    input: {
      scale: 2.3,
      vx_ratio: 0,
      vy_ratio: -0.01,
      flag_remap: true,
      flag_do_crop: true,
      source_image: "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
      driving_video: "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
      flag_relative: true,
      flag_crop_driving_video: false,
      scale_crop_driving_video: 2.2,
      vx_ratio_crop_driving_video: 0,
      vy_ratio_crop_driving_video: -0.1,
      flag_video_editing_head_rotation: false,
      driving_smooth_observation_variance: 0.01
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run okaris/live-portrait using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "okaris/live-portrait:8be2edeab144ba0865f9fa84168f621ee417a2003db947802f900519f7c43300",
    input={
        "scale": 2.3,
        "vx_ratio": 0,
        "vy_ratio": -0.01,
        "flag_remap": True,
        "flag_do_crop": True,
        "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
        "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
        "flag_relative": True,
        "flag_crop_driving_video": False,
        "scale_crop_driving_video": 2.2,
        "vx_ratio_crop_driving_video": 0,
        "vy_ratio_crop_driving_video": -0.1,
        "flag_video_editing_head_rotation": False,
        "driving_smooth_observation_variance": 0.01
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run okaris/live-portrait using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "8be2edeab144ba0865f9fa84168f621ee417a2003db947802f900519f7c43300",
    "input": {
      "scale": 2.3,
      "vx_ratio": 0,
      "vy_ratio": -0.01,
      "flag_remap": true,
      "flag_do_crop": true,
      "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
      "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
      "flag_relative": true,
      "flag_crop_driving_video": false,
      "scale_crop_driving_video": 2.2,
      "vx_ratio_crop_driving_video": 0,
      "vy_ratio_crop_driving_video": -0.1,
      "flag_video_editing_head_rotation": false,
      "driving_smooth_observation_variance": 0.01
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/okaris/live-portrait@sha256:8be2edeab144ba0865f9fa84168f621ee417a2003db947802f900519f7c43300 \
  -i 'scale=2.3' \
  -i 'vx_ratio=0' \
  -i 'vy_ratio=-0.01' \
  -i 'flag_remap=true' \
  -i 'flag_do_crop=true' \
  -i 'source_image="https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp"' \
  -i 'driving_video="https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov"' \
  -i 'flag_relative=true' \
  -i 'flag_crop_driving_video=false' \
  -i 'scale_crop_driving_video=2.2' \
  -i 'vx_ratio_crop_driving_video=0' \
  -i 'vy_ratio_crop_driving_video=-0.1' \
  -i 'flag_video_editing_head_rotation=false' \
  -i 'driving_smooth_observation_variance=0.01'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/okaris/live-portrait@sha256:8be2edeab144ba0865f9fa84168f621ee417a2003db947802f900519f7c43300
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "scale": 2.3,
      "vx_ratio": 0,
      "vy_ratio": -0.01,
      "flag_remap": true,
      "flag_do_crop": true,
      "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
      "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
      "flag_relative": true,
      "flag_crop_driving_video": false,
      "scale_crop_driving_video": 2.2,
      "vx_ratio_crop_driving_video": 0,
      "vy_ratio_crop_driving_video": -0.1,
      "flag_video_editing_head_rotation": false,
      "driving_smooth_observation_variance": 0.01
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2024-07-30T06:37:50.639102Z",
  "created_at": "2024-07-30T06:35:36.928000Z",
  "data_removed": false,
  "error": null,
  "id": "66e6tkztc1rgp0ch09cszkka1m",
  "input": {
    "scale": 2.3,
    "vx_ratio": 0,
    "vy_ratio": -0.01,
    "flag_remap": true,
    "flag_do_crop": true,
    "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
    "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
    "flag_relative": true,
    "flag_crop_driving_video": false,
    "scale_crop_driving_video": 2.2,
    "vx_ratio_crop_driving_video": 0,
    "vy_ratio_crop_driving_video": -0.1,
    "flag_video_editing_head_rotation": false,
    "driving_smooth_observation_variance": 0.01
  },
  "logs": "args_user:  {'source': '/tmp/tmpnmdzy8ebimage-13.webp', 'driving': '/tmp/tmp9yp2f4vhIMG_2228.mov', 'flag_relative_motion': True, 'flag_do_crop': True, 'flag_pasteback': True, 'flag_crop_driving_video': False, 'flag_video_editing_head_rotation': False, 'scale': 2.3, 'vx_ratio': 0.0, 'vy_ratio': -0.01, 'scale_crop_driving_video': 2.2, 'vx_ratio_crop_driving_video': 0.0, 'vy_ratio_crop_driving_video': -0.1, 'driving_smooth_observation_variance': 0.01}\nLoad source image from                   live_portrait_pipeline.py:90\n/tmp/tmpnmdzy8ebimage-13.webp\nLoad driving video from:                live_portrait_pipeline.py:128\n/tmp/tmp9yp2f4vhIMG_2228.mov, FPS is 30\n[06:37:20] Start making driving motion template... live_portrait_pipeline.py:134\nMaking motion templates... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01\n[06:37:26] Dump motion template to                 live_portrait_pipeline.py:159\n/tmp/tmp9yp2f4vhIMG_2228.pkl\nPrepared pasteback mask done.           live_portrait_pipeline.py:168\n[06:37:27] The animated video consists of 154      live_portrait_pipeline.py:238\nframes.\n🚀Animating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:14\nConcatenating result... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00\n[06:37:46] Audio is selected from                  live_portrait_pipeline.py:387\n/tmp/tmp9yp2f4vhIMG_2228.mov, concat\nmode\nVideo with audio generated successfully:                 video.py:203\nanimations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_conc\nat_with_audio.mp4\nReplace                                 live_portrait_pipeline.py:390\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_concat.mp4 with\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_concat_with_audio.mp4\n[06:37:49] Audio is selected from                  live_portrait_pipeline.py:404\n/tmp/tmp9yp2f4vhIMG_2228.mov\nVideo with audio generated successfully:                 video.py:203\nanimations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_with\n_audio.mp4\nReplace                                 live_portrait_pipeline.py:407\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228.mp4 with\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_with_audio.mp4\nAnimated template:                      live_portrait_pipeline.py:411\n/tmp/tmp9yp2f4vhIMG_2228.pkl, you can\nspecify `-d` argument with this\ntemplate path next time to avoid\ncropping video, motion making and\nprotecting privacy.\nAnimated video:                         live_portrait_pipeline.py:412\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228.mp4\nAnimated video with concat:             live_portrait_pipeline.py:413\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_concat.mp4",
  "metrics": {
    "predict_time": 39.488535516,
    "total_time": 133.711102
  },
  "output": [
    "https://replicate.delivery/pbxt/cBknJMmX48o2P90FnkN7ukE3NkL7KIMV6G9r1uK3BKhP8YzE/output_0.mp4",
    "https://replicate.delivery/pbxt/EvurHeU9zZUlX6wr7kJUwGqGakj3nKe9ZCkFJkAegPX9hHbmA/output_1.mp4"
  ],
  "started_at": "2024-07-30T06:37:11.150566Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/66e6tkztc1rgp0ch09cszkka1m",
    "cancel": "https://api.replicate.com/v1/predictions/66e6tkztc1rgp0ch09cszkka1m/cancel"
  },
  "version": "4e1ef915c27727ed5fd72b026178f5f37e354b10610401c1b65a5df73aad1fda"
}

Generated in

39.5 seconds

Tweak it ShareReport

args_user:  {'source': '/tmp/tmpnmdzy8ebimage-13.webp', 'driving': '/tmp/tmp9yp2f4vhIMG_2228.mov', 'flag_relative_motion': True, 'flag_do_crop': True, 'flag_pasteback': True, 'flag_crop_driving_video': False, 'flag_video_editing_head_rotation': False, 'scale': 2.3, 'vx_ratio': 0.0, 'vy_ratio': -0.01, 'scale_crop_driving_video': 2.2, 'vx_ratio_crop_driving_video': 0.0, 'vy_ratio_crop_driving_video': -0.1, 'driving_smooth_observation_variance': 0.01}
Load source image from                   live_portrait_pipeline.py:90
/tmp/tmpnmdzy8ebimage-13.webp
Load driving video from:                live_portrait_pipeline.py:128
/tmp/tmp9yp2f4vhIMG_2228.mov, FPS is 30
[06:37:20] Start making driving motion template... live_portrait_pipeline.py:134
Making motion templates... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01
[06:37:26] Dump motion template to                 live_portrait_pipeline.py:159
/tmp/tmp9yp2f4vhIMG_2228.pkl
Prepared pasteback mask done.           live_portrait_pipeline.py:168
[06:37:27] The animated video consists of 154      live_portrait_pipeline.py:238
frames.
🚀Animating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:14
Concatenating result... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
[06:37:46] Audio is selected from                  live_portrait_pipeline.py:387
/tmp/tmp9yp2f4vhIMG_2228.mov, concat
mode
Video with audio generated successfully:                 video.py:203
animations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_conc
at_with_audio.mp4
Replace                                 live_portrait_pipeline.py:390
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_concat.mp4 with
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_concat_with_audio.mp4
[06:37:49] Audio is selected from                  live_portrait_pipeline.py:404
/tmp/tmp9yp2f4vhIMG_2228.mov
Video with audio generated successfully:                 video.py:203
animations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_with
_audio.mp4
Replace                                 live_portrait_pipeline.py:407
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228.mp4 with
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_with_audio.mp4
Animated template:                      live_portrait_pipeline.py:411
/tmp/tmp9yp2f4vhIMG_2228.pkl, you can
specify `-d` argument with this
template path next time to avoid
cropping video, motion making and
protecting privacy.
Animated video:                         live_portrait_pipeline.py:412
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228.mp4
Animated video with concat:             live_portrait_pipeline.py:413
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_concat.mp4

This output was created using a different version of the model, okaris/live-portrait:4e1ef915.

Examples

View more examples

Run time and cost

This model costs approximately $0.051 to run on Replicate, or 19 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 52 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.