okaris/live-portrait:4e1ef915 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

okaris /live-portrait:4e1ef915

Playground API

Input

source_image

*file

Preview

Source image for animation

source_video

file

Source video for animation

driving_video

*file

Preview

Driving video for animation

flag_relative

boolean

Use relative motion

Default: true

flag_do_crop

boolean

Crop source image/video

Default: true

flag_remap

boolean

Use paste-back

Default: true

flag_crop_driving_video

boolean

Crop driving video

Default: false

flag_video_editing_head_rotation

boolean

Use relative head rotation (v2v)

Default: false

scale

number

(minimum: 1.8, maximum: 3.2)

Source crop scale

Default: 2.3

vx_ratio

number

(minimum: -0.5, maximum: 0.5)

Source crop x

Default: 0

vy_ratio

number

(minimum: -0.5, maximum: 0.5)

Source crop y

Default: -0.125

scale_crop_driving_video

number

(minimum: 1.8, maximum: 3.2)

Driving crop scale

Default: 2.2

vx_ratio_crop_driving_video

number

(minimum: -0.5, maximum: 0.5)

Driving crop x

Default: 0

vy_ratio_crop_driving_video

number

(minimum: -0.5, maximum: 0.5)

Driving crop y

Default: -0.1

driving_smooth_observation_variance

number

(minimum: 1e-11, maximum: 0.01)

Motion smooth strength (v2v)

Default: 3e-7

Run this model in Node.js with one line of code:

npx create-replicate --model=okaris/live-portrait

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run okaris/live-portrait using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "okaris/live-portrait:4e1ef915c27727ed5fd72b026178f5f37e354b10610401c1b65a5df73aad1fda",
  {
    input: {
      scale: 2.3,
      vx_ratio: 0,
      vy_ratio: -0.01,
      flag_remap: true,
      flag_do_crop: true,
      source_image: "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
      driving_video: "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
      flag_relative: true,
      flag_crop_driving_video: false,
      scale_crop_driving_video: 2.2,
      vx_ratio_crop_driving_video: 0,
      vy_ratio_crop_driving_video: -0.1,
      flag_video_editing_head_rotation: false,
      driving_smooth_observation_variance: 0.01
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run okaris/live-portrait using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "okaris/live-portrait:4e1ef915c27727ed5fd72b026178f5f37e354b10610401c1b65a5df73aad1fda",
    input={
        "scale": 2.3,
        "vx_ratio": 0,
        "vy_ratio": -0.01,
        "flag_remap": True,
        "flag_do_crop": True,
        "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
        "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
        "flag_relative": True,
        "flag_crop_driving_video": False,
        "scale_crop_driving_video": 2.2,
        "vx_ratio_crop_driving_video": 0,
        "vy_ratio_crop_driving_video": -0.1,
        "flag_video_editing_head_rotation": False,
        "driving_smooth_observation_variance": 0.01
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run okaris/live-portrait using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "4e1ef915c27727ed5fd72b026178f5f37e354b10610401c1b65a5df73aad1fda",
    "input": {
      "scale": 2.3,
      "vx_ratio": 0,
      "vy_ratio": -0.01,
      "flag_remap": true,
      "flag_do_crop": true,
      "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
      "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
      "flag_relative": true,
      "flag_crop_driving_video": false,
      "scale_crop_driving_video": 2.2,
      "vx_ratio_crop_driving_video": 0,
      "vy_ratio_crop_driving_video": -0.1,
      "flag_video_editing_head_rotation": false,
      "driving_smooth_observation_variance": 0.01
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/okaris/live-portrait@sha256:4e1ef915c27727ed5fd72b026178f5f37e354b10610401c1b65a5df73aad1fda \
  -i 'scale=2.3' \
  -i 'vx_ratio=0' \
  -i 'vy_ratio=-0.01' \
  -i 'flag_remap=true' \
  -i 'flag_do_crop=true' \
  -i 'source_image="https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp"' \
  -i 'driving_video="https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov"' \
  -i 'flag_relative=true' \
  -i 'flag_crop_driving_video=false' \
  -i 'scale_crop_driving_video=2.2' \
  -i 'vx_ratio_crop_driving_video=0' \
  -i 'vy_ratio_crop_driving_video=-0.1' \
  -i 'flag_video_editing_head_rotation=false' \
  -i 'driving_smooth_observation_variance=0.01'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/okaris/live-portrait@sha256:4e1ef915c27727ed5fd72b026178f5f37e354b10610401c1b65a5df73aad1fda
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "scale": 2.3,
      "vx_ratio": 0,
      "vy_ratio": -0.01,
      "flag_remap": true,
      "flag_do_crop": true,
      "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
      "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
      "flag_relative": true,
      "flag_crop_driving_video": false,
      "scale_crop_driving_video": 2.2,
      "vx_ratio_crop_driving_video": 0,
      "vy_ratio_crop_driving_video": -0.1,
      "flag_video_editing_head_rotation": false,
      "driving_smooth_observation_variance": 0.01
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2024-07-30T06:37:50.639102Z",
  "created_at": "2024-07-30T06:35:36.928000Z",
  "data_removed": false,
  "error": null,
  "id": "66e6tkztc1rgp0ch09cszkka1m",
  "input": {
    "scale": 2.3,
    "vx_ratio": 0,
    "vy_ratio": -0.01,
    "flag_remap": true,
    "flag_do_crop": true,
    "source_image": "https://replicate.delivery/pbxt/LM1eHhq9VUnim4cZmYHDLBZp7eKRj3QXxTid79ifYn37NJoi/image-13.webp",
    "driving_video": "https://replicate.delivery/pbxt/LLyKx8PIeZAMBkTTpxCpoM1lWSx6YBBrRGXeMhBOxjFdpyFe/IMG_2228.mov",
    "flag_relative": true,
    "flag_crop_driving_video": false,
    "scale_crop_driving_video": 2.2,
    "vx_ratio_crop_driving_video": 0,
    "vy_ratio_crop_driving_video": -0.1,
    "flag_video_editing_head_rotation": false,
    "driving_smooth_observation_variance": 0.01
  },
  "logs": "args_user:  {'source': '/tmp/tmpnmdzy8ebimage-13.webp', 'driving': '/tmp/tmp9yp2f4vhIMG_2228.mov', 'flag_relative_motion': True, 'flag_do_crop': True, 'flag_pasteback': True, 'flag_crop_driving_video': False, 'flag_video_editing_head_rotation': False, 'scale': 2.3, 'vx_ratio': 0.0, 'vy_ratio': -0.01, 'scale_crop_driving_video': 2.2, 'vx_ratio_crop_driving_video': 0.0, 'vy_ratio_crop_driving_video': -0.1, 'driving_smooth_observation_variance': 0.01}\nLoad source image from                   live_portrait_pipeline.py:90\n/tmp/tmpnmdzy8ebimage-13.webp\nLoad driving video from:                live_portrait_pipeline.py:128\n/tmp/tmp9yp2f4vhIMG_2228.mov, FPS is 30\n[06:37:20] Start making driving motion template... live_portrait_pipeline.py:134\nMaking motion templates... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01\n[06:37:26] Dump motion template to                 live_portrait_pipeline.py:159\n/tmp/tmp9yp2f4vhIMG_2228.pkl\nPrepared pasteback mask done.           live_portrait_pipeline.py:168\n[06:37:27] The animated video consists of 154      live_portrait_pipeline.py:238\nframes.\n🚀Animating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:14\nConcatenating result... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00\n[06:37:46] Audio is selected from                  live_portrait_pipeline.py:387\n/tmp/tmp9yp2f4vhIMG_2228.mov, concat\nmode\nVideo with audio generated successfully:                 video.py:203\nanimations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_conc\nat_with_audio.mp4\nReplace                                 live_portrait_pipeline.py:390\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_concat.mp4 with\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_concat_with_audio.mp4\n[06:37:49] Audio is selected from                  live_portrait_pipeline.py:404\n/tmp/tmp9yp2f4vhIMG_2228.mov\nVideo with audio generated successfully:                 video.py:203\nanimations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_with\n_audio.mp4\nReplace                                 live_portrait_pipeline.py:407\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228.mp4 with\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_with_audio.mp4\nAnimated template:                      live_portrait_pipeline.py:411\n/tmp/tmp9yp2f4vhIMG_2228.pkl, you can\nspecify `-d` argument with this\ntemplate path next time to avoid\ncropping video, motion making and\nprotecting privacy.\nAnimated video:                         live_portrait_pipeline.py:412\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228.mp4\nAnimated video with concat:             live_portrait_pipeline.py:413\nanimations/tmpnmdzy8ebimage-13--tmp9yp2\nf4vhIMG_2228_concat.mp4",
  "metrics": {
    "predict_time": 39.488535516,
    "total_time": 133.711102
  },
  "output": [
    "https://replicate.delivery/pbxt/cBknJMmX48o2P90FnkN7ukE3NkL7KIMV6G9r1uK3BKhP8YzE/output_0.mp4",
    "https://replicate.delivery/pbxt/EvurHeU9zZUlX6wr7kJUwGqGakj3nKe9ZCkFJkAegPX9hHbmA/output_1.mp4"
  ],
  "started_at": "2024-07-30T06:37:11.150566Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/66e6tkztc1rgp0ch09cszkka1m",
    "cancel": "https://api.replicate.com/v1/predictions/66e6tkztc1rgp0ch09cszkka1m/cancel"
  },
  "version": "4e1ef915c27727ed5fd72b026178f5f37e354b10610401c1b65a5df73aad1fda"
}

Generated in

39.5 seconds

Tweak it ShareReport

args_user:  {'source': '/tmp/tmpnmdzy8ebimage-13.webp', 'driving': '/tmp/tmp9yp2f4vhIMG_2228.mov', 'flag_relative_motion': True, 'flag_do_crop': True, 'flag_pasteback': True, 'flag_crop_driving_video': False, 'flag_video_editing_head_rotation': False, 'scale': 2.3, 'vx_ratio': 0.0, 'vy_ratio': -0.01, 'scale_crop_driving_video': 2.2, 'vx_ratio_crop_driving_video': 0.0, 'vy_ratio_crop_driving_video': -0.1, 'driving_smooth_observation_variance': 0.01}
Load source image from                   live_portrait_pipeline.py:90
/tmp/tmpnmdzy8ebimage-13.webp
Load driving video from:                live_portrait_pipeline.py:128
/tmp/tmp9yp2f4vhIMG_2228.mov, FPS is 30
[06:37:20] Start making driving motion template... live_portrait_pipeline.py:134
Making motion templates... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01
[06:37:26] Dump motion template to                 live_portrait_pipeline.py:159
/tmp/tmp9yp2f4vhIMG_2228.pkl
Prepared pasteback mask done.           live_portrait_pipeline.py:168
[06:37:27] The animated video consists of 154      live_portrait_pipeline.py:238
frames.
🚀Animating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:14
Concatenating result... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
[06:37:46] Audio is selected from                  live_portrait_pipeline.py:387
/tmp/tmp9yp2f4vhIMG_2228.mov, concat
mode
Video with audio generated successfully:                 video.py:203
animations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_conc
at_with_audio.mp4
Replace                                 live_portrait_pipeline.py:390
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_concat.mp4 with
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_concat_with_audio.mp4
[06:37:49] Audio is selected from                  live_portrait_pipeline.py:404
/tmp/tmp9yp2f4vhIMG_2228.mov
Video with audio generated successfully:                 video.py:203
animations/tmpnmdzy8ebimage-13--tmp9yp2f4vhIMG_2228_with
_audio.mp4
Replace                                 live_portrait_pipeline.py:407
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228.mp4 with
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_with_audio.mp4
Animated template:                      live_portrait_pipeline.py:411
/tmp/tmp9yp2f4vhIMG_2228.pkl, you can
specify `-d` argument with this
template path next time to avoid
cropping video, motion making and
protecting privacy.
Animated video:                         live_portrait_pipeline.py:412
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228.mp4
Animated video with concat:             live_portrait_pipeline.py:413
animations/tmpnmdzy8ebimage-13--tmp9yp2
f4vhIMG_2228_concat.mp4