fofr / video-morpher

Generate a video that morphs between subjects, with an optional style

Cold

Public
14.6K runs
L40S
GitHub
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Prediction

fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf

Model

fofr/video-morpher:355c6bba

zq1ppbr981rgg0cf22trg7ex1r

Status

Succeeded

Source

Web

Hardware

A40 (Large)

Total duration

4m 47s

Created

about 1 year ago by @fofr

Input

mode: upscaled-and-interpolated
prompt
checkpoint: anime
style_image
aspect_ratio: 4:3
style_strength: 0.25
negative_prompt
subject_image_1
subject_image_2
subject_image_3
subject_image_4

{
  "mode": "upscaled-and-interpolated",
  "prompt": "",
  "checkpoint": "anime",
  "style_image": "https://replicate.delivery/pbxt/Knt8RPT8KLlsXLreP04hXAULTlrL29TH9W8NNUV3eKDfGkug/replicate-prediction-6da3fldbhkwkmaeba4bhzif72m.png",
  "aspect_ratio": "4:3",
  "style_strength": 0.25,
  "negative_prompt": "",
  "subject_image_1": "https://replicate.delivery/pbxt/Knt8R9B6sVKljclLEsUe1tUz5gELYq3WQ9mebHcdEnpaKEvY/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
  "subject_image_2": "https://replicate.delivery/pbxt/Knt8QyjO5NMhX7S9wCjJ6ZKCxYnL2cBSKyeU9oJlpABFIjHb/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
  "subject_image_3": "https://replicate.delivery/pbxt/Knt8REy4ySLJqPNNsw6RPAnxfdmSEXyWWjFsfqsMwpIX11tK/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
  "subject_image_4": "https://replicate.delivery/pbxt/Knt8RgLdi4GZ7AOdQ7INvzI4SMKWlR1eJQjfIxK8cZIzifIA/ComfyUI_02710_.png"
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf",
  {
    input: {
      mode: "upscaled-and-interpolated",
      prompt: "",
      checkpoint: "anime",
      style_image: "https://replicate.delivery/pbxt/Knt8RPT8KLlsXLreP04hXAULTlrL29TH9W8NNUV3eKDfGkug/replicate-prediction-6da3fldbhkwkmaeba4bhzif72m.png",
      aspect_ratio: "4:3",
      style_strength: 0.25,
      negative_prompt: "",
      subject_image_1: "https://replicate.delivery/pbxt/Knt8R9B6sVKljclLEsUe1tUz5gELYq3WQ9mebHcdEnpaKEvY/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
      subject_image_2: "https://replicate.delivery/pbxt/Knt8QyjO5NMhX7S9wCjJ6ZKCxYnL2cBSKyeU9oJlpABFIjHb/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
      subject_image_3: "https://replicate.delivery/pbxt/Knt8REy4ySLJqPNNsw6RPAnxfdmSEXyWWjFsfqsMwpIX11tK/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
      subject_image_4: "https://replicate.delivery/pbxt/Knt8RgLdi4GZ7AOdQ7INvzI4SMKWlR1eJQjfIxK8cZIzifIA/ComfyUI_02710_.png"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf",
    input={
        "mode": "upscaled-and-interpolated",
        "prompt": "",
        "checkpoint": "anime",
        "style_image": "https://replicate.delivery/pbxt/Knt8RPT8KLlsXLreP04hXAULTlrL29TH9W8NNUV3eKDfGkug/replicate-prediction-6da3fldbhkwkmaeba4bhzif72m.png",
        "aspect_ratio": "4:3",
        "style_strength": 0.25,
        "negative_prompt": "",
        "subject_image_1": "https://replicate.delivery/pbxt/Knt8R9B6sVKljclLEsUe1tUz5gELYq3WQ9mebHcdEnpaKEvY/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
        "subject_image_2": "https://replicate.delivery/pbxt/Knt8QyjO5NMhX7S9wCjJ6ZKCxYnL2cBSKyeU9oJlpABFIjHb/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
        "subject_image_3": "https://replicate.delivery/pbxt/Knt8REy4ySLJqPNNsw6RPAnxfdmSEXyWWjFsfqsMwpIX11tK/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
        "subject_image_4": "https://replicate.delivery/pbxt/Knt8RgLdi4GZ7AOdQ7INvzI4SMKWlR1eJQjfIxK8cZIzifIA/ComfyUI_02710_.png"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf",
    "input": {
      "mode": "upscaled-and-interpolated",
      "prompt": "",
      "checkpoint": "anime",
      "style_image": "https://replicate.delivery/pbxt/Knt8RPT8KLlsXLreP04hXAULTlrL29TH9W8NNUV3eKDfGkug/replicate-prediction-6da3fldbhkwkmaeba4bhzif72m.png",
      "aspect_ratio": "4:3",
      "style_strength": 0.25,
      "negative_prompt": "",
      "subject_image_1": "https://replicate.delivery/pbxt/Knt8R9B6sVKljclLEsUe1tUz5gELYq3WQ9mebHcdEnpaKEvY/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
      "subject_image_2": "https://replicate.delivery/pbxt/Knt8QyjO5NMhX7S9wCjJ6ZKCxYnL2cBSKyeU9oJlpABFIjHb/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
      "subject_image_3": "https://replicate.delivery/pbxt/Knt8REy4ySLJqPNNsw6RPAnxfdmSEXyWWjFsfqsMwpIX11tK/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
      "subject_image_4": "https://replicate.delivery/pbxt/Knt8RgLdi4GZ7AOdQ7INvzI4SMKWlR1eJQjfIxK8cZIzifIA/ComfyUI_02710_.png"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-04-24T15:28:58.190754Z",
  "created_at": "2024-04-24T15:24:10.944000Z",
  "data_removed": false,
  "error": null,
  "id": "zq1ppbr981rgg0cf22trg7ex1r",
  "input": {
    "mode": "upscaled-and-interpolated",
    "prompt": "",
    "checkpoint": "anime",
    "style_image": "https://replicate.delivery/pbxt/Knt8RPT8KLlsXLreP04hXAULTlrL29TH9W8NNUV3eKDfGkug/replicate-prediction-6da3fldbhkwkmaeba4bhzif72m.png",
    "aspect_ratio": "4:3",
    "style_strength": 0.25,
    "negative_prompt": "",
    "subject_image_1": "https://replicate.delivery/pbxt/Knt8R9B6sVKljclLEsUe1tUz5gELYq3WQ9mebHcdEnpaKEvY/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
    "subject_image_2": "https://replicate.delivery/pbxt/Knt8QyjO5NMhX7S9wCjJ6ZKCxYnL2cBSKyeU9oJlpABFIjHb/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
    "subject_image_3": "https://replicate.delivery/pbxt/Knt8REy4ySLJqPNNsw6RPAnxfdmSEXyWWjFsfqsMwpIX11tK/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
    "subject_image_4": "https://replicate.delivery/pbxt/Knt8RgLdi4GZ7AOdQ7INvzI4SMKWlR1eJQjfIxK8cZIzifIA/ComfyUI_02710_.png"
  },
  "logs": "Random seed set to: 665779694\nChecking inputs\n✅ /tmp/inputs/2.png\n✅ /tmp/inputs/1.png\n✅ /tmp/inputs/3.png\n✅ /tmp/inputs/4.png\n✅ /tmp/inputs/circles.mp4\n✅ /tmp/inputs/style.png\n====================================\nChecking weights\nIncluding weights for IPAdapter preset: PLUS (high strength)\n⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.60s, size: 808.26MB\n✅ ip-adapter-plus_sdxl_vit-h.safetensors\n⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models\n⌛️ Downloaded RealESRGAN_x4.pth in 0.15s, size: 63.94MB\n✅ RealESRGAN_x4.pth\n⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae\n⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.35s, size: 319.14MB\n✅ vae-ft-mse-840000-ema-pruned.safetensors\n⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision\n⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 1.94s, size: 2411.24MB\n✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\n⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.18s, size: 93.63MB\n✅ ip-adapter-plus_sd15.safetensors\n⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film\n⌛️ Downloaded film_net_fp32.pt in 0.25s, size: 131.53MB\n✅ film_net_fp32.pt\n⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras\n⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.23s, size: 128.39MB\n✅ AnimateLCM_sd15_t2v_lora.safetensors\n⏳ Downloading toonyou_beta6.safetensors to ComfyUI/models/checkpoints\n⌛️ Downloaded toonyou_beta6.safetensors in 1.36s, size: 2193.39MB\n✅ toonyou_beta6.safetensors\n⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet\n⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.55s, size: 689.12MB\n✅ control_v1p_sd15_qrcode_monster.safetensors\n⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models\n⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.16s, size: 1729.05MB\n✅ AnimateLCM_sd15_t2v.ckpt\n====================================\nRunning workflow\ngot prompt\nExecuting node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple\nmodel_type EPS\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nclip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']\nloaded straight to GPU\nRequested to load BaseModel\nLoading 1 new model\nExecuting node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2\nExecuting node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel\nExecuting node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic\nExecuting node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple\nExecuting node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions\nExecuting node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling\nExecuting node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader\n\u001b[33mINFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\u001b[0m\n\u001b[33mINFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors\u001b[0m\nExecuting node 142, title: Load Image, class type: LoadImage\nExecuting node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nRequested to load CLIPVisionModelProjection\nLoading 1 new model\nExecuting node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 135, title: Load Image, class type: LoadImage\nExecuting node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 680, title: Load Image, class type: LoadImage\nExecuting node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nExecuting node 683, title: Load Image, class type: LoadImage\nExecuting node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 752, title: Load Image, class type: LoadImage\nExecuting node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nRequested to load SD1ClipModel\nLoading 1 new model\nExecuting node 565, title: Positive, class type: CLIPTextEncode\nExecuting node 566, title: Negative, class type: CLIPTextEncode\nExecuting node 134, title: Empty Latent Image, class type: EmptyLatentImage\nExecuting node 80, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\nRequested to load BaseModel\nRequested to load AnimateDiffModel\nLoading 2 new models\n  0%|          | 0/8 [00:00<?, ?it/s]\n 12%|█▎        | 1/8 [00:02<00:14,  2.09s/it]\n 25%|██▌       | 2/8 [00:04<00:11,  1.99s/it]\n 38%|███▊      | 3/8 [00:05<00:09,  1.96s/it]\n 50%|█████     | 4/8 [00:07<00:07,  1.95s/it]\n 62%|██████▎   | 5/8 [00:09<00:05,  1.94s/it]\n 75%|███████▌  | 6/8 [00:11<00:03,  1.94s/it]\n 88%|████████▊ | 7/8 [00:13<00:01,  1.93s/it]\n100%|██████████| 8/8 [00:15<00:00,  1.93s/it]\n100%|██████████| 8/8 [00:15<00:00,  1.95s/it]\nExecuting node 85, title: Load VAE, class type: VAELoader\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nRequested to load AutoencoderKL\nLoading 1 new model\nExecuting node 84, title: VAE Decode, class type: VAEDecode\nExecuting node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 203, title: Upscale Image By, class type: ImageScaleBy\nExecuting node 204, title: VAE Encode, class type: VAEEncode\nExecuting node 198, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\n  0%|          | 0/8 [00:00<?, ?it/s]\n 12%|█▎        | 1/8 [00:03<00:27,  3.88s/it]\n 25%|██▌       | 2/8 [00:07<00:23,  3.88s/it]\n 38%|███▊      | 3/8 [00:11<00:19,  3.89s/it]\n 50%|█████     | 4/8 [00:15<00:15,  3.89s/it]\n 62%|██████▎   | 5/8 [00:19<00:11,  3.95s/it]\n 75%|███████▌  | 6/8 [00:23<00:07,  3.94s/it]\n 88%|████████▊ | 7/8 [00:27<00:03,  3.93s/it]\n100%|██████████| 8/8 [00:31<00:00,  3.93s/it]\n100%|██████████| 8/8 [00:31<00:00,  3.92s/it]\nExecuting node 201, title: VAE Decode, class type: VAEDecode\nExecuting node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 270, title: Load Upscale Model, class type: UpscaleModelLoader\nExecuting node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel\nExecuting node 279, title: Upscale Image, class type: ImageScale\nExecuting node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 770, title: FILM VFI, class type: FILM VFI\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Final clearing cache...\nComfy-VFI: Done cache clearing\nExecuting node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 250.36 seconds\noutputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\npreview_00001.mp4\nupscaled_00001.mp4\nupscaled_model_00001.mp4\ninterpolated_00001.mp4",
  "metrics": {
    "predict_time": 265.163699,
    "total_time": 287.246754
  },
  "output": [
    "https://replicate.delivery/pbxt/K9dkJNIWZC72LRQeSGRxexJgSSkUy8Ib7vt2s0goVjp3cttSA/preview_00001.mp4",
    "https://replicate.delivery/pbxt/ff9G37f5SNWF6o7a56G5hrS1YQLwcBYJRi123cjvw3cw5ablA/upscaled_00001.mp4",
    "https://replicate.delivery/pbxt/RqySCgbh21KvGVkmpCvImkFNJALWAAKPqZOeOPuiELucu2WJA/upscaled_model_00001.mp4",
    "https://replicate.delivery/pbxt/5VWDfWRTrF0nMivSriscwK3SSdZl1oy7amOshxofepdy5ablA/interpolated_00001.mp4"
  ],
  "started_at": "2024-04-24T15:24:33.027055Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/zq1ppbr981rgg0cf22trg7ex1r",
    "cancel": "https://api.replicate.com/v1/predictions/zq1ppbr981rgg0cf22trg7ex1r/cancel"
  },
  "version": "355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf"
}

Generated in

4 minutes 25 seconds

Tweak it Share Report

Random seed set to: 665779694
Checking inputs
✅ /tmp/inputs/2.png
✅ /tmp/inputs/1.png
✅ /tmp/inputs/3.png
✅ /tmp/inputs/4.png
✅ /tmp/inputs/circles.mp4
✅ /tmp/inputs/style.png
====================================
Checking weights
Including weights for IPAdapter preset: PLUS (high strength)
⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.60s, size: 808.26MB
✅ ip-adapter-plus_sdxl_vit-h.safetensors
⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models
⌛️ Downloaded RealESRGAN_x4.pth in 0.15s, size: 63.94MB
✅ RealESRGAN_x4.pth
⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae
⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.35s, size: 319.14MB
✅ vae-ft-mse-840000-ema-pruned.safetensors
⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision
⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 1.94s, size: 2411.24MB
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.18s, size: 93.63MB
✅ ip-adapter-plus_sd15.safetensors
⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film
⌛️ Downloaded film_net_fp32.pt in 0.25s, size: 131.53MB
✅ film_net_fp32.pt
⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras
⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.23s, size: 128.39MB
✅ AnimateLCM_sd15_t2v_lora.safetensors
⏳ Downloading toonyou_beta6.safetensors to ComfyUI/models/checkpoints
⌛️ Downloaded toonyou_beta6.safetensors in 1.36s, size: 2193.39MB
✅ toonyou_beta6.safetensors
⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet
⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.55s, size: 689.12MB
✅ control_v1p_sd15_qrcode_monster.safetensors
⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models
⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.16s, size: 1729.05MB
✅ AnimateLCM_sd15_t2v.ckpt
====================================
Running workflow
got prompt
Executing node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
Executing node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly
[AnimateDiffEvo] - INFO - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2
Executing node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel
Executing node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic
Executing node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple
Executing node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions
Executing node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling
Executing node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader
INFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
INFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors
Executing node 142, title: Load Image, class type: LoadImage
Executing node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Requested to load CLIPVisionModelProjection
Loading 1 new model
Executing node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 135, title: Load Image, class type: LoadImage
Executing node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 680, title: Load Image, class type: LoadImage
Executing node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Executing node 683, title: Load Image, class type: LoadImage
Executing node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 752, title: Load Image, class type: LoadImage
Executing node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Requested to load SD1ClipModel
Loading 1 new model
Executing node 565, title: Positive, class type: CLIPTextEncode
Executing node 566, title: Negative, class type: CLIPTextEncode
Executing node 134, title: Empty Latent Image, class type: EmptyLatentImage
Executing node 80, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
Requested to load BaseModel
Requested to load AnimateDiffModel
Loading 2 new models
  0%|          | 0/8 [00:00<?, ?it/s]
 12%|█▎        | 1/8 [00:02<00:14,  2.09s/it]
 25%|██▌       | 2/8 [00:04<00:11,  1.99s/it]
 38%|███▊      | 3/8 [00:05<00:09,  1.96s/it]
 50%|█████     | 4/8 [00:07<00:07,  1.95s/it]
 62%|██████▎   | 5/8 [00:09<00:05,  1.94s/it]
 75%|███████▌  | 6/8 [00:11<00:03,  1.94s/it]
 88%|████████▊ | 7/8 [00:13<00:01,  1.93s/it]
100%|██████████| 8/8 [00:15<00:00,  1.93s/it]
100%|██████████| 8/8 [00:15<00:00,  1.95s/it]
Executing node 85, title: Load VAE, class type: VAELoader
Using pytorch attention in VAE
Using pytorch attention in VAE
Requested to load AutoencoderKL
Loading 1 new model
Executing node 84, title: VAE Decode, class type: VAEDecode
Executing node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 203, title: Upscale Image By, class type: ImageScaleBy
Executing node 204, title: VAE Encode, class type: VAEEncode
Executing node 198, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
  0%|          | 0/8 [00:00<?, ?it/s]
 12%|█▎        | 1/8 [00:03<00:27,  3.88s/it]
 25%|██▌       | 2/8 [00:07<00:23,  3.88s/it]
 38%|███▊      | 3/8 [00:11<00:19,  3.89s/it]
 50%|█████     | 4/8 [00:15<00:15,  3.89s/it]
 62%|██████▎   | 5/8 [00:19<00:11,  3.95s/it]
 75%|███████▌  | 6/8 [00:23<00:07,  3.94s/it]
 88%|████████▊ | 7/8 [00:27<00:03,  3.93s/it]
100%|██████████| 8/8 [00:31<00:00,  3.93s/it]
100%|██████████| 8/8 [00:31<00:00,  3.92s/it]
Executing node 201, title: VAE Decode, class type: VAEDecode
Executing node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 270, title: Load Upscale Model, class type: UpscaleModelLoader
Executing node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel
Executing node 279, title: Upscale Image, class type: ImageScale
Executing node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 770, title: FILM VFI, class type: FILM VFI
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Final clearing cache...
Comfy-VFI: Done cache clearing
Executing node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 250.36 seconds
outputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
preview_00001.mp4
upscaled_00001.mp4
upscaled_model_00001.mp4
interpolated_00001.mp4

Prediction

fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7

Model

fofr/video-morpher:e70e9750

0j54ep0shhrgg0cf26gbreemam

Status

Succeeded

Source

Web

Hardware

A40 (Large)

Total duration

5m 33s

Created

about 1 year ago by @fofr

Input

mode: upscaled-and-interpolated
prompt: bright, vibrant, high contrast
checkpoint: illustrated
style_image
aspect_ratio: 3:4
style_strength: 1
use_controlnet
negative_prompt: dark, gloomy
subject_image_1
subject_image_2
subject_image_3
subject_image_4

{
  "mode": "upscaled-and-interpolated",
  "prompt": "bright, vibrant, high contrast",
  "checkpoint": "illustrated",
  "style_image": "https://replicate.delivery/pbxt/Knst8xuE7YWv1wOj7AzV6JMZbyBx4yqlMw0YgMBqwI13ll5U/tshirt_01829_.png",
  "aspect_ratio": "3:4",
  "style_strength": 1,
  "use_controlnet": true,
  "negative_prompt": "dark, gloomy",
  "subject_image_1": "https://replicate.delivery/pbxt/Knst8Jmhr7nUhLLsEQ5y6shXtzUNKApvXsHmpsfmvQi3ak9t/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
  "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
  "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
  "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
  {
    input: {
      mode: "upscaled-and-interpolated",
      prompt: "bright, vibrant, high contrast",
      checkpoint: "illustrated",
      style_image: "https://replicate.delivery/pbxt/Knst8xuE7YWv1wOj7AzV6JMZbyBx4yqlMw0YgMBqwI13ll5U/tshirt_01829_.png",
      aspect_ratio: "3:4",
      style_strength: 1,
      use_controlnet: true,
      negative_prompt: "dark, gloomy",
      subject_image_1: "https://replicate.delivery/pbxt/Knst8Jmhr7nUhLLsEQ5y6shXtzUNKApvXsHmpsfmvQi3ak9t/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
      subject_image_2: "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
      subject_image_3: "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
      subject_image_4: "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
    input={
        "mode": "upscaled-and-interpolated",
        "prompt": "bright, vibrant, high contrast",
        "checkpoint": "illustrated",
        "style_image": "https://replicate.delivery/pbxt/Knst8xuE7YWv1wOj7AzV6JMZbyBx4yqlMw0YgMBqwI13ll5U/tshirt_01829_.png",
        "aspect_ratio": "3:4",
        "style_strength": 1,
        "use_controlnet": True,
        "negative_prompt": "dark, gloomy",
        "subject_image_1": "https://replicate.delivery/pbxt/Knst8Jmhr7nUhLLsEQ5y6shXtzUNKApvXsHmpsfmvQi3ak9t/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
        "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
        "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
        "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
    "input": {
      "mode": "upscaled-and-interpolated",
      "prompt": "bright, vibrant, high contrast",
      "checkpoint": "illustrated",
      "style_image": "https://replicate.delivery/pbxt/Knst8xuE7YWv1wOj7AzV6JMZbyBx4yqlMw0YgMBqwI13ll5U/tshirt_01829_.png",
      "aspect_ratio": "3:4",
      "style_strength": 1,
      "use_controlnet": true,
      "negative_prompt": "dark, gloomy",
      "subject_image_1": "https://replicate.delivery/pbxt/Knst8Jmhr7nUhLLsEQ5y6shXtzUNKApvXsHmpsfmvQi3ak9t/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
      "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
      "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
      "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-04-24T19:46:28.871671Z",
  "created_at": "2024-04-24T19:40:56.076000Z",
  "data_removed": false,
  "error": null,
  "id": "0j54ep0shhrgg0cf26gbreemam",
  "input": {
    "mode": "upscaled-and-interpolated",
    "prompt": "bright, vibrant, high contrast",
    "checkpoint": "illustrated",
    "style_image": "https://replicate.delivery/pbxt/Knst8xuE7YWv1wOj7AzV6JMZbyBx4yqlMw0YgMBqwI13ll5U/tshirt_01829_.png",
    "aspect_ratio": "3:4",
    "style_strength": 1,
    "use_controlnet": true,
    "negative_prompt": "dark, gloomy",
    "subject_image_1": "https://replicate.delivery/pbxt/Knst8Jmhr7nUhLLsEQ5y6shXtzUNKApvXsHmpsfmvQi3ak9t/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
    "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
    "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
    "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
  },
  "logs": "Random seed set to: 3365804795\nChecking inputs\n✅ /tmp/inputs/2.png\n✅ /tmp/inputs/1.png\n✅ /tmp/inputs/3.png\n✅ /tmp/inputs/4.png\n✅ /tmp/inputs/circles.mp4\n✅ /tmp/inputs/style.png\n====================================\nChecking weights\nIncluding weights for IPAdapter preset: PLUS (high strength)\n⏳ Downloading dreamshaper_8.safetensors to ComfyUI/models/checkpoints\n⌛️ Downloaded dreamshaper_8.safetensors in 1.41s, size: 2033.83MB\n✅ dreamshaper_8.safetensors\n⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film\n⌛️ Downloaded film_net_fp32.pt in 0.28s, size: 131.53MB\n✅ film_net_fp32.pt\n⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models\n⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.43s, size: 1729.05MB\n✅ AnimateLCM_sd15_t2v.ckpt\n⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.25s, size: 93.63MB\n✅ ip-adapter-plus_sd15.safetensors\n⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.64s, size: 808.26MB\n✅ ip-adapter-plus_sdxl_vit-h.safetensors\n⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae\n⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.41s, size: 319.14MB\n✅ vae-ft-mse-840000-ema-pruned.safetensors\n⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet\n⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.61s, size: 689.12MB\n✅ control_v1p_sd15_qrcode_monster.safetensors\n⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras\n⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.25s, size: 128.39MB\n✅ AnimateLCM_sd15_t2v_lora.safetensors\n⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models\n⌛️ Downloaded RealESRGAN_x4.pth in 0.24s, size: 63.94MB\n✅ RealESRGAN_x4.pth\n⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision\n⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 1.64s, size: 2411.24MB\n✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\n====================================\nRunning workflow\ngot prompt\nExecuting node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple\nmodel_type EPS\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nclip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']\nloaded straight to GPU\nRequested to load BaseModel\nLoading 1 new model\nExecuting node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2\nExecuting node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel\nExecuting node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic\nExecuting node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple\nExecuting node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions\nExecuting node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling\nExecuting node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader\n\u001b[33mINFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\u001b[0m\n\u001b[33mINFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors\u001b[0m\nExecuting node 142, title: Load Image, class type: LoadImage\nExecuting node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nRequested to load CLIPVisionModelProjection\nLoading 1 new model\nExecuting node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 135, title: Load Image, class type: LoadImage\nExecuting node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 680, title: Load Image, class type: LoadImage\nExecuting node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 683, title: Load Image, class type: LoadImage\nExecuting node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nExecuting node 752, title: Load Image, class type: LoadImage\nExecuting node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nExecuting node 565, title: Positive, class type: CLIPTextEncode\nRequested to load SD1ClipModel\nLoading 1 new model\nExecuting node 566, title: Negative, class type: CLIPTextEncode\nExecuting node 127, title: Load Advanced ControlNet Model 🛂🅐🅒🅝, class type: ControlNetLoaderAdvanced\nExecuting node 134, title: Empty Latent Image, class type: EmptyLatentImage\nExecuting node 569, title: 🔧 Batch Count, class type: BatchCount+\nExecuting node 746, title: Load Video (Upload) 🎥🅥🅗🅢, class type: VHS_LoadVideo\nExecuting node 461, title: 🔧 Simple Math, class type: SimpleMath+\nExecuting node 454, title: RepeatImageBatch, class type: RepeatImageBatch\nExecuting node 458, title: Split Image Batch 🎥🅥🅗🅢, class type: VHS_SplitImages\nExecuting node 125, title: Apply Advanced ControlNet 🛂🅐🅒🅝, class type: ACN_AdvancedControlNetApply\nExecuting node 80, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\nRequested to load AnimateDiffModel\nRequested to load BaseModel\nRequested to load ControlNet\nLoading 3 new models\n  0%|          | 0/11 [00:00<?, ?it/s]\n  9%|▉         | 1/11 [00:04<00:49,  4.91s/it]\n 18%|█▊        | 2/11 [00:07<00:29,  3.28s/it]\n 27%|██▋       | 3/11 [00:09<00:22,  2.86s/it]\n 36%|███▋      | 4/11 [00:11<00:18,  2.67s/it]\n 45%|████▌     | 5/11 [00:13<00:14,  2.39s/it]\n 55%|█████▍    | 6/11 [00:15<00:11,  2.22s/it]\n 64%|██████▎   | 7/11 [00:17<00:08,  2.12s/it]\n 73%|███████▎  | 8/11 [00:19<00:06,  2.05s/it]\n 82%|████████▏ | 9/11 [00:21<00:03,  2.00s/it]\n 91%|█████████ | 10/11 [00:23<00:01,  1.97s/it]\n100%|██████████| 11/11 [00:25<00:00,  1.95s/it]\n100%|██████████| 11/11 [00:25<00:00,  2.28s/it]\nExecuting node 85, title: Load VAE, class type: VAELoader\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nRequested to load AutoencoderKL\nLoading 1 new model\nExecuting node 84, title: VAE Decode, class type: VAEDecode\nExecuting node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 203, title: Upscale Image By, class type: ImageScaleBy\nExecuting node 204, title: VAE Encode, class type: VAEEncode\nExecuting node 198, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\n  0%|          | 0/11 [00:00<?, ?it/s]\n  9%|▉         | 1/11 [00:03<00:38,  3.84s/it]\n 18%|█▊        | 2/11 [00:07<00:34,  3.83s/it]\n 27%|██▋       | 3/11 [00:11<00:30,  3.83s/it]\n 36%|███▋      | 4/11 [00:15<00:26,  3.83s/it]\n 45%|████▌     | 5/11 [00:19<00:22,  3.83s/it]\n 55%|█████▍    | 6/11 [00:22<00:19,  3.83s/it]\n 64%|██████▎   | 7/11 [00:26<00:15,  3.84s/it]\n 73%|███████▎  | 8/11 [00:30<00:11,  3.84s/it]\n 82%|████████▏ | 9/11 [00:34<00:07,  3.84s/it]\n 91%|█████████ | 10/11 [00:38<00:03,  3.84s/it]\n100%|██████████| 11/11 [00:42<00:00,  3.84s/it]\n100%|██████████| 11/11 [00:42<00:00,  3.84s/it]\nExecuting node 201, title: VAE Decode, class type: VAEDecode\nExecuting node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 270, title: Load Upscale Model, class type: UpscaleModelLoader\nExecuting node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel\nExecuting node 279, title: Upscale Image, class type: ImageScale\nExecuting node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 770, title: FILM VFI, class type: FILM VFI\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Final clearing cache...\nComfy-VFI: Done cache clearing\nExecuting node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 277.90 seconds\noutputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\npreview_00001.mp4\nupscaled_00001.mp4\nupscaled_model_00001.mp4\ninterpolated_00001.mp4",
  "metrics": {
    "predict_time": 291.526079,
    "total_time": 332.795671
  },
  "output": [
    "https://replicate.delivery/pbxt/UEnKdoDxVDbmFluBmlyNBdBfZcaSXdsK1FmPuvcSVRhIn4WJA/preview_00001.mp4",
    "https://replicate.delivery/pbxt/jks6HpcRn5rfTil8aWs4hQvRet618blrRURZLYtukKWTOxtSA/upscaled_00001.mp4",
    "https://replicate.delivery/pbxt/Hfb7RalBgR3qK6XmxlWCL1G0Ko2fHkuiRRF3idzdblTTOxtSA/upscaled_model_00001.mp4",
    "https://replicate.delivery/pbxt/Emvw5LrbvP4fKyA9yJg194pHvIWfMkHybue87hUr1HYociblA/interpolated_00001.mp4"
  ],
  "started_at": "2024-04-24T19:41:37.345592Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/0j54ep0shhrgg0cf26gbreemam",
    "cancel": "https://api.replicate.com/v1/predictions/0j54ep0shhrgg0cf26gbreemam/cancel"
  },
  "version": "e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7"
}

Generated in

4 minutes 52 seconds

Tweak it Share Report

Random seed set to: 3365804795
Checking inputs
✅ /tmp/inputs/2.png
✅ /tmp/inputs/1.png
✅ /tmp/inputs/3.png
✅ /tmp/inputs/4.png
✅ /tmp/inputs/circles.mp4
✅ /tmp/inputs/style.png
====================================
Checking weights
Including weights for IPAdapter preset: PLUS (high strength)
⏳ Downloading dreamshaper_8.safetensors to ComfyUI/models/checkpoints
⌛️ Downloaded dreamshaper_8.safetensors in 1.41s, size: 2033.83MB
✅ dreamshaper_8.safetensors
⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film
⌛️ Downloaded film_net_fp32.pt in 0.28s, size: 131.53MB
✅ film_net_fp32.pt
⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models
⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.43s, size: 1729.05MB
✅ AnimateLCM_sd15_t2v.ckpt
⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.25s, size: 93.63MB
✅ ip-adapter-plus_sd15.safetensors
⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.64s, size: 808.26MB
✅ ip-adapter-plus_sdxl_vit-h.safetensors
⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae
⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.41s, size: 319.14MB
✅ vae-ft-mse-840000-ema-pruned.safetensors
⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet
⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.61s, size: 689.12MB
✅ control_v1p_sd15_qrcode_monster.safetensors
⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras
⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.25s, size: 128.39MB
✅ AnimateLCM_sd15_t2v_lora.safetensors
⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models
⌛️ Downloaded RealESRGAN_x4.pth in 0.24s, size: 63.94MB
✅ RealESRGAN_x4.pth
⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision
⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 1.64s, size: 2411.24MB
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
====================================
Running workflow
got prompt
Executing node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
Executing node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly
[AnimateDiffEvo] - INFO - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2
Executing node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel
Executing node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic
Executing node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple
Executing node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions
Executing node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling
Executing node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader
INFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
INFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors
Executing node 142, title: Load Image, class type: LoadImage
Executing node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Requested to load CLIPVisionModelProjection
Loading 1 new model
Executing node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 135, title: Load Image, class type: LoadImage
Executing node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 680, title: Load Image, class type: LoadImage
Executing node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 683, title: Load Image, class type: LoadImage
Executing node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Executing node 752, title: Load Image, class type: LoadImage
Executing node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Executing node 565, title: Positive, class type: CLIPTextEncode
Requested to load SD1ClipModel
Loading 1 new model
Executing node 566, title: Negative, class type: CLIPTextEncode
Executing node 127, title: Load Advanced ControlNet Model 🛂🅐🅒🅝, class type: ControlNetLoaderAdvanced
Executing node 134, title: Empty Latent Image, class type: EmptyLatentImage
Executing node 569, title: 🔧 Batch Count, class type: BatchCount+
Executing node 746, title: Load Video (Upload) 🎥🅥🅗🅢, class type: VHS_LoadVideo
Executing node 461, title: 🔧 Simple Math, class type: SimpleMath+
Executing node 454, title: RepeatImageBatch, class type: RepeatImageBatch
Executing node 458, title: Split Image Batch 🎥🅥🅗🅢, class type: VHS_SplitImages
Executing node 125, title: Apply Advanced ControlNet 🛂🅐🅒🅝, class type: ACN_AdvancedControlNetApply
Executing node 80, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
Requested to load AnimateDiffModel
Requested to load BaseModel
Requested to load ControlNet
Loading 3 new models
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:04<00:49,  4.91s/it]
 18%|█▊        | 2/11 [00:07<00:29,  3.28s/it]
 27%|██▋       | 3/11 [00:09<00:22,  2.86s/it]
 36%|███▋      | 4/11 [00:11<00:18,  2.67s/it]
 45%|████▌     | 5/11 [00:13<00:14,  2.39s/it]
 55%|█████▍    | 6/11 [00:15<00:11,  2.22s/it]
 64%|██████▎   | 7/11 [00:17<00:08,  2.12s/it]
 73%|███████▎  | 8/11 [00:19<00:06,  2.05s/it]
 82%|████████▏ | 9/11 [00:21<00:03,  2.00s/it]
 91%|█████████ | 10/11 [00:23<00:01,  1.97s/it]
100%|██████████| 11/11 [00:25<00:00,  1.95s/it]
100%|██████████| 11/11 [00:25<00:00,  2.28s/it]
Executing node 85, title: Load VAE, class type: VAELoader
Using pytorch attention in VAE
Using pytorch attention in VAE
Requested to load AutoencoderKL
Loading 1 new model
Executing node 84, title: VAE Decode, class type: VAEDecode
Executing node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 203, title: Upscale Image By, class type: ImageScaleBy
Executing node 204, title: VAE Encode, class type: VAEEncode
Executing node 198, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:03<00:38,  3.84s/it]
 18%|█▊        | 2/11 [00:07<00:34,  3.83s/it]
 27%|██▋       | 3/11 [00:11<00:30,  3.83s/it]
 36%|███▋      | 4/11 [00:15<00:26,  3.83s/it]
 45%|████▌     | 5/11 [00:19<00:22,  3.83s/it]
 55%|█████▍    | 6/11 [00:22<00:19,  3.83s/it]
 64%|██████▎   | 7/11 [00:26<00:15,  3.84s/it]
 73%|███████▎  | 8/11 [00:30<00:11,  3.84s/it]
 82%|████████▏ | 9/11 [00:34<00:07,  3.84s/it]
 91%|█████████ | 10/11 [00:38<00:03,  3.84s/it]
100%|██████████| 11/11 [00:42<00:00,  3.84s/it]
100%|██████████| 11/11 [00:42<00:00,  3.84s/it]
Executing node 201, title: VAE Decode, class type: VAEDecode
Executing node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 270, title: Load Upscale Model, class type: UpscaleModelLoader
Executing node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel
Executing node 279, title: Upscale Image, class type: ImageScale
Executing node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 770, title: FILM VFI, class type: FILM VFI
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Final clearing cache...
Comfy-VFI: Done cache clearing
Executing node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 277.90 seconds
outputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
preview_00001.mp4
upscaled_00001.mp4
upscaled_model_00001.mp4
interpolated_00001.mp4

Prediction

fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7

Model

fofr/video-morpher:e70e9750

9esbb1jcmxrgg0cf26rsptyyf8

Status

Succeeded

Source

Web

Hardware

A40 (Large)

Total duration

5m 27s

Created

about 1 year ago by @fofr

Input

mode: upscaled-and-interpolated
prompt: bright, vibrant, high contrast
checkpoint: 3D
style_image
aspect_ratio: 3:4
style_strength: 0.5
use_controlnet
negative_prompt: dark, gloomy
subject_image_1
subject_image_2
subject_image_3
subject_image_4

{
  "mode": "upscaled-and-interpolated",
  "prompt": "bright, vibrant, high contrast",
  "checkpoint": "3D",
  "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
  "aspect_ratio": "3:4",
  "style_strength": 0.5,
  "use_controlnet": true,
  "negative_prompt": "dark, gloomy",
  "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
  "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
  "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
  "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
  {
    input: {
      mode: "upscaled-and-interpolated",
      prompt: "bright, vibrant, high contrast",
      checkpoint: "3D",
      style_image: "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
      aspect_ratio: "3:4",
      style_strength: 0.5,
      use_controlnet: true,
      negative_prompt: "dark, gloomy",
      subject_image_1: "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
      subject_image_2: "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
      subject_image_3: "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
      subject_image_4: "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
    input={
        "mode": "upscaled-and-interpolated",
        "prompt": "bright, vibrant, high contrast",
        "checkpoint": "3D",
        "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
        "aspect_ratio": "3:4",
        "style_strength": 0.5,
        "use_controlnet": True,
        "negative_prompt": "dark, gloomy",
        "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
        "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
        "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
        "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
    "input": {
      "mode": "upscaled-and-interpolated",
      "prompt": "bright, vibrant, high contrast",
      "checkpoint": "3D",
      "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
      "aspect_ratio": "3:4",
      "style_strength": 0.5,
      "use_controlnet": true,
      "negative_prompt": "dark, gloomy",
      "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
      "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
      "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
      "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-04-24T20:05:10.327727Z",
  "created_at": "2024-04-24T19:59:43.271000Z",
  "data_removed": false,
  "error": null,
  "id": "9esbb1jcmxrgg0cf26rsptyyf8",
  "input": {
    "mode": "upscaled-and-interpolated",
    "prompt": "bright, vibrant, high contrast",
    "checkpoint": "3D",
    "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
    "aspect_ratio": "3:4",
    "style_strength": 0.5,
    "use_controlnet": true,
    "negative_prompt": "dark, gloomy",
    "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
    "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
    "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
    "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
  },
  "logs": "Random seed set to: 2661073672\nChecking inputs\n✅ /tmp/inputs/2.png\n✅ /tmp/inputs/1.png\n✅ /tmp/inputs/3.png\n✅ /tmp/inputs/4.png\n✅ /tmp/inputs/circles.mp4\n✅ /tmp/inputs/style.png\n====================================\nChecking weights\nIncluding weights for IPAdapter preset: PLUS (high strength)\n⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae\n⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.41s, size: 319.14MB\n✅ vae-ft-mse-840000-ema-pruned.safetensors\n⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models\n⌛️ Downloaded RealESRGAN_x4.pth in 0.18s, size: 63.94MB\n✅ RealESRGAN_x4.pth\n⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models\n⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.20s, size: 1729.05MB\n✅ AnimateLCM_sd15_t2v.ckpt\n⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras\n⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.92s, size: 128.39MB\n✅ AnimateLCM_sd15_t2v_lora.safetensors\n⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.25s, size: 93.63MB\n✅ ip-adapter-plus_sd15.safetensors\n⏳ Downloading rcnzCartoon3d_v20.safetensors to ComfyUI/models/checkpoints\n⌛️ Downloaded rcnzCartoon3d_v20.safetensors in 2.04s, size: 2033.83MB\n✅ rcnzCartoon3d_v20.safetensors\n⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.67s, size: 808.26MB\n✅ ip-adapter-plus_sdxl_vit-h.safetensors\n⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision\n⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 11.24s, size: 2411.24MB\n✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\n⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film\n⌛️ Downloaded film_net_fp32.pt in 1.16s, size: 131.53MB\n✅ film_net_fp32.pt\n⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet\n⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.69s, size: 689.12MB\n✅ control_v1p_sd15_qrcode_monster.safetensors\n====================================\nRunning workflow\ngot prompt\nExecuting node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple\nmodel_type EPS\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nclip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']\nloaded straight to GPU\nRequested to load BaseModel\nLoading 1 new model\nExecuting node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly\nExecuting node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2\nExecuting node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic\nExecuting node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple\nExecuting node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions\nExecuting node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling\nExecuting node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader\n\u001b[33mINFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\u001b[0m\n\u001b[33mINFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors\u001b[0m\nExecuting node 142, title: Load Image, class type: LoadImage\nExecuting node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nRequested to load CLIPVisionModelProjection\nLoading 1 new model\nExecuting node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 135, title: Load Image, class type: LoadImage\nExecuting node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 680, title: Load Image, class type: LoadImage\nExecuting node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 683, title: Load Image, class type: LoadImage\nExecuting node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 752, title: Load Image, class type: LoadImage\nExecuting node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nExecuting node 565, title: Positive, class type: CLIPTextEncode\nRequested to load SD1ClipModel\nLoading 1 new model\nExecuting node 566, title: Negative, class type: CLIPTextEncode\nExecuting node 127, title: Load Advanced ControlNet Model 🛂🅐🅒🅝, class type: ControlNetLoaderAdvanced\nExecuting node 134, title: Empty Latent Image, class type: EmptyLatentImage\nExecuting node 569, title: 🔧 Batch Count, class type: BatchCount+\nExecuting node 746, title: Load Video (Upload) 🎥🅥🅗🅢, class type: VHS_LoadVideo\nExecuting node 461, title: 🔧 Simple Math, class type: SimpleMath+\nExecuting node 454, title: RepeatImageBatch, class type: RepeatImageBatch\nExecuting node 458, title: Split Image Batch 🎥🅥🅗🅢, class type: VHS_SplitImages\nExecuting node 125, title: Apply Advanced ControlNet 🛂🅐🅒🅝, class type: ACN_AdvancedControlNetApply\nExecuting node 80, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\nRequested to load AnimateDiffModel\nRequested to load BaseModel\nRequested to load ControlNet\nLoading 3 new models\n  0%|          | 0/11 [00:00<?, ?it/s]\n  9%|▉         | 1/11 [00:03<00:30,  3.10s/it]\n 18%|█▊        | 2/11 [00:05<00:22,  2.50s/it]\n 27%|██▋       | 3/11 [00:07<00:19,  2.43s/it]\n 36%|███▋      | 4/11 [00:09<00:16,  2.40s/it]\n 45%|████▌     | 5/11 [00:11<00:13,  2.22s/it]\n 55%|█████▍    | 6/11 [00:13<00:10,  2.11s/it]\n 64%|██████▎   | 7/11 [00:15<00:08,  2.04s/it]\n 73%|███████▎  | 8/11 [00:17<00:05,  1.99s/it]\n 82%|████████▏ | 9/11 [00:19<00:03,  1.96s/it]\n 91%|█████████ | 10/11 [00:21<00:01,  1.94s/it]\n100%|██████████| 11/11 [00:23<00:00,  1.93s/it]\n100%|██████████| 11/11 [00:23<00:00,  2.11s/it]\nExecuting node 85, title: Load VAE, class type: VAELoader\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nRequested to load AutoencoderKL\nLoading 1 new model\nExecuting node 84, title: VAE Decode, class type: VAEDecode\nExecuting node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 203, title: Upscale Image By, class type: ImageScaleBy\nExecuting node 204, title: VAE Encode, class type: VAEEncode\nExecuting node 198, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\n  0%|          | 0/11 [00:00<?, ?it/s]\n  9%|▉         | 1/11 [00:03<00:38,  3.86s/it]\n 18%|█▊        | 2/11 [00:07<00:34,  3.85s/it]\n 27%|██▋       | 3/11 [00:11<00:30,  3.85s/it]\n 36%|███▋      | 4/11 [00:15<00:27,  3.86s/it]\n 45%|████▌     | 5/11 [00:19<00:23,  3.86s/it]\n 55%|█████▍    | 6/11 [00:23<00:19,  3.86s/it]\n 64%|██████▎   | 7/11 [00:27<00:15,  3.86s/it]\n 73%|███████▎  | 8/11 [00:30<00:11,  3.86s/it]\n 82%|████████▏ | 9/11 [00:34<00:07,  3.87s/it]\n 91%|█████████ | 10/11 [00:38<00:03,  3.87s/it]\n100%|██████████| 11/11 [00:42<00:00,  3.87s/it]\n100%|██████████| 11/11 [00:42<00:00,  3.86s/it]\nExecuting node 201, title: VAE Decode, class type: VAEDecode\nExecuting node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 270, title: Load Upscale Model, class type: UpscaleModelLoader\nExecuting node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel\nExecuting node 279, title: Upscale Image, class type: ImageScale\nExecuting node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 770, title: FILM VFI, class type: FILM VFI\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Final clearing cache...\nComfy-VFI: Done cache clearing\nExecuting node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 270.81 seconds\noutputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\npreview_00001.mp4\nupscaled_00001.mp4\nupscaled_model_00001.mp4\ninterpolated_00001.mp4",
  "metrics": {
    "predict_time": 301.418943,
    "total_time": 327.056727
  },
  "output": [
    "https://replicate.delivery/pbxt/GPTWM5fboOShBKlFRQtMapKDoUBohf9qFxrySPt7AF4zfiblA/preview_00001.mp4",
    "https://replicate.delivery/pbxt/GZV1nIJXsIJLMpIg1w6XHWxSfN3qm36kEkTIa2A1F6L6v4WJA/upscaled_00001.mp4",
    "https://replicate.delivery/pbxt/N2u5EY0jm4ovC5GAztQaUrvL4KiyJTKzr1g5Ia2nKQe6v4WJA/upscaled_model_00001.mp4",
    "https://replicate.delivery/pbxt/w3xiWkOFJEb7KRmQq5itovpqr1FGVZ033Nj6YBe1jiz6v4WJA/interpolated_00001.mp4"
  ],
  "started_at": "2024-04-24T20:00:08.908784Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/9esbb1jcmxrgg0cf26rsptyyf8",
    "cancel": "https://api.replicate.com/v1/predictions/9esbb1jcmxrgg0cf26rsptyyf8/cancel"
  },
  "version": "e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7"
}

Generated in

5 minutes

Tweak it Share Report

Random seed set to: 2661073672
Checking inputs
✅ /tmp/inputs/2.png
✅ /tmp/inputs/1.png
✅ /tmp/inputs/3.png
✅ /tmp/inputs/4.png
✅ /tmp/inputs/circles.mp4
✅ /tmp/inputs/style.png
====================================
Checking weights
Including weights for IPAdapter preset: PLUS (high strength)
⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae
⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.41s, size: 319.14MB
✅ vae-ft-mse-840000-ema-pruned.safetensors
⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models
⌛️ Downloaded RealESRGAN_x4.pth in 0.18s, size: 63.94MB
✅ RealESRGAN_x4.pth
⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models
⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.20s, size: 1729.05MB
✅ AnimateLCM_sd15_t2v.ckpt
⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras
⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.92s, size: 128.39MB
✅ AnimateLCM_sd15_t2v_lora.safetensors
⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.25s, size: 93.63MB
✅ ip-adapter-plus_sd15.safetensors
⏳ Downloading rcnzCartoon3d_v20.safetensors to ComfyUI/models/checkpoints
⌛️ Downloaded rcnzCartoon3d_v20.safetensors in 2.04s, size: 2033.83MB
✅ rcnzCartoon3d_v20.safetensors
⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.67s, size: 808.26MB
✅ ip-adapter-plus_sdxl_vit-h.safetensors
⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision
⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 11.24s, size: 2411.24MB
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film
⌛️ Downloaded film_net_fp32.pt in 1.16s, size: 131.53MB
✅ film_net_fp32.pt
⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet
⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.69s, size: 689.12MB
✅ control_v1p_sd15_qrcode_monster.safetensors
====================================
Running workflow
got prompt
Executing node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
Executing node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly
Executing node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel[AnimateDiffEvo] - INFO - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2
Executing node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic
Executing node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple
Executing node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions
Executing node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling
Executing node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader
INFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
INFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors
Executing node 142, title: Load Image, class type: LoadImage
Executing node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Requested to load CLIPVisionModelProjection
Loading 1 new model
Executing node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 135, title: Load Image, class type: LoadImage
Executing node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 680, title: Load Image, class type: LoadImage
Executing node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 683, title: Load Image, class type: LoadImage
Executing node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 752, title: Load Image, class type: LoadImage
Executing node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Executing node 565, title: Positive, class type: CLIPTextEncode
Requested to load SD1ClipModel
Loading 1 new model
Executing node 566, title: Negative, class type: CLIPTextEncode
Executing node 127, title: Load Advanced ControlNet Model 🛂🅐🅒🅝, class type: ControlNetLoaderAdvanced
Executing node 134, title: Empty Latent Image, class type: EmptyLatentImage
Executing node 569, title: 🔧 Batch Count, class type: BatchCount+
Executing node 746, title: Load Video (Upload) 🎥🅥🅗🅢, class type: VHS_LoadVideo
Executing node 461, title: 🔧 Simple Math, class type: SimpleMath+
Executing node 454, title: RepeatImageBatch, class type: RepeatImageBatch
Executing node 458, title: Split Image Batch 🎥🅥🅗🅢, class type: VHS_SplitImages
Executing node 125, title: Apply Advanced ControlNet 🛂🅐🅒🅝, class type: ACN_AdvancedControlNetApply
Executing node 80, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
Requested to load AnimateDiffModel
Requested to load BaseModel
Requested to load ControlNet
Loading 3 new models
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:03<00:30,  3.10s/it]
 18%|█▊        | 2/11 [00:05<00:22,  2.50s/it]
 27%|██▋       | 3/11 [00:07<00:19,  2.43s/it]
 36%|███▋      | 4/11 [00:09<00:16,  2.40s/it]
 45%|████▌     | 5/11 [00:11<00:13,  2.22s/it]
 55%|█████▍    | 6/11 [00:13<00:10,  2.11s/it]
 64%|██████▎   | 7/11 [00:15<00:08,  2.04s/it]
 73%|███████▎  | 8/11 [00:17<00:05,  1.99s/it]
 82%|████████▏ | 9/11 [00:19<00:03,  1.96s/it]
 91%|█████████ | 10/11 [00:21<00:01,  1.94s/it]
100%|██████████| 11/11 [00:23<00:00,  1.93s/it]
100%|██████████| 11/11 [00:23<00:00,  2.11s/it]
Executing node 85, title: Load VAE, class type: VAELoader
Using pytorch attention in VAE
Using pytorch attention in VAE
Requested to load AutoencoderKL
Loading 1 new model
Executing node 84, title: VAE Decode, class type: VAEDecode
Executing node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 203, title: Upscale Image By, class type: ImageScaleBy
Executing node 204, title: VAE Encode, class type: VAEEncode
Executing node 198, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:03<00:38,  3.86s/it]
 18%|█▊        | 2/11 [00:07<00:34,  3.85s/it]
 27%|██▋       | 3/11 [00:11<00:30,  3.85s/it]
 36%|███▋      | 4/11 [00:15<00:27,  3.86s/it]
 45%|████▌     | 5/11 [00:19<00:23,  3.86s/it]
 55%|█████▍    | 6/11 [00:23<00:19,  3.86s/it]
 64%|██████▎   | 7/11 [00:27<00:15,  3.86s/it]
 73%|███████▎  | 8/11 [00:30<00:11,  3.86s/it]
 82%|████████▏ | 9/11 [00:34<00:07,  3.87s/it]
 91%|█████████ | 10/11 [00:38<00:03,  3.87s/it]
100%|██████████| 11/11 [00:42<00:00,  3.87s/it]
100%|██████████| 11/11 [00:42<00:00,  3.86s/it]
Executing node 201, title: VAE Decode, class type: VAEDecode
Executing node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 270, title: Load Upscale Model, class type: UpscaleModelLoader
Executing node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel
Executing node 279, title: Upscale Image, class type: ImageScale
Executing node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 770, title: FILM VFI, class type: FILM VFI
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Final clearing cache...
Comfy-VFI: Done cache clearing
Executing node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 270.81 seconds
outputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
preview_00001.mp4
upscaled_00001.mp4
upscaled_model_00001.mp4
interpolated_00001.mp4

Prediction

fofr/video-morpher:a75043c99151a0d97d02ea55998168172188e9687c6f2e66c102ea99fd7ca4de

Model

fofr/video-morpher:a75043c9

36pah0gewxrgm0cf225ak122wm

Status

Succeeded

Source

Web

Hardware

A40 (Large)

Total duration

4m 58s

Created

about 1 year ago

Input

mode: upscaled-and-interpolated
prompt
checkpoint: 3D
style_image
aspect_ratio: 4:3
style_strength: 1
negative_prompt
subject_image_1
subject_image_2
subject_image_3
subject_image_4

{
  "mode": "upscaled-and-interpolated",
  "prompt": "",
  "checkpoint": "3D",
  "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
  "aspect_ratio": "4:3",
  "style_strength": 1,
  "negative_prompt": "",
  "subject_image_1": "https://replicate.delivery/pbxt/Kns41KZNEQhNyIjeVyW9mA7PV21EkcRDZ14rYbfHa0ZeNug1/ComfyUI_03072_.png",
  "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
  "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
  "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/video-morpher:a75043c99151a0d97d02ea55998168172188e9687c6f2e66c102ea99fd7ca4de",
  {
    input: {
      mode: "upscaled-and-interpolated",
      prompt: "",
      checkpoint: "3D",
      style_image: "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
      aspect_ratio: "4:3",
      style_strength: 1,
      negative_prompt: "",
      subject_image_1: "https://replicate.delivery/pbxt/Kns41KZNEQhNyIjeVyW9mA7PV21EkcRDZ14rYbfHa0ZeNug1/ComfyUI_03072_.png",
      subject_image_2: "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
      subject_image_3: "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
      subject_image_4: "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/video-morpher:a75043c99151a0d97d02ea55998168172188e9687c6f2e66c102ea99fd7ca4de",
    input={
        "mode": "upscaled-and-interpolated",
        "prompt": "",
        "checkpoint": "3D",
        "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
        "aspect_ratio": "4:3",
        "style_strength": 1,
        "negative_prompt": "",
        "subject_image_1": "https://replicate.delivery/pbxt/Kns41KZNEQhNyIjeVyW9mA7PV21EkcRDZ14rYbfHa0ZeNug1/ComfyUI_03072_.png",
        "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
        "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
        "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/video-morpher:a75043c99151a0d97d02ea55998168172188e9687c6f2e66c102ea99fd7ca4de",
    "input": {
      "mode": "upscaled-and-interpolated",
      "prompt": "",
      "checkpoint": "3D",
      "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
      "aspect_ratio": "4:3",
      "style_strength": 1,
      "negative_prompt": "",
      "subject_image_1": "https://replicate.delivery/pbxt/Kns41KZNEQhNyIjeVyW9mA7PV21EkcRDZ14rYbfHa0ZeNug1/ComfyUI_03072_.png",
      "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
      "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
      "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-04-24T14:42:12.708949Z",
  "created_at": "2024-04-24T14:37:14.343000Z",
  "data_removed": false,
  "error": null,
  "id": "36pah0gewxrgm0cf225ak122wm",
  "input": {
    "mode": "upscaled-and-interpolated",
    "prompt": "",
    "checkpoint": "3D",
    "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
    "aspect_ratio": "4:3",
    "style_strength": 1,
    "negative_prompt": "",
    "subject_image_1": "https://replicate.delivery/pbxt/Kns41KZNEQhNyIjeVyW9mA7PV21EkcRDZ14rYbfHa0ZeNug1/ComfyUI_03072_.png",
    "subject_image_2": "https://replicate.delivery/pbxt/KnroDfMF7HJay9msYtZMHB4vwMz9PXpvuY4vldEEmrGFxfFb/ComfyUI_02602_.png",
    "subject_image_3": "https://replicate.delivery/pbxt/KnroE2qG6XaIwokhg66ApTDHh8URpKEg8IP2CFXQ0L8isxpP/ComfyUI_02575_.png",
    "subject_image_4": "https://replicate.delivery/pbxt/KnroE49VZGwosByNYd6XCI5dStBtrSQRPmiW89bSdXv9f1Lx/863x1200-828592993.jpg"
  },
  "logs": "Random seed set to: 3961438974\nChecking inputs\n✅ /tmp/inputs/2.png\n✅ /tmp/inputs/1.png\n✅ /tmp/inputs/3.png\n✅ /tmp/inputs/4.png\n✅ /tmp/inputs/circles.mp4\n✅ /tmp/inputs/style.png\n====================================\nChecking weights\nIncluding weights for IPAdapter preset: PLUS (high strength)\n⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet\n⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.58s, size: 689.12MB\n✅ control_v1p_sd15_qrcode_monster.safetensors\n⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras\n⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.25s, size: 128.39MB\n✅ AnimateLCM_sd15_t2v_lora.safetensors\n⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae\n⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.34s, size: 319.14MB\n✅ vae-ft-mse-840000-ema-pruned.safetensors\n⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models\n⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.37s, size: 1729.05MB\n✅ AnimateLCM_sd15_t2v.ckpt\n⏳ Downloading rcnzCartoon3d_v20.safetensors to ComfyUI/models/checkpoints\n⌛️ Downloaded rcnzCartoon3d_v20.safetensors in 10.13s, size: 2033.83MB\n✅ rcnzCartoon3d_v20.safetensors\n⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.30s, size: 93.63MB\n✅ ip-adapter-plus_sd15.safetensors\n⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision\n⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 1.46s, size: 2411.24MB\n✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\n⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models\n⌛️ Downloaded RealESRGAN_x4.pth in 0.18s, size: 63.94MB\n✅ RealESRGAN_x4.pth\n⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter\n⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.72s, size: 808.26MB\n✅ ip-adapter-plus_sdxl_vit-h.safetensors\n⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film\n⌛️ Downloaded film_net_fp32.pt in 0.22s, size: 131.53MB\n✅ film_net_fp32.pt\n====================================\nRunning workflow\ngot prompt\nExecuting node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple\nmodel_type EPS\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nclip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']\nloaded straight to GPU\nRequested to load BaseModel\nLoading 1 new model\nExecuting node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2\nExecuting node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel\nExecuting node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic\nExecuting node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple\nExecuting node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions\nExecuting node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling\nExecuting node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader\n\u001b[33mINFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\u001b[0m\n\u001b[33mINFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors\u001b[0m\nExecuting node 142, title: Load Image, class type: LoadImage\nExecuting node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nRequested to load CLIPVisionModelProjection\nLoading 1 new model\nExecuting node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 135, title: Load Image, class type: LoadImage\nExecuting node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 680, title: Load Image, class type: LoadImage\nExecuting node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 683, title: Load Image, class type: LoadImage\nExecuting node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced\nExecuting node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nExecuting node 752, title: Load Image, class type: LoadImage\nExecuting node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nRequested to load SD1ClipModel\nLoading 1 new model\nExecuting node 565, title: Positive, class type: CLIPTextEncode\nExecuting node 566, title: Negative, class type: CLIPTextEncode\nExecuting node 134, title: Empty Latent Image, class type: EmptyLatentImage\nExecuting node 80, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\nRequested to load BaseModel\nRequested to load AnimateDiffModel\nLoading 2 new models\n  0%|          | 0/8 [00:00<?, ?it/s]\n 12%|█▎        | 1/8 [00:03<00:22,  3.27s/it]\n 25%|██▌       | 2/8 [00:05<00:14,  2.47s/it]\n 38%|███▊      | 3/8 [00:07<00:11,  2.22s/it]\n 50%|█████     | 4/8 [00:09<00:08,  2.10s/it]\n 62%|██████▎   | 5/8 [00:10<00:06,  2.03s/it]\n 75%|███████▌  | 6/8 [00:12<00:03,  2.00s/it]\n 88%|████████▊ | 7/8 [00:14<00:01,  1.97s/it]\n100%|██████████| 8/8 [00:16<00:00,  1.96s/it]\n100%|██████████| 8/8 [00:16<00:00,  2.09s/it]\nExecuting node 85, title: Load VAE, class type: VAELoader\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nRequested to load AutoencoderKL\nLoading 1 new model\nExecuting node 84, title: VAE Decode, class type: VAEDecode\nExecuting node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 203, title: Upscale Image By, class type: ImageScaleBy\nExecuting node 204, title: VAE Encode, class type: VAEEncode\nExecuting node 198, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\n  0%|          | 0/8 [00:00<?, ?it/s]\n 12%|█▎        | 1/8 [00:03<00:27,  3.89s/it]\n 25%|██▌       | 2/8 [00:07<00:23,  3.87s/it]\n 38%|███▊      | 3/8 [00:11<00:19,  3.94s/it]\n 50%|█████     | 4/8 [00:15<00:15,  3.92s/it]\n 62%|██████▎   | 5/8 [00:19<00:11,  3.90s/it]\n 75%|███████▌  | 6/8 [00:23<00:07,  3.89s/it]\n 88%|████████▊ | 7/8 [00:27<00:03,  3.89s/it]\n100%|██████████| 8/8 [00:31<00:00,  3.89s/it]\n100%|██████████| 8/8 [00:31<00:00,  3.89s/it]\nExecuting node 201, title: VAE Decode, class type: VAEDecode\nExecuting node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 270, title: Load Upscale Model, class type: UpscaleModelLoader\nExecuting node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel\nExecuting node 279, title: Upscale Image, class type: ImageScale\nExecuting node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 770, title: FILM VFI, class type: FILM VFI\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Final clearing cache...\nComfy-VFI: Done cache clearing\nExecuting node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 250.04 seconds\noutputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\npreview_00001.mp4\nupscaled_00001.mp4\nupscaled_model_00001.mp4\ninterpolated_00001.mp4",
  "metrics": {
    "predict_time": 270.722101,
    "total_time": 298.365949
  },
  "output": [
    "https://replicate.delivery/pbxt/YhuZXJfRfipvOU7Qf73LXnPDkBcVqf9i9sBqJpWqVtaIEz2KB/preview_00001.mp4",
    "https://replicate.delivery/pbxt/38LFmx7PZNarONApyM2amJJapsO6deZ2Tn5ldlucnu0hY2WJA/upscaled_00001.mp4",
    "https://replicate.delivery/pbxt/rp0xUZfVntSzHyXLrW2uajlXgzt2kPZyersE1u2AGVYDxstSA/upscaled_model_00001.mp4",
    "https://replicate.delivery/pbxt/MAxPGUCoA95ZE9cVfVSj9fZukRIbKF0Fsh3VVgdVJaIExstSA/interpolated_00001.mp4"
  ],
  "started_at": "2024-04-24T14:37:41.986848Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/36pah0gewxrgm0cf225ak122wm",
    "cancel": "https://api.replicate.com/v1/predictions/36pah0gewxrgm0cf225ak122wm/cancel"
  },
  "version": "a75043c99151a0d97d02ea55998168172188e9687c6f2e66c102ea99fd7ca4de"
}

Generated in

4 minutes 31 seconds

Tweak it Share Report

Random seed set to: 3961438974
Checking inputs
✅ /tmp/inputs/2.png
✅ /tmp/inputs/1.png
✅ /tmp/inputs/3.png
✅ /tmp/inputs/4.png
✅ /tmp/inputs/circles.mp4
✅ /tmp/inputs/style.png
====================================
Checking weights
Including weights for IPAdapter preset: PLUS (high strength)
⏳ Downloading control_v1p_sd15_qrcode_monster.safetensors to ComfyUI/models/controlnet
⌛️ Downloaded control_v1p_sd15_qrcode_monster.safetensors in 0.58s, size: 689.12MB
✅ control_v1p_sd15_qrcode_monster.safetensors
⏳ Downloading AnimateLCM_sd15_t2v_lora.safetensors to ComfyUI/models/loras
⌛️ Downloaded AnimateLCM_sd15_t2v_lora.safetensors in 0.25s, size: 128.39MB
✅ AnimateLCM_sd15_t2v_lora.safetensors
⏳ Downloading vae-ft-mse-840000-ema-pruned.safetensors to ComfyUI/models/vae
⌛️ Downloaded vae-ft-mse-840000-ema-pruned.safetensors in 0.34s, size: 319.14MB
✅ vae-ft-mse-840000-ema-pruned.safetensors
⏳ Downloading AnimateLCM_sd15_t2v.ckpt to ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models
⌛️ Downloaded AnimateLCM_sd15_t2v.ckpt in 1.37s, size: 1729.05MB
✅ AnimateLCM_sd15_t2v.ckpt
⏳ Downloading rcnzCartoon3d_v20.safetensors to ComfyUI/models/checkpoints
⌛️ Downloaded rcnzCartoon3d_v20.safetensors in 10.13s, size: 2033.83MB
✅ rcnzCartoon3d_v20.safetensors
⏳ Downloading ip-adapter-plus_sd15.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sd15.safetensors in 0.30s, size: 93.63MB
✅ ip-adapter-plus_sd15.safetensors
⏳ Downloading CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors to ComfyUI/models/clip_vision
⌛️ Downloaded CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors in 1.46s, size: 2411.24MB
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
⏳ Downloading RealESRGAN_x4.pth to ComfyUI/models/upscale_models
⌛️ Downloaded RealESRGAN_x4.pth in 0.18s, size: 63.94MB
✅ RealESRGAN_x4.pth
⏳ Downloading ip-adapter-plus_sdxl_vit-h.safetensors to ComfyUI/models/ipadapter
⌛️ Downloaded ip-adapter-plus_sdxl_vit-h.safetensors in 0.72s, size: 808.26MB
✅ ip-adapter-plus_sdxl_vit-h.safetensors
⏳ Downloading film_net_fp32.pt to ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation/ckpts/film
⌛️ Downloaded film_net_fp32.pt in 0.22s, size: 131.53MB
✅ film_net_fp32.pt
====================================
Running workflow
got prompt
Executing node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
Executing node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly
[AnimateDiffEvo] - INFO - Loading motion module AnimateLCM_sd15_t2v.ckpt via Gen2
Executing node 87, title: Load AnimateDiff Model 🎭🅐🅓②, class type: ADE_LoadAnimateDiffModel
Executing node 256, title: Motion Scale 🎭🅐🅓, class type: ADE_MultivalDynamic
Executing node 79, title: Apply AnimateDiff Model 🎭🅐🅓②, class type: ADE_ApplyAnimateDiffModelSimple
Executing node 156, title: Context Options◆Looped Uniform 🎭🅐🅓, class type: ADE_LoopedUniformContextOptions
Executing node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling
Executing node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader
INFO: Clip Vision model loaded from /src/ComfyUI/models/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
INFO: IPAdapter model loaded from /src/ComfyUI/models/ipadapter/ip-adapter-plus_sd15.safetensors
Executing node 142, title: Load Image, class type: LoadImage
Executing node 701, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Requested to load CLIPVisionModelProjection
Loading 1 new model
Executing node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 135, title: Load Image, class type: LoadImage
Executing node 707, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 680, title: Load Image, class type: LoadImage
Executing node 710, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 683, title: Load Image, class type: LoadImage
Executing node 713, title: CreateFadeMaskAdvanced, class type: CreateFadeMaskAdvanced
Executing node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Executing node 752, title: Load Image, class type: LoadImage
Executing node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Requested to load SD1ClipModel
Loading 1 new model
Executing node 565, title: Positive, class type: CLIPTextEncode
Executing node 566, title: Negative, class type: CLIPTextEncode
Executing node 134, title: Empty Latent Image, class type: EmptyLatentImage
Executing node 80, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
Requested to load BaseModel
Requested to load AnimateDiffModel
Loading 2 new models
  0%|          | 0/8 [00:00<?, ?it/s]
 12%|█▎        | 1/8 [00:03<00:22,  3.27s/it]
 25%|██▌       | 2/8 [00:05<00:14,  2.47s/it]
 38%|███▊      | 3/8 [00:07<00:11,  2.22s/it]
 50%|█████     | 4/8 [00:09<00:08,  2.10s/it]
 62%|██████▎   | 5/8 [00:10<00:06,  2.03s/it]
 75%|███████▌  | 6/8 [00:12<00:03,  2.00s/it]
 88%|████████▊ | 7/8 [00:14<00:01,  1.97s/it]
100%|██████████| 8/8 [00:16<00:00,  1.96s/it]
100%|██████████| 8/8 [00:16<00:00,  2.09s/it]
Executing node 85, title: Load VAE, class type: VAELoader
Using pytorch attention in VAE
Using pytorch attention in VAE
Requested to load AutoencoderKL
Loading 1 new model
Executing node 84, title: VAE Decode, class type: VAEDecode
Executing node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 203, title: Upscale Image By, class type: ImageScaleBy
Executing node 204, title: VAE Encode, class type: VAEEncode
Executing node 198, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
  0%|          | 0/8 [00:00<?, ?it/s]
 12%|█▎        | 1/8 [00:03<00:27,  3.89s/it]
 25%|██▌       | 2/8 [00:07<00:23,  3.87s/it]
 38%|███▊      | 3/8 [00:11<00:19,  3.94s/it]
 50%|█████     | 4/8 [00:15<00:15,  3.92s/it]
 62%|██████▎   | 5/8 [00:19<00:11,  3.90s/it]
 75%|███████▌  | 6/8 [00:23<00:07,  3.89s/it]
 88%|████████▊ | 7/8 [00:27<00:03,  3.89s/it]
100%|██████████| 8/8 [00:31<00:00,  3.89s/it]
100%|██████████| 8/8 [00:31<00:00,  3.89s/it]
Executing node 201, title: VAE Decode, class type: VAEDecode
Executing node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 270, title: Load Upscale Model, class type: UpscaleModelLoader
Executing node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel
Executing node 279, title: Upscale Image, class type: ImageScale
Executing node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 770, title: FILM VFI, class type: FILM VFI
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Final clearing cache...
Comfy-VFI: Done cache clearing
Executing node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 250.04 seconds
outputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
preview_00001.mp4
upscaled_00001.mp4
upscaled_model_00001.mp4
interpolated_00001.mp4

Prediction

fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf

Model

fofr/video-morpher:355c6bba

4r54mjhhzdrgg0cf22mrkbzee0

Status

Succeeded

Source

Web

Hardware

A40 (Large)

Total duration

4m 16s

Created

about 1 year ago by @fofr

Input

mode: upscaled-and-interpolated
prompt
checkpoint: 3D
style_image
aspect_ratio: 4:3
style_strength: 1
negative_prompt
subject_image_1
subject_image_2
subject_image_3
subject_image_4

{
  "mode": "upscaled-and-interpolated",
  "prompt": "",
  "checkpoint": "3D",
  "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
  "aspect_ratio": "4:3",
  "style_strength": 1,
  "negative_prompt": "",
  "subject_image_1": "https://replicate.delivery/pbxt/KnswBtSnF1TUPOiWuMQKiVbV6MFwe6XiwqPAYvKg6gT3lQ71/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
  "subject_image_2": "https://replicate.delivery/pbxt/KnswBrEyhOFdEojl0CdENgE5DlTj9kGhEK4z2iCbRAsABGAk/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
  "subject_image_3": "https://replicate.delivery/pbxt/KnswBqwa5yZrbPOaHiY76FDYWfgXwOM5ccRezIVKyjOcP73B/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
  "subject_image_4": "https://replicate.delivery/pbxt/KnswBvWex3ox1iONPTqc8SjNLtYCIOXMa6WbFK4hWrsGii99/ComfyUI_02710_.png"
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf",
  {
    input: {
      mode: "upscaled-and-interpolated",
      prompt: "",
      checkpoint: "3D",
      style_image: "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
      aspect_ratio: "4:3",
      style_strength: 1,
      negative_prompt: "",
      subject_image_1: "https://replicate.delivery/pbxt/KnswBtSnF1TUPOiWuMQKiVbV6MFwe6XiwqPAYvKg6gT3lQ71/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
      subject_image_2: "https://replicate.delivery/pbxt/KnswBrEyhOFdEojl0CdENgE5DlTj9kGhEK4z2iCbRAsABGAk/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
      subject_image_3: "https://replicate.delivery/pbxt/KnswBqwa5yZrbPOaHiY76FDYWfgXwOM5ccRezIVKyjOcP73B/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
      subject_image_4: "https://replicate.delivery/pbxt/KnswBvWex3ox1iONPTqc8SjNLtYCIOXMa6WbFK4hWrsGii99/ComfyUI_02710_.png"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf",
    input={
        "mode": "upscaled-and-interpolated",
        "prompt": "",
        "checkpoint": "3D",
        "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
        "aspect_ratio": "4:3",
        "style_strength": 1,
        "negative_prompt": "",
        "subject_image_1": "https://replicate.delivery/pbxt/KnswBtSnF1TUPOiWuMQKiVbV6MFwe6XiwqPAYvKg6gT3lQ71/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
        "subject_image_2": "https://replicate.delivery/pbxt/KnswBrEyhOFdEojl0CdENgE5DlTj9kGhEK4z2iCbRAsABGAk/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
        "subject_image_3": "https://replicate.delivery/pbxt/KnswBqwa5yZrbPOaHiY76FDYWfgXwOM5ccRezIVKyjOcP73B/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
        "subject_image_4": "https://replicate.delivery/pbxt/KnswBvWex3ox1iONPTqc8SjNLtYCIOXMa6WbFK4hWrsGii99/ComfyUI_02710_.png"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/video-morpher:355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf",
    "input": {
      "mode": "upscaled-and-interpolated",
      "prompt": "",
      "checkpoint": "3D",
      "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
      "aspect_ratio": "4:3",
      "style_strength": 1,
      "negative_prompt": "",
      "subject_image_1": "https://replicate.delivery/pbxt/KnswBtSnF1TUPOiWuMQKiVbV6MFwe6XiwqPAYvKg6gT3lQ71/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
      "subject_image_2": "https://replicate.delivery/pbxt/KnswBrEyhOFdEojl0CdENgE5DlTj9kGhEK4z2iCbRAsABGAk/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
      "subject_image_3": "https://replicate.delivery/pbxt/KnswBqwa5yZrbPOaHiY76FDYWfgXwOM5ccRezIVKyjOcP73B/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
      "subject_image_4": "https://replicate.delivery/pbxt/KnswBvWex3ox1iONPTqc8SjNLtYCIOXMa6WbFK4hWrsGii99/ComfyUI_02710_.png"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-04-24T15:15:30.860405Z",
  "created_at": "2024-04-24T15:11:14.939000Z",
  "data_removed": false,
  "error": null,
  "id": "4r54mjhhzdrgg0cf22mrkbzee0",
  "input": {
    "mode": "upscaled-and-interpolated",
    "prompt": "",
    "checkpoint": "3D",
    "style_image": "https://replicate.delivery/pbxt/Kns41UhVhW7rmSPxTxTGK7oWd441x2Xjv2DI5w8oep07lE0o/hero-char.png",
    "aspect_ratio": "4:3",
    "style_strength": 1,
    "negative_prompt": "",
    "subject_image_1": "https://replicate.delivery/pbxt/KnswBtSnF1TUPOiWuMQKiVbV6MFwe6XiwqPAYvKg6gT3lQ71/fofr_a_middle_aged_man_with_thick_glasses_16f412ff-db00-4e06-acdf-828885df6c58.webp",
    "subject_image_2": "https://replicate.delivery/pbxt/KnswBrEyhOFdEojl0CdENgE5DlTj9kGhEK4z2iCbRAsABGAk/fofr_a_blonde_woman_studio_headshot_cf000993-a68c-499f-9619-026601661d97.png",
    "subject_image_3": "https://replicate.delivery/pbxt/KnswBqwa5yZrbPOaHiY76FDYWfgXwOM5ccRezIVKyjOcP73B/marble-statue-antinous-height-180-cm-9513049.jpg.webp",
    "subject_image_4": "https://replicate.delivery/pbxt/KnswBvWex3ox1iONPTqc8SjNLtYCIOXMa6WbFK4hWrsGii99/ComfyUI_02710_.png"
  },
  "logs": "Random seed set to: 1561674246\nChecking inputs\n✅ /tmp/inputs/2.png\n✅ /tmp/inputs/1.png\n✅ /tmp/inputs/3.png\n✅ /tmp/inputs/4.png\n✅ /tmp/inputs/circles.mp4\n✅ /tmp/inputs/style.png\n====================================\nChecking weights\nIncluding weights for IPAdapter preset: PLUS (high strength)\n✅ AnimateLCM_sd15_t2v.ckpt\n✅ ip-adapter-plus_sd15.safetensors\n✅ vae-ft-mse-840000-ema-pruned.safetensors\n✅ film_net_fp32.pt\n⏳ Downloading rcnzCartoon3d_v20.safetensors to ComfyUI/models/checkpoints\n⌛️ Downloaded rcnzCartoon3d_v20.safetensors in 1.87s, size: 2033.83MB\n✅ rcnzCartoon3d_v20.safetensors\n✅ control_v1p_sd15_qrcode_monster.safetensors\n✅ RealESRGAN_x4.pth\n✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\n✅ ip-adapter-plus_sdxl_vit-h.safetensors\n✅ AnimateLCM_sd15_t2v_lora.safetensors\n====================================\nRunning workflow\ngot prompt\nExecuting node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple\nmodel_type EPS\nUsing pytorch attention in VAE\nUsing pytorch attention in VAE\nclip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']\nloaded straight to GPU\nRequested to load BaseModel\nLoading 1 new model\nExecuting node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly\nExecuting node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling\nExecuting node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader\nExecuting node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 135, title: Load Image, class type: LoadImage\nExecuting node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 680, title: Load Image, class type: LoadImage\nExecuting node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nExecuting node 683, title: Load Image, class type: LoadImage\nExecuting node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\nExecuting node 752, title: Load Image, class type: LoadImage\nExecuting node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch\n\u001b[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\u001b[0m\nRequested to load SD1ClipModel\nLoading 1 new model\nExecuting node 565, title: Positive, class type: CLIPTextEncode\nExecuting node 566, title: Negative, class type: CLIPTextEncode\nExecuting node 134, title: Empty Latent Image, class type: EmptyLatentImage\nExecuting node 80, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\nRequested to load BaseModel\nLoading 1 new model\n  0%|          | 0/8 [00:00<?, ?it/s]\n 12%|█▎        | 1/8 [00:01<00:13,  1.90s/it]\n 25%|██▌       | 2/8 [00:03<00:11,  1.91s/it]\n 38%|███▊      | 3/8 [00:05<00:09,  1.91s/it]\n 50%|█████     | 4/8 [00:07<00:07,  1.91s/it]\n 62%|██████▎   | 5/8 [00:09<00:05,  1.91s/it]\n 75%|███████▌  | 6/8 [00:11<00:03,  1.91s/it]\n 88%|████████▊ | 7/8 [00:13<00:01,  1.91s/it]\n100%|██████████| 8/8 [00:15<00:00,  1.91s/it]\n100%|██████████| 8/8 [00:15<00:00,  1.91s/it]\nExecuting node 84, title: VAE Decode, class type: VAEDecode\nExecuting node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 203, title: Upscale Image By, class type: ImageScaleBy\nExecuting node 204, title: VAE Encode, class type: VAEEncode\nExecuting node 198, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\n  0%|          | 0/8 [00:00<?, ?it/s]\n 12%|█▎        | 1/8 [00:03<00:27,  3.87s/it]\n 25%|██▌       | 2/8 [00:07<00:23,  3.85s/it]\n 38%|███▊      | 3/8 [00:11<00:19,  3.85s/it]\n 50%|█████     | 4/8 [00:15<00:15,  3.85s/it]\n 62%|██████▎   | 5/8 [00:19<00:11,  3.85s/it]\n 75%|███████▌  | 6/8 [00:23<00:07,  3.85s/it]\n 88%|████████▊ | 7/8 [00:26<00:03,  3.85s/it]\n100%|██████████| 8/8 [00:30<00:00,  3.85s/it]\n100%|██████████| 8/8 [00:30<00:00,  3.85s/it]\nRequested to load AutoencoderKL\nLoading 1 new model\nExecuting node 201, title: VAE Decode, class type: VAEDecode\nExecuting node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel\nExecuting node 279, title: Upscale Image, class type: ImageScale\nExecuting node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 770, title: FILM VFI, class type: FILM VFI\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Final clearing cache...\nComfy-VFI: Done cache clearing\nExecuting node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 246.67 seconds\noutputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\npreview_00001.mp4\nupscaled_00001.mp4\nupscaled_model_00001.mp4\ninterpolated_00001.mp4",
  "metrics": {
    "predict_time": 255.879579,
    "total_time": 255.921405
  },
  "output": [
    "https://replicate.delivery/pbxt/eZTBeeFyOwOtxIe0SgRUcXBFmcrLBfwjAKGmWgUx8NV9BqtVC/preview_00001.mp4",
    "https://replicate.delivery/pbxt/zrphtsj2VTphCtRLfsJABGNV4XA5W59BXFqkj1EyHu2Io2WJA/upscaled_00001.mp4",
    "https://replicate.delivery/pbxt/68ecY1ceCPu9VE8uy9tfRUiJ5FbYEAerqH5DjLD4MfgOCqtVC/upscaled_model_00001.mp4",
    "https://replicate.delivery/pbxt/199QGHuGw8IvE5hvUSLTRpexvf9bLwZ8sgToTS0oWLWSQttSA/interpolated_00001.mp4"
  ],
  "started_at": "2024-04-24T15:11:14.980826Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/4r54mjhhzdrgg0cf22mrkbzee0",
    "cancel": "https://api.replicate.com/v1/predictions/4r54mjhhzdrgg0cf22mrkbzee0/cancel"
  },
  "version": "355c6bbaf8bc2deeafd3e2384a50af51bc2091a8be96dc082f1ef02c74640baf"
}

Generated in

4 minutes 16 seconds

Tweak it Share Report

Random seed set to: 1561674246
Checking inputs
✅ /tmp/inputs/2.png
✅ /tmp/inputs/1.png
✅ /tmp/inputs/3.png
✅ /tmp/inputs/4.png
✅ /tmp/inputs/circles.mp4
✅ /tmp/inputs/style.png
====================================
Checking weights
Including weights for IPAdapter preset: PLUS (high strength)
✅ AnimateLCM_sd15_t2v.ckpt
✅ ip-adapter-plus_sd15.safetensors
✅ vae-ft-mse-840000-ema-pruned.safetensors
✅ film_net_fp32.pt
⏳ Downloading rcnzCartoon3d_v20.safetensors to ComfyUI/models/checkpoints
⌛️ Downloaded rcnzCartoon3d_v20.safetensors in 1.87s, size: 2033.83MB
✅ rcnzCartoon3d_v20.safetensors
✅ control_v1p_sd15_qrcode_monster.safetensors
✅ RealESRGAN_x4.pth
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
✅ ip-adapter-plus_sdxl_vit-h.safetensors
✅ AnimateLCM_sd15_t2v_lora.safetensors
====================================
Running workflow
got prompt
Executing node 564, title: Load Checkpoint, class type: CheckpointLoaderSimple
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
Executing node 563, title: LoraLoaderModelOnly, class type: LoraLoaderModelOnly
Executing node 77, title: Use Evolved Sampling 🎭🅐🅓②, class type: ADE_UseEvolvedSampling
Executing node 573, title: IPAdapter Unified Loader, class type: IPAdapterUnifiedLoader
Executing node 545, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 135, title: Load Image, class type: LoadImage
Executing node 548, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 680, title: Load Image, class type: LoadImage
Executing node 681, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Executing node 683, title: Load Image, class type: LoadImage
Executing node 682, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
Executing node 752, title: Load Image, class type: LoadImage
Executing node 751, title: IPAdapter Batch (Adv.), class type: IPAdapterBatch
INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.
Requested to load SD1ClipModel
Loading 1 new model
Executing node 565, title: Positive, class type: CLIPTextEncode
Executing node 566, title: Negative, class type: CLIPTextEncode
Executing node 134, title: Empty Latent Image, class type: EmptyLatentImage
Executing node 80, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
Requested to load BaseModel
Loading 1 new model
  0%|          | 0/8 [00:00<?, ?it/s]
 12%|█▎        | 1/8 [00:01<00:13,  1.90s/it]
 25%|██▌       | 2/8 [00:03<00:11,  1.91s/it]
 38%|███▊      | 3/8 [00:05<00:09,  1.91s/it]
 50%|█████     | 4/8 [00:07<00:07,  1.91s/it]
 62%|██████▎   | 5/8 [00:09<00:05,  1.91s/it]
 75%|███████▌  | 6/8 [00:11<00:03,  1.91s/it]
 88%|████████▊ | 7/8 [00:13<00:01,  1.91s/it]
100%|██████████| 8/8 [00:15<00:00,  1.91s/it]
100%|██████████| 8/8 [00:15<00:00,  1.91s/it]
Executing node 84, title: VAE Decode, class type: VAEDecode
Executing node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 203, title: Upscale Image By, class type: ImageScaleBy
Executing node 204, title: VAE Encode, class type: VAEEncode
Executing node 198, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
  0%|          | 0/8 [00:00<?, ?it/s]
 12%|█▎        | 1/8 [00:03<00:27,  3.87s/it]
 25%|██▌       | 2/8 [00:07<00:23,  3.85s/it]
 38%|███▊      | 3/8 [00:11<00:19,  3.85s/it]
 50%|█████     | 4/8 [00:15<00:15,  3.85s/it]
 62%|██████▎   | 5/8 [00:19<00:11,  3.85s/it]
 75%|███████▌  | 6/8 [00:23<00:07,  3.85s/it]
 88%|████████▊ | 7/8 [00:26<00:03,  3.85s/it]
100%|██████████| 8/8 [00:30<00:00,  3.85s/it]
100%|██████████| 8/8 [00:30<00:00,  3.85s/it]
Requested to load AutoencoderKL
Loading 1 new model
Executing node 201, title: VAE Decode, class type: VAEDecode
Executing node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel
Executing node 279, title: Upscale Image, class type: ImageScale
Executing node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 770, title: FILM VFI, class type: FILM VFI
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Final clearing cache...
Comfy-VFI: Done cache clearing
Executing node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 246.67 seconds
outputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
preview_00001.mp4
upscaled_00001.mp4
upscaled_model_00001.mp4
interpolated_00001.mp4

Prediction

fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7

Model

fofr/video-morpher:e70e9750

ykjrhncvmdrm80cke9s9rhsqc4

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

Created

6 months ago

Input

mode: upscaled-and-interpolated
prompt: bright, vibrant, high contrast
checkpoint: 3D
style_image
aspect_ratio: 3:4
style_strength: 0.5
use_controlnet
negative_prompt: dark, gloomy
subject_image_1
subject_image_2
subject_image_3
subject_image_4

{
  "mode": "upscaled-and-interpolated",
  "prompt": "bright, vibrant, high contrast",
  "checkpoint": "3D",
  "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
  "aspect_ratio": "3:4",
  "style_strength": 0.5,
  "use_controlnet": true,
  "negative_prompt": "dark, gloomy",
  "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
  "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
  "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
  "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
  {
    input: {
      mode: "upscaled-and-interpolated",
      prompt: "bright, vibrant, high contrast",
      checkpoint: "3D",
      style_image: "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
      aspect_ratio: "3:4",
      style_strength: 0.5,
      use_controlnet: true,
      negative_prompt: "dark, gloomy",
      subject_image_1: "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
      subject_image_2: "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
      subject_image_3: "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
      subject_image_4: "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
    input={
        "mode": "upscaled-and-interpolated",
        "prompt": "bright, vibrant, high contrast",
        "checkpoint": "3D",
        "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
        "aspect_ratio": "3:4",
        "style_strength": 0.5,
        "use_controlnet": True,
        "negative_prompt": "dark, gloomy",
        "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
        "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
        "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
        "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run fofr/video-morpher using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/video-morpher:e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7",
    "input": {
      "mode": "upscaled-and-interpolated",
      "prompt": "bright, vibrant, high contrast",
      "checkpoint": "3D",
      "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
      "aspect_ratio": "3:4",
      "style_strength": 0.5,
      "use_controlnet": true,
      "negative_prompt": "dark, gloomy",
      "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
      "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
      "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
      "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-11-28T11:08:35.822944Z",
  "created_at": "2024-11-28T11:05:33.859000Z",
  "data_removed": false,
  "error": null,
  "id": "ykjrhncvmdrm80cke9s9rhsqc4",
  "input": {
    "mode": "upscaled-and-interpolated",
    "prompt": "bright, vibrant, high contrast",
    "checkpoint": "3D",
    "style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
    "aspect_ratio": "3:4",
    "style_strength": 0.5,
    "use_controlnet": true,
    "negative_prompt": "dark, gloomy",
    "subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
    "subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
    "subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
    "subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
  },
  "logs": "Random seed set to: 4049182242\nChecking inputs\n✅ /tmp/inputs/2.png\n✅ /tmp/inputs/1.png\n✅ /tmp/inputs/3.png\n✅ /tmp/inputs/4.png\n✅ /tmp/inputs/circles.mp4\n✅ /tmp/inputs/style.png\n====================================\nChecking weights\nIncluding weights for IPAdapter preset: PLUS (high strength)\n✅ RealESRGAN_x4.pth\n✅ film_net_fp32.pt\n✅ ip-adapter-plus_sdxl_vit-h.safetensors\n✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\n✅ AnimateLCM_sd15_t2v.ckpt\n✅ control_v1p_sd15_qrcode_monster.safetensors\n✅ vae-ft-mse-840000-ema-pruned.safetensors\n✅ ip-adapter-plus_sd15.safetensors\n✅ AnimateLCM_sd15_t2v_lora.safetensors\n✅ rcnzCartoon3d_v20.safetensors\n====================================\nRunning workflow\ngot prompt\nExecuting node 746, title: Load Video (Upload) 🎥🅥🅗🅢, class type: VHS_LoadVideo\nExecuting node 461, title: 🔧 Simple Math, class type: SimpleMath+\nExecuting node 454, title: RepeatImageBatch, class type: RepeatImageBatch\nExecuting node 458, title: Split Image Batch 🎥🅥🅗🅢, class type: VHS_SplitImages\nExecuting node 125, title: Apply Advanced ControlNet 🛂🅐🅒🅝, class type: ACN_AdvancedControlNetApply\nExecuting node 80, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\nRequested to load ControlNet\nLoading 1 new model\n  0%|          | 0/11 [00:00<?, ?it/s]\n  9%|▉         | 1/11 [00:01<00:11,  1.11s/it]\n 18%|█▊        | 2/11 [00:02<00:09,  1.11s/it]\n 27%|██▋       | 3/11 [00:03<00:09,  1.18s/it]\n 36%|███▋      | 4/11 [00:04<00:08,  1.21s/it]\n 45%|████▌     | 5/11 [00:05<00:06,  1.14s/it]\n 55%|█████▍    | 6/11 [00:06<00:05,  1.11s/it]\n 64%|██████▎   | 7/11 [00:07<00:04,  1.08s/it]\n 73%|███████▎  | 8/11 [00:09<00:03,  1.13s/it]\n 82%|████████▏ | 9/11 [00:10<00:02,  1.12s/it]\n 91%|█████████ | 10/11 [00:11<00:01,  1.09s/it]\n100%|██████████| 11/11 [00:12<00:00,  1.07s/it]\n100%|██████████| 11/11 [00:12<00:00,  1.11s/it]\nExecuting node 84, title: VAE Decode, class type: VAEDecode\nExecuting node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 203, title: Upscale Image By, class type: ImageScaleBy\nExecuting node 204, title: VAE Encode, class type: VAEEncode\nExecuting node 198, title: KSampler, class type: KSampler\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.\n[AnimateDiffEvo] - \u001b[0;32mINFO\u001b[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.\n  0%|          | 0/11 [00:00<?, ?it/s]\n  9%|▉         | 1/11 [00:02<00:21,  2.20s/it]\n 18%|█▊        | 2/11 [00:04<00:19,  2.19s/it]\n 27%|██▋       | 3/11 [00:06<00:17,  2.19s/it]\n 36%|███▋      | 4/11 [00:08<00:15,  2.19s/it]\n 45%|████▌     | 5/11 [00:10<00:13,  2.19s/it]\n 55%|█████▍    | 6/11 [00:13<00:10,  2.19s/it]\n 64%|██████▎   | 7/11 [00:15<00:08,  2.19s/it]\n 73%|███████▎  | 8/11 [00:17<00:06,  2.19s/it]\n 82%|████████▏ | 9/11 [00:19<00:04,  2.18s/it]\n 91%|█████████ | 10/11 [00:21<00:02,  2.18s/it]\n100%|██████████| 11/11 [00:24<00:00,  2.18s/it]\n100%|██████████| 11/11 [00:24<00:00,  2.19s/it]\nRequested to load AutoencoderKL\nLoading 1 new model\nExecuting node 201, title: VAE Decode, class type: VAEDecode\nExecuting node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel\nExecuting node 279, title: Upscale Image, class type: ImageScale\nExecuting node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nExecuting node 770, title: FILM VFI, class type: FILM VFI\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Clearing cache...\nComfy-VFI: Done cache clearing\nComfy-VFI: Final clearing cache...\nComfy-VFI: Done cache clearing\nExecuting node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 176.75 seconds\noutputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\npreview_00001.mp4\nupscaled_00001.mp4\nupscaled_model_00001.mp4\ninterpolated_00001.mp4",
  "metrics": {
    "predict_time": 181.956443524,
    "total_time": 181.963944
  },
  "output": [
    "https://replicate.delivery/xezq/fqOdvvfinVpSLUUMepVqdywzaSMyP2ru3DE33XUAAetPTAWPB/preview_00001.mp4",
    "https://replicate.delivery/xezq/fFvJWDTii62RfEK5ExHpy88YYFFVuvYUoR2sqBtcBb0zEg1TA/upscaled_00001.mp4",
    "https://replicate.delivery/xezq/oKSuEYjOC2rFF99foM6NVy0e1053GKPZn1AarSgHrAtzEg1TA/upscaled_model_00001.mp4",
    "https://replicate.delivery/xezq/sZQRsofi4U2Yf0x5wfrwJzRr4IipUs8tuixRqdIRJM3mJArnA/interpolated_00001.mp4"
  ],
  "started_at": "2024-11-28T11:05:33.866501Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://stream.replicate.com/v1/files/bcwr-tyikq3iqnxxvcbm44lu25xmm75evlu5c47aadad5o23zloamp4ka",
    "get": "https://api.replicate.com/v1/predictions/ykjrhncvmdrm80cke9s9rhsqc4",
    "cancel": "https://api.replicate.com/v1/predictions/ykjrhncvmdrm80cke9s9rhsqc4/cancel"
  },
  "version": "e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7"
}

Generated in

3 minutes

Tweak it Share Report

Random seed set to: 4049182242
Checking inputs
✅ /tmp/inputs/2.png
✅ /tmp/inputs/1.png
✅ /tmp/inputs/3.png
✅ /tmp/inputs/4.png
✅ /tmp/inputs/circles.mp4
✅ /tmp/inputs/style.png
====================================
Checking weights
Including weights for IPAdapter preset: PLUS (high strength)
✅ RealESRGAN_x4.pth
✅ film_net_fp32.pt
✅ ip-adapter-plus_sdxl_vit-h.safetensors
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
✅ AnimateLCM_sd15_t2v.ckpt
✅ control_v1p_sd15_qrcode_monster.safetensors
✅ vae-ft-mse-840000-ema-pruned.safetensors
✅ ip-adapter-plus_sd15.safetensors
✅ AnimateLCM_sd15_t2v_lora.safetensors
✅ rcnzCartoon3d_v20.safetensors
====================================
Running workflow
got prompt
Executing node 746, title: Load Video (Upload) 🎥🅥🅗🅢, class type: VHS_LoadVideo
Executing node 461, title: 🔧 Simple Math, class type: SimpleMath+
Executing node 454, title: RepeatImageBatch, class type: RepeatImageBatch
Executing node 458, title: Split Image Batch 🎥🅥🅗🅢, class type: VHS_SplitImages
Executing node 125, title: Apply Advanced ControlNet 🛂🅐🅒🅝, class type: ACN_AdvancedControlNetApply
Executing node 80, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
Requested to load ControlNet
Loading 1 new model
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:01<00:11,  1.11s/it]
 18%|█▊        | 2/11 [00:02<00:09,  1.11s/it]
 27%|██▋       | 3/11 [00:03<00:09,  1.18s/it]
 36%|███▋      | 4/11 [00:04<00:08,  1.21s/it]
 45%|████▌     | 5/11 [00:05<00:06,  1.14s/it]
 55%|█████▍    | 6/11 [00:06<00:05,  1.11s/it]
 64%|██████▎   | 7/11 [00:07<00:04,  1.08s/it]
 73%|███████▎  | 8/11 [00:09<00:03,  1.13s/it]
 82%|████████▏ | 9/11 [00:10<00:02,  1.12s/it]
 91%|█████████ | 10/11 [00:11<00:01,  1.09s/it]
100%|██████████| 11/11 [00:12<00:00,  1.07s/it]
100%|██████████| 11/11 [00:12<00:00,  1.11s/it]
Executing node 84, title: VAE Decode, class type: VAEDecode
Executing node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 203, title: Upscale Image By, class type: ImageScaleBy
Executing node 204, title: VAE Encode, class type: VAEEncode
Executing node 198, title: KSampler, class type: KSampler
[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:02<00:21,  2.20s/it]
 18%|█▊        | 2/11 [00:04<00:19,  2.19s/it]
 27%|██▋       | 3/11 [00:06<00:17,  2.19s/it]
 36%|███▋      | 4/11 [00:08<00:15,  2.19s/it]
 45%|████▌     | 5/11 [00:10<00:13,  2.19s/it]
 55%|█████▍    | 6/11 [00:13<00:10,  2.19s/it]
 64%|██████▎   | 7/11 [00:15<00:08,  2.19s/it]
 73%|███████▎  | 8/11 [00:17<00:06,  2.19s/it]
 82%|████████▏ | 9/11 [00:19<00:04,  2.18s/it]
 91%|█████████ | 10/11 [00:21<00:02,  2.18s/it]
100%|██████████| 11/11 [00:24<00:00,  2.18s/it]
100%|██████████| 11/11 [00:24<00:00,  2.19s/it]
Requested to load AutoencoderKL
Loading 1 new model
Executing node 201, title: VAE Decode, class type: VAEDecode
Executing node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel
Executing node 279, title: Upscale Image, class type: ImageScale
Executing node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 770, title: FILM VFI, class type: FILM VFI
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Final clearing cache...
Comfy-VFI: Done cache clearing
Executing node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 176.75 seconds
outputs:  {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
preview_00001.mp4
upscaled_00001.mp4
upscaled_model_00001.mp4
interpolated_00001.mp4