Readme
ToonCrafter
🤗 ToonCrafter can interpolate two cartoon images by leveraging the pre-trained image-to-video diffusion priors.
https://github.com/ToonCrafter/ToonCrafter
This model built with ComfyUI
Built using ComfyUI and Kijai’s custom nodes:
fofr
/
tooncrafter
Create videos from illustrated input images
Input
Run this model in Node.js with one line of code:
npx create-replicate --model=fofr/tooncrafter
or set up a project from scratch
Install Replicate’s Node.js client library:
npm install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import and set up the client:
import Replicate from "replicate";
const replicate = new Replicate({
auth: process.env.REPLICATE_API_TOKEN,
});Run fofr/tooncrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run(
"fofr/tooncrafter:0486ff07368e816ec3d5c69b9581e7a09b55817f567a0d74caad9395c9295c77",
{
input: {
loop: false,
prompt: "",
image_1: "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp",
image_2: "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp",
image_3: "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp",
max_width: 512,
max_height: 512,
interpolate: false,
negative_prompt: "",
color_correction: true
}
}
);
console.log(output);To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:
pip install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import the client:
import replicateRun fofr/tooncrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run(
"fofr/tooncrafter:0486ff07368e816ec3d5c69b9581e7a09b55817f567a0d74caad9395c9295c77",
input={
"loop": False,
"prompt": "",
"image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp",
"image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp",
"image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp",
"max_width": 512,
"max_height": 512,
"interpolate": False,
"negative_prompt": "",
"color_correction": True
}
)
print(output)To learn more, take a look at the guide on getting started with Python.
Set the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Run fofr/tooncrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
-H "Content-Type: application/json" \
-H "Prefer: wait" \
-d $'{
"version": "0486ff07368e816ec3d5c69b9581e7a09b55817f567a0d74caad9395c9295c77",
"input": {
"loop": false,
"prompt": "",
"image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp",
"image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp",
"image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp",
"max_width": 512,
"max_height": 512,
"interpolate": false,
"negative_prompt": "",
"color_correction": true
}
}' \
https://api.replicate.com/v1/predictionsTo learn more, take a look at Replicate’s HTTP API reference docs.
Add a payment method to run this model.
Sign in with GitHub
By signing in, you agree to our
terms of service and privacy policy
Output
{
"completed_at": "2024-06-02T21:59:40.667881Z",
"created_at": "2024-06-02T21:57:22.192000Z",
"data_removed": false,
"error": null,
"id": "gw8thp06j1rgp0cfvbtsvrq9vc",
"input": {
"loop": false,
"prompt": "",
"image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp",
"image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp",
"image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp",
"max_width": 512,
"max_height": 512,
"interpolate": false,
"negative_prompt": "",
"color_correction": true
},
"logs": "Random seed set to: 1500914532\nChecking inputs\n✅ /tmp/inputs/input_1.png\n✅ /tmp/inputs/input_2.png\n✅ /tmp/inputs/input_3.png\n====================================\nChecking weights\n✅ tooncrafter_512_interp-fp16.safetensors\n✅ stable-diffusion-2-1-clip-fp16.safetensors\n✅ CLIP-ViT-H-fp16.safetensors\n====================================\nRunning workflow\ngot prompt\nExecuting node 1, title: Load Image, class type: LoadImage\nDownloading model to: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors\nExecuting node 52, title: DownloadAndLoadDynamiCrafterModel, class type: DownloadAndLoadDynamiCrafterModel\nFetching 1 files: 0%| | 0/1 [00:00<?, ?it/s]/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`.\nFor more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.\nwarnings.warn(\nFetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it]\nFetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it]\nLoading model from: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors\nLatentVisualDiffusion: Running in v-prediction mode\nAE working on z of shape (1, 4, 32, 32) = 4096 dimensions.\nWorking with z of shape (1, 4, 32, 32) = 4096 dimensions.\nvanilla\nmaking attention of type 'vanilla' with 512 in_channels\nmemory-efficient-cross-attn-fusion\nmaking attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels\nmemory-efficient-cross-attn-fusion\nmaking attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels\n>>> model checkpoint loaded.\nModel using dtype: torch.float16\nExecuting node 61, title: DownloadAndLoadCLIPVisionModel, class type: DownloadAndLoadCLIPVisionModel\nLoading model from: /src/ComfyUI/models/clip_vision/CLIP-ViT-H-fp16.safetensors\nExecuting node 59, title: DownloadAndLoadCLIPModel, class type: DownloadAndLoadCLIPModel\nclip missing: ['text_model.encoder.layers.23.layer_norm1.weight', 'text_model.encoder.layers.23.layer_norm1.bias', 'text_model.encoder.layers.23.self_attn.q_proj.weight', 'text_model.encoder.layers.23.self_attn.q_proj.bias', 'text_model.encoder.layers.23.self_attn.k_proj.weight', 'text_model.encoder.layers.23.self_attn.k_proj.bias', 'text_model.encoder.layers.23.self_attn.v_proj.weight', 'text_model.encoder.layers.23.self_attn.v_proj.bias', 'text_model.encoder.layers.23.self_attn.out_proj.weight', 'text_model.encoder.layers.23.self_attn.out_proj.bias', 'text_model.encoder.layers.23.layer_norm2.weight', 'text_model.encoder.layers.23.layer_norm2.bias', 'text_model.encoder.layers.23.mlp.fc1.weight', 'text_model.encoder.layers.23.mlp.fc1.bias', 'text_model.encoder.layers.23.mlp.fc2.weight', 'text_model.encoder.layers.23.mlp.fc2.bias', 'text_projection.weight']\nLoading model from: /src/ComfyUI/models/clip/stable-diffusion-2-1-clip-fp16.safetensors\nRequested to load SD2ClipModel\nLoading 1 new model\nExecuting node 49, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode\nExecuting node 50, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode\nExecuting node 70, title: 🔧 Image Resize, class type: ImageResize+\nExecuting node 2, title: Load Image, class type: LoadImage\nExecuting node 303, title: Load Image, class type: LoadImage\nExecuting node 28, title: Image Batch Multi, class type: ImageBatchMulti\nExecuting node 6, title: Get Image Size & Count, class type: GetImageSizeAndCount\nExecuting node 65, title: 🔧 Image Resize, class type: ImageResize+\nExecuting node 57, title: ToonCrafterInterpolation, class type: ToonCrafterInterpolation\nVAE using dtype: torch.bfloat16\nRequested to load CLIPVisionModelProjection\nLoading 1 new model\nDDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s]\nDDIM Sampler: 5%|▌ | 1/20 [00:00<00:13, 1.37it/s]\nDDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.51it/s]\nDDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.56it/s]\nDDIM Sampler: 20%|██ | 4/20 [00:02<00:10, 1.59it/s]\nDDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.60it/s]\nDDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.61it/s]\nDDIM Sampler: 35%|███▌ | 7/20 [00:04<00:08, 1.62it/s]\nDDIM Sampler: 40%|████ | 8/20 [00:05<00:07, 1.62it/s]\nDDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.62it/s]\nDDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.62it/s]\nDDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s]\nDDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s]\nDDIM Sampler: 65%|██████▌ | 13/20 [00:08<00:04, 1.63it/s]\nDDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s]\nDDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s]\nDDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s]\nDDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s]\nDDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s]\nDDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.61it/s]\nDDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s]\nDDIM Sampler: 5%|▌ | 1/20 [00:00<00:11, 1.63it/s]\nDDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.63it/s]\nDDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.63it/s]\nDDIM Sampler: 20%|██ | 4/20 [00:02<00:09, 1.63it/s]\nDDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.63it/s]\nDDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.63it/s]\nDDIM Sampler: 35%|███▌ | 7/20 [00:04<00:07, 1.63it/s]\nDDIM Sampler: 40%|████ | 8/20 [00:04<00:07, 1.63it/s]\nDDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.63it/s]\nDDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.63it/s]\nDDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s]\nDDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s]\nDDIM Sampler: 65%|██████▌ | 13/20 [00:07<00:04, 1.63it/s]\nDDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s]\nDDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s]\nDDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s]\nDDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s]\nDDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s]\nDDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]\nExecuting node 58, title: ToonCrafterDecode, class type: ToonCrafterDecode\nVAE using dtype: torch.bfloat16\nUsing xformers\n/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/conv.py:605: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)\nreturn F.conv3d(\nUsing xformers\nExecuting node 67, title: Color Match, class type: ColorMatch\nExecuting node 29, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 59.29 seconds\noutputs: {'6': {'text': ['3x512x512']}, '29': {'gifs': [{'filename': 'ToonCrafter_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\nToonCrafter_00001.png\nToonCrafter_00001.mp4",
"metrics": {
"predict_time": 61.900853,
"total_time": 138.475881
},
"output": [
"https://replicate.delivery/pbxt/aDDntBsXmmL6DZMIRURYbxjeer1PZxmT9TIl0MOfqcrYqT1lA/ToonCrafter_00001.mp4"
],
"started_at": "2024-06-02T21:58:38.767028Z",
"status": "succeeded",
"urls": {
"get": "https://api.replicate.com/v1/predictions/gw8thp06j1rgp0cfvbtsvrq9vc",
"cancel": "https://api.replicate.com/v1/predictions/gw8thp06j1rgp0cfvbtsvrq9vc/cancel"
},
"version": "51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40ed"
}Generated in
Random seed set to: 1500914532
Checking inputs
✅ /tmp/inputs/input_1.png
✅ /tmp/inputs/input_2.png
✅ /tmp/inputs/input_3.png
====================================
Checking weights
✅ tooncrafter_512_interp-fp16.safetensors
✅ stable-diffusion-2-1-clip-fp16.safetensors
✅ CLIP-ViT-H-fp16.safetensors
====================================
Running workflow
got prompt
Executing node 1, title: Load Image, class type: LoadImage
Downloading model to: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors
Executing node 52, title: DownloadAndLoadDynamiCrafterModel, class type: DownloadAndLoadDynamiCrafterModel
Fetching 1 files: 0%| | 0/1 [00:00<?, ?it/s]/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`.
For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.
warnings.warn(
Fetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it]
Fetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it]
Loading model from: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors
LatentVisualDiffusion: Running in v-prediction mode
AE working on z of shape (1, 4, 32, 32) = 4096 dimensions.
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
vanilla
making attention of type 'vanilla' with 512 in_channels
memory-efficient-cross-attn-fusion
making attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels
memory-efficient-cross-attn-fusion
making attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels
>>> model checkpoint loaded.
Model using dtype: torch.float16
Executing node 61, title: DownloadAndLoadCLIPVisionModel, class type: DownloadAndLoadCLIPVisionModel
Loading model from: /src/ComfyUI/models/clip_vision/CLIP-ViT-H-fp16.safetensors
Executing node 59, title: DownloadAndLoadCLIPModel, class type: DownloadAndLoadCLIPModel
clip missing: ['text_model.encoder.layers.23.layer_norm1.weight', 'text_model.encoder.layers.23.layer_norm1.bias', 'text_model.encoder.layers.23.self_attn.q_proj.weight', 'text_model.encoder.layers.23.self_attn.q_proj.bias', 'text_model.encoder.layers.23.self_attn.k_proj.weight', 'text_model.encoder.layers.23.self_attn.k_proj.bias', 'text_model.encoder.layers.23.self_attn.v_proj.weight', 'text_model.encoder.layers.23.self_attn.v_proj.bias', 'text_model.encoder.layers.23.self_attn.out_proj.weight', 'text_model.encoder.layers.23.self_attn.out_proj.bias', 'text_model.encoder.layers.23.layer_norm2.weight', 'text_model.encoder.layers.23.layer_norm2.bias', 'text_model.encoder.layers.23.mlp.fc1.weight', 'text_model.encoder.layers.23.mlp.fc1.bias', 'text_model.encoder.layers.23.mlp.fc2.weight', 'text_model.encoder.layers.23.mlp.fc2.bias', 'text_projection.weight']
Loading model from: /src/ComfyUI/models/clip/stable-diffusion-2-1-clip-fp16.safetensors
Requested to load SD2ClipModel
Loading 1 new model
Executing node 49, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode
Executing node 50, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode
Executing node 70, title: 🔧 Image Resize, class type: ImageResize+
Executing node 2, title: Load Image, class type: LoadImage
Executing node 303, title: Load Image, class type: LoadImage
Executing node 28, title: Image Batch Multi, class type: ImageBatchMulti
Executing node 6, title: Get Image Size & Count, class type: GetImageSizeAndCount
Executing node 65, title: 🔧 Image Resize, class type: ImageResize+
Executing node 57, title: ToonCrafterInterpolation, class type: ToonCrafterInterpolation
VAE using dtype: torch.bfloat16
Requested to load CLIPVisionModelProjection
Loading 1 new model
DDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s]
DDIM Sampler: 5%|▌ | 1/20 [00:00<00:13, 1.37it/s]
DDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.51it/s]
DDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.56it/s]
DDIM Sampler: 20%|██ | 4/20 [00:02<00:10, 1.59it/s]
DDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.60it/s]
DDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.61it/s]
DDIM Sampler: 35%|███▌ | 7/20 [00:04<00:08, 1.62it/s]
DDIM Sampler: 40%|████ | 8/20 [00:05<00:07, 1.62it/s]
DDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.62it/s]
DDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.62it/s]
DDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s]
DDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s]
DDIM Sampler: 65%|██████▌ | 13/20 [00:08<00:04, 1.63it/s]
DDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s]
DDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s]
DDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s]
DDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s]
DDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s]
DDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s]
DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]
DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.61it/s]
DDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s]
DDIM Sampler: 5%|▌ | 1/20 [00:00<00:11, 1.63it/s]
DDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.63it/s]
DDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.63it/s]
DDIM Sampler: 20%|██ | 4/20 [00:02<00:09, 1.63it/s]
DDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.63it/s]
DDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.63it/s]
DDIM Sampler: 35%|███▌ | 7/20 [00:04<00:07, 1.63it/s]
DDIM Sampler: 40%|████ | 8/20 [00:04<00:07, 1.63it/s]
DDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.63it/s]
DDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.63it/s]
DDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s]
DDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s]
DDIM Sampler: 65%|██████▌ | 13/20 [00:07<00:04, 1.63it/s]
DDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s]
DDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s]
DDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s]
DDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s]
DDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s]
DDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s]
DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]
DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]
Executing node 58, title: ToonCrafterDecode, class type: ToonCrafterDecode
VAE using dtype: torch.bfloat16
Using xformers
/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/conv.py:605: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)
return F.conv3d(
Using xformers
Executing node 67, title: Color Match, class type: ColorMatch
Executing node 29, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 59.29 seconds
outputs: {'6': {'text': ['3x512x512']}, '29': {'gifs': [{'filename': 'ToonCrafter_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
ToonCrafter_00001.png
ToonCrafter_00001.mp4This example was created by a different version, fofr/tooncrafter:51bf654d.
Examples
Run time and cost
This model costs approximately $0.074 to run on Replicate, or 13 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.
This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 53 seconds. The predict time for this model varies significantly based on the inputs.