fofr/tooncrafter

Create videos from illustrated input images

flan-t5-xl trained on the Memory Alpha Star Trek Wiki

gpt-j-6b trained on the Memory Alpha Star Trek Wiki

llama-7b trained on the Memory Alpha Star Trek Wiki

Generate image prompts for Midjourney. Prefix inputs with "Image: "

Split a video into frames

Convert a set of frames to a video

Create a waveform video from audio

A fine-tuned SDXL lora based on Tron Legacy

A fine-tuned SDXL lora based on the Barbie movie



Exploratory SDXL fine-tuning on text "FOFR"

SDXL fine-tuned on both Barbie and Tron Legacy

An SDXL fine-tune on Apple Vision Pro


An SDXL fine-tune based on Matrix Code art


Prediction
fofr/tooncrafter:51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40edIDgw8thp06j1rgp0cfvbtsvrq9vcStatusSucceededSourceWebHardwareA100 (40GB)Total durationCreatedby @fofrInput
- loop
- prompt
- max_width
- 512
- max_height
- 512
- interpolate
- negative_prompt
- color_correction
{ "loop": false, "prompt": "", "image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp", "image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp", "image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp", "max_width": 512, "max_height": 512, "interpolate": false, "negative_prompt": "", "color_correction": true }
Install Replicate’s Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; import fs from "node:fs"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run fofr/tooncrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "fofr/tooncrafter:51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40ed", { input: { loop: false, prompt: "", image_1: "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp", image_2: "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp", image_3: "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp", max_width: 512, max_height: 512, interpolate: false, negative_prompt: "", color_correction: true } } ); // To access the file URL: console.log(output[0].url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output[0]);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:pip install replicate
Import the client:import replicate
Run fofr/tooncrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "fofr/tooncrafter:51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40ed", input={ "loop": False, "prompt": "", "image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp", "image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp", "image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp", "max_width": 512, "max_height": 512, "interpolate": False, "negative_prompt": "", "color_correction": True } ) # To access the file URL: print(output[0].url()) #=> "http://example.com" # To write the file to disk: with open("my-image.png", "wb") as file: file.write(output[0].read())
To learn more, take a look at the guide on getting started with Python.
Run fofr/tooncrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "fofr/tooncrafter:51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40ed", "input": { "loop": false, "prompt": "", "image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp", "image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp", "image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp", "max_width": 512, "max_height": 512, "interpolate": false, "negative_prompt": "", "color_correction": true } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
You can run this model locally using Cog. First, install Cog:brew install cog
If you don’t have Homebrew, there are other installation options available.
Run this to download the model and run it in your local environment:
cog predict r8.im/fofr/tooncrafter@sha256:51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40ed \ -i 'loop=false' \ -i 'prompt=""' \ -i 'image_1="https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp"' \ -i 'image_2="https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp"' \ -i 'image_3="https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp"' \ -i 'max_width=512' \ -i 'max_height=512' \ -i 'interpolate=false' \ -i 'negative_prompt=""' \ -i 'color_correction=true'
To learn more, take a look at the Cog documentation.
Run this to download the model and run it in your local environment:
docker run -d -p 5000:5000 --gpus=all r8.im/fofr/tooncrafter@sha256:51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40ed
curl -s -X POST \ -H "Content-Type: application/json" \ -d $'{ "input": { "loop": false, "prompt": "", "image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp", "image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp", "image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp", "max_width": 512, "max_height": 512, "interpolate": false, "negative_prompt": "", "color_correction": true } }' \ http://localhost:5000/predictions
To learn more, take a look at the Cog documentation.
Output
{ "completed_at": "2024-06-02T21:59:40.667881Z", "created_at": "2024-06-02T21:57:22.192000Z", "data_removed": false, "error": null, "id": "gw8thp06j1rgp0cfvbtsvrq9vc", "input": { "loop": false, "prompt": "", "image_1": "https://replicate.delivery/pbxt/L1pQdyf4fPVRzU5WxhhHAdH2Eo05X3zhirvNzwAKJ80lA7Qh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-0.webp", "image_2": "https://replicate.delivery/pbxt/L1pQeBF582rKH3FFAYJCxdFUurBZ1axNFVwKxEd1wIALydhh/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-1.webp", "image_3": "https://replicate.delivery/pbxt/L1pQdTPwSZxnfDkPkM3eArBmHWd5xttTnSkKBhszXJ88pIff/replicate-prediction-5cvynz9d91rgg0cfsvqschdpww-3.webp", "max_width": 512, "max_height": 512, "interpolate": false, "negative_prompt": "", "color_correction": true }, "logs": "Random seed set to: 1500914532\nChecking inputs\n✅ /tmp/inputs/input_1.png\n✅ /tmp/inputs/input_2.png\n✅ /tmp/inputs/input_3.png\n====================================\nChecking weights\n✅ tooncrafter_512_interp-fp16.safetensors\n✅ stable-diffusion-2-1-clip-fp16.safetensors\n✅ CLIP-ViT-H-fp16.safetensors\n====================================\nRunning workflow\ngot prompt\nExecuting node 1, title: Load Image, class type: LoadImage\nDownloading model to: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors\nExecuting node 52, title: DownloadAndLoadDynamiCrafterModel, class type: DownloadAndLoadDynamiCrafterModel\nFetching 1 files: 0%| | 0/1 [00:00<?, ?it/s]/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`.\nFor more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.\nwarnings.warn(\nFetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it]\nFetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it]\nLoading model from: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors\nLatentVisualDiffusion: Running in v-prediction mode\nAE working on z of shape (1, 4, 32, 32) = 4096 dimensions.\nWorking with z of shape (1, 4, 32, 32) = 4096 dimensions.\nvanilla\nmaking attention of type 'vanilla' with 512 in_channels\nmemory-efficient-cross-attn-fusion\nmaking attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels\nmemory-efficient-cross-attn-fusion\nmaking attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels\n>>> model checkpoint loaded.\nModel using dtype: torch.float16\nExecuting node 61, title: DownloadAndLoadCLIPVisionModel, class type: DownloadAndLoadCLIPVisionModel\nLoading model from: /src/ComfyUI/models/clip_vision/CLIP-ViT-H-fp16.safetensors\nExecuting node 59, title: DownloadAndLoadCLIPModel, class type: DownloadAndLoadCLIPModel\nclip missing: ['text_model.encoder.layers.23.layer_norm1.weight', 'text_model.encoder.layers.23.layer_norm1.bias', 'text_model.encoder.layers.23.self_attn.q_proj.weight', 'text_model.encoder.layers.23.self_attn.q_proj.bias', 'text_model.encoder.layers.23.self_attn.k_proj.weight', 'text_model.encoder.layers.23.self_attn.k_proj.bias', 'text_model.encoder.layers.23.self_attn.v_proj.weight', 'text_model.encoder.layers.23.self_attn.v_proj.bias', 'text_model.encoder.layers.23.self_attn.out_proj.weight', 'text_model.encoder.layers.23.self_attn.out_proj.bias', 'text_model.encoder.layers.23.layer_norm2.weight', 'text_model.encoder.layers.23.layer_norm2.bias', 'text_model.encoder.layers.23.mlp.fc1.weight', 'text_model.encoder.layers.23.mlp.fc1.bias', 'text_model.encoder.layers.23.mlp.fc2.weight', 'text_model.encoder.layers.23.mlp.fc2.bias', 'text_projection.weight']\nLoading model from: /src/ComfyUI/models/clip/stable-diffusion-2-1-clip-fp16.safetensors\nRequested to load SD2ClipModel\nLoading 1 new model\nExecuting node 49, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode\nExecuting node 50, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode\nExecuting node 70, title: 🔧 Image Resize, class type: ImageResize+\nExecuting node 2, title: Load Image, class type: LoadImage\nExecuting node 303, title: Load Image, class type: LoadImage\nExecuting node 28, title: Image Batch Multi, class type: ImageBatchMulti\nExecuting node 6, title: Get Image Size & Count, class type: GetImageSizeAndCount\nExecuting node 65, title: 🔧 Image Resize, class type: ImageResize+\nExecuting node 57, title: ToonCrafterInterpolation, class type: ToonCrafterInterpolation\nVAE using dtype: torch.bfloat16\nRequested to load CLIPVisionModelProjection\nLoading 1 new model\nDDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s]\nDDIM Sampler: 5%|▌ | 1/20 [00:00<00:13, 1.37it/s]\nDDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.51it/s]\nDDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.56it/s]\nDDIM Sampler: 20%|██ | 4/20 [00:02<00:10, 1.59it/s]\nDDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.60it/s]\nDDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.61it/s]\nDDIM Sampler: 35%|███▌ | 7/20 [00:04<00:08, 1.62it/s]\nDDIM Sampler: 40%|████ | 8/20 [00:05<00:07, 1.62it/s]\nDDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.62it/s]\nDDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.62it/s]\nDDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s]\nDDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s]\nDDIM Sampler: 65%|██████▌ | 13/20 [00:08<00:04, 1.63it/s]\nDDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s]\nDDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s]\nDDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s]\nDDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s]\nDDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s]\nDDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.61it/s]\nDDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s]\nDDIM Sampler: 5%|▌ | 1/20 [00:00<00:11, 1.63it/s]\nDDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.63it/s]\nDDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.63it/s]\nDDIM Sampler: 20%|██ | 4/20 [00:02<00:09, 1.63it/s]\nDDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.63it/s]\nDDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.63it/s]\nDDIM Sampler: 35%|███▌ | 7/20 [00:04<00:07, 1.63it/s]\nDDIM Sampler: 40%|████ | 8/20 [00:04<00:07, 1.63it/s]\nDDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.63it/s]\nDDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.63it/s]\nDDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s]\nDDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s]\nDDIM Sampler: 65%|██████▌ | 13/20 [00:07<00:04, 1.63it/s]\nDDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s]\nDDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s]\nDDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s]\nDDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s]\nDDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s]\nDDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s]\nExecuting node 58, title: ToonCrafterDecode, class type: ToonCrafterDecode\nVAE using dtype: torch.bfloat16\nUsing xformers\n/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/conv.py:605: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)\nreturn F.conv3d(\nUsing xformers\nExecuting node 67, title: Color Match, class type: ColorMatch\nExecuting node 29, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\nPrompt executed in 59.29 seconds\noutputs: {'6': {'text': ['3x512x512']}, '29': {'gifs': [{'filename': 'ToonCrafter_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}\n====================================\nToonCrafter_00001.png\nToonCrafter_00001.mp4", "metrics": { "predict_time": 61.900853, "total_time": 138.475881 }, "output": [ "https://replicate.delivery/pbxt/aDDntBsXmmL6DZMIRURYbxjeer1PZxmT9TIl0MOfqcrYqT1lA/ToonCrafter_00001.mp4" ], "started_at": "2024-06-02T21:58:38.767028Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/gw8thp06j1rgp0cfvbtsvrq9vc", "cancel": "https://api.replicate.com/v1/predictions/gw8thp06j1rgp0cfvbtsvrq9vc/cancel" }, "version": "51bf654d60d307ab45c4ffe09546a3c9606f8f33861ab28f5bb0e43ad3fa40ed" }
Generated inRandom seed set to: 1500914532 Checking inputs ✅ /tmp/inputs/input_1.png ✅ /tmp/inputs/input_2.png ✅ /tmp/inputs/input_3.png ==================================== Checking weights ✅ tooncrafter_512_interp-fp16.safetensors ✅ stable-diffusion-2-1-clip-fp16.safetensors ✅ CLIP-ViT-H-fp16.safetensors ==================================== Running workflow got prompt Executing node 1, title: Load Image, class type: LoadImage Downloading model to: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors Executing node 52, title: DownloadAndLoadDynamiCrafterModel, class type: DownloadAndLoadDynamiCrafterModel Fetching 1 files: 0%| | 0/1 [00:00<?, ?it/s]/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`. For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder. warnings.warn( Fetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it] Fetching 1 files: 100%|██████████| 1/1 [00:13<00:00, 13.65s/it] Loading model from: /src/ComfyUI/models/checkpoints/dynamicrafter/tooncrafter_512_interp-fp16.safetensors LatentVisualDiffusion: Running in v-prediction mode AE working on z of shape (1, 4, 32, 32) = 4096 dimensions. Working with z of shape (1, 4, 32, 32) = 4096 dimensions. vanilla making attention of type 'vanilla' with 512 in_channels memory-efficient-cross-attn-fusion making attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels memory-efficient-cross-attn-fusion making attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels >>> model checkpoint loaded. Model using dtype: torch.float16 Executing node 61, title: DownloadAndLoadCLIPVisionModel, class type: DownloadAndLoadCLIPVisionModel Loading model from: /src/ComfyUI/models/clip_vision/CLIP-ViT-H-fp16.safetensors Executing node 59, title: DownloadAndLoadCLIPModel, class type: DownloadAndLoadCLIPModel clip missing: ['text_model.encoder.layers.23.layer_norm1.weight', 'text_model.encoder.layers.23.layer_norm1.bias', 'text_model.encoder.layers.23.self_attn.q_proj.weight', 'text_model.encoder.layers.23.self_attn.q_proj.bias', 'text_model.encoder.layers.23.self_attn.k_proj.weight', 'text_model.encoder.layers.23.self_attn.k_proj.bias', 'text_model.encoder.layers.23.self_attn.v_proj.weight', 'text_model.encoder.layers.23.self_attn.v_proj.bias', 'text_model.encoder.layers.23.self_attn.out_proj.weight', 'text_model.encoder.layers.23.self_attn.out_proj.bias', 'text_model.encoder.layers.23.layer_norm2.weight', 'text_model.encoder.layers.23.layer_norm2.bias', 'text_model.encoder.layers.23.mlp.fc1.weight', 'text_model.encoder.layers.23.mlp.fc1.bias', 'text_model.encoder.layers.23.mlp.fc2.weight', 'text_model.encoder.layers.23.mlp.fc2.bias', 'text_projection.weight'] Loading model from: /src/ComfyUI/models/clip/stable-diffusion-2-1-clip-fp16.safetensors Requested to load SD2ClipModel Loading 1 new model Executing node 49, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode Executing node 50, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode Executing node 70, title: 🔧 Image Resize, class type: ImageResize+ Executing node 2, title: Load Image, class type: LoadImage Executing node 303, title: Load Image, class type: LoadImage Executing node 28, title: Image Batch Multi, class type: ImageBatchMulti Executing node 6, title: Get Image Size & Count, class type: GetImageSizeAndCount Executing node 65, title: 🔧 Image Resize, class type: ImageResize+ Executing node 57, title: ToonCrafterInterpolation, class type: ToonCrafterInterpolation VAE using dtype: torch.bfloat16 Requested to load CLIPVisionModelProjection Loading 1 new model DDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s] DDIM Sampler: 5%|▌ | 1/20 [00:00<00:13, 1.37it/s] DDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.51it/s] DDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.56it/s] DDIM Sampler: 20%|██ | 4/20 [00:02<00:10, 1.59it/s] DDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.60it/s] DDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.61it/s] DDIM Sampler: 35%|███▌ | 7/20 [00:04<00:08, 1.62it/s] DDIM Sampler: 40%|████ | 8/20 [00:05<00:07, 1.62it/s] DDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.62it/s] DDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.62it/s] DDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s] DDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s] DDIM Sampler: 65%|██████▌ | 13/20 [00:08<00:04, 1.63it/s] DDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s] DDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s] DDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s] DDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s] DDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s] DDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s] DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s] DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.61it/s] DDIM Sampler: 0%| | 0/20 [00:00<?, ?it/s] DDIM Sampler: 5%|▌ | 1/20 [00:00<00:11, 1.63it/s] DDIM Sampler: 10%|█ | 2/20 [00:01<00:11, 1.63it/s] DDIM Sampler: 15%|█▌ | 3/20 [00:01<00:10, 1.63it/s] DDIM Sampler: 20%|██ | 4/20 [00:02<00:09, 1.63it/s] DDIM Sampler: 25%|██▌ | 5/20 [00:03<00:09, 1.63it/s] DDIM Sampler: 30%|███ | 6/20 [00:03<00:08, 1.63it/s] DDIM Sampler: 35%|███▌ | 7/20 [00:04<00:07, 1.63it/s] DDIM Sampler: 40%|████ | 8/20 [00:04<00:07, 1.63it/s] DDIM Sampler: 45%|████▌ | 9/20 [00:05<00:06, 1.63it/s] DDIM Sampler: 50%|█████ | 10/20 [00:06<00:06, 1.63it/s] DDIM Sampler: 55%|█████▌ | 11/20 [00:06<00:05, 1.63it/s] DDIM Sampler: 60%|██████ | 12/20 [00:07<00:04, 1.63it/s] DDIM Sampler: 65%|██████▌ | 13/20 [00:07<00:04, 1.63it/s] DDIM Sampler: 70%|███████ | 14/20 [00:08<00:03, 1.63it/s] DDIM Sampler: 75%|███████▌ | 15/20 [00:09<00:03, 1.63it/s] DDIM Sampler: 80%|████████ | 16/20 [00:09<00:02, 1.63it/s] DDIM Sampler: 85%|████████▌ | 17/20 [00:10<00:01, 1.63it/s] DDIM Sampler: 90%|█████████ | 18/20 [00:11<00:01, 1.63it/s] DDIM Sampler: 95%|█████████▌| 19/20 [00:11<00:00, 1.63it/s] DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s] DDIM Sampler: 100%|██████████| 20/20 [00:12<00:00, 1.63it/s] Executing node 58, title: ToonCrafterDecode, class type: ToonCrafterDecode VAE using dtype: torch.bfloat16 Using xformers /root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/conv.py:605: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.) return F.conv3d( Using xformers Executing node 67, title: Color Match, class type: ColorMatch Executing node 29, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine Prompt executed in 59.29 seconds outputs: {'6': {'text': ['3x512x512']}, '29': {'gifs': [{'filename': 'ToonCrafter_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}} ==================================== ToonCrafter_00001.png ToonCrafter_00001.mp4
Want to make some of these yourself?
Run this model