cloneofsimo / hotshot-xl-lora-controlnet
Text-to-gif using SDXL, with controlnet and lora support
Prediction
cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568bIDve5nvsdbvn4fk2cp6iorzzcj3iStatusSucceededSourceWebHardwareA40 (Large)Total durationCreatedInput
- steps
- 30
- width
- 512
- height
- 512
- prompt
- homer simpson facepalms in a star trek uniform
- control_type
- depth
- target_width
- 1024
- video_length
- 8
- target_height
- 1024
- original_width
- 1920
- video_duration
- 1000
- negative_prompt
- blurry, soft, black and white, distorted, broken, ugly, weird
- original_height
- 1080
- control_guidance_end
- 0.7
- replicate_weights_url
- control_guidance_start
- 0
- controlnet_conditioning_scale
- 0.8
{ "gif": "https://replicate.delivery/pbxt/JeI4fCMjTroJ6zw3rC7Ur9CW9ZPoKRNrQGURf4S4qZgRexuk/patrick-stewart-facepalm-gif.gif", "steps": 30, "width": 512, "height": 512, "prompt": "homer simpson facepalms in a star trek uniform", "control_type": "depth", "target_width": 1024, "video_length": 8, "target_height": 1024, "original_width": 1920, "video_duration": 1000, "negative_prompt": "blurry, soft, black and white, distorted, broken, ugly, weird", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 }
Install Replicate’s Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", { input: { gif: "https://replicate.delivery/pbxt/JeI4fCMjTroJ6zw3rC7Ur9CW9ZPoKRNrQGURf4S4qZgRexuk/patrick-stewart-facepalm-gif.gif", steps: 30, width: 512, height: 512, prompt: "homer simpson facepalms in a star trek uniform", control_type: "depth", target_width: 1024, video_length: 8, target_height: 1024, original_width: 1920, video_duration: 1000, negative_prompt: "blurry, soft, black and white, distorted, broken, ugly, weird", original_height: 1080, control_guidance_end: 0.7, replicate_weights_url: "", control_guidance_start: 0, controlnet_conditioning_scale: 0.8 } } ); // To access the file URL: console.log(output.url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:pip install replicate
Import the client:import replicate
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", input={ "gif": "https://replicate.delivery/pbxt/JeI4fCMjTroJ6zw3rC7Ur9CW9ZPoKRNrQGURf4S4qZgRexuk/patrick-stewart-facepalm-gif.gif", "steps": 30, "width": 512, "height": 512, "prompt": "homer simpson facepalms in a star trek uniform", "control_type": "depth", "target_width": 1024, "video_length": 8, "target_height": 1024, "original_width": 1920, "video_duration": 1000, "negative_prompt": "blurry, soft, black and white, distorted, broken, ugly, weird", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 } ) print(output)
To learn more, take a look at the guide on getting started with Python.
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", "input": { "gif": "https://replicate.delivery/pbxt/JeI4fCMjTroJ6zw3rC7Ur9CW9ZPoKRNrQGURf4S4qZgRexuk/patrick-stewart-facepalm-gif.gif", "steps": 30, "width": 512, "height": 512, "prompt": "homer simpson facepalms in a star trek uniform", "control_type": "depth", "target_width": 1024, "video_length": 8, "target_height": 1024, "original_width": 1920, "video_duration": 1000, "negative_prompt": "blurry, soft, black and white, distorted, broken, ugly, weird", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
Output
{ "completed_at": "2023-10-05T22:24:22.812445Z", "created_at": "2023-10-05T22:23:44.900061Z", "data_removed": false, "error": null, "id": "ve5nvsdbvn4fk2cp6iorzzcj3i", "input": { "gif": "https://replicate.delivery/pbxt/JeI4fCMjTroJ6zw3rC7Ur9CW9ZPoKRNrQGURf4S4qZgRexuk/patrick-stewart-facepalm-gif.gif", "steps": 30, "width": 512, "height": 512, "prompt": "homer simpson facepalms in a star trek uniform", "control_type": "depth", "target_width": 1024, "video_length": 8, "target_height": 1024, "original_width": 1920, "video_duration": 1000, "negative_prompt": "blurry, soft, black and white, distorted, broken, ugly, weird", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 }, "logs": "Loading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s]\nLoading pipeline components...: 14%|█▍ | 1/7 [00:02<00:15, 2.63s/it]\nLoading pipeline components...: 29%|██▊ | 2/7 [00:03<00:07, 1.45s/it]\nLoading pipeline components...: 86%|████████▌ | 6/7 [00:03<00:00, 2.69it/s]\nLoading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 2.90it/s]\nLoading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 1.89it/s]\n 0%| | 0/30 [00:00<?, ?it/s]\n 3%|▎ | 1/30 [00:00<00:16, 1.72it/s]\n 7%|▋ | 2/30 [00:01<00:19, 1.45it/s]\n 10%|█ | 3/30 [00:02<00:19, 1.38it/s]\n 13%|█▎ | 4/30 [00:02<00:19, 1.36it/s]\n 17%|█▋ | 5/30 [00:03<00:18, 1.34it/s]\n 20%|██ | 6/30 [00:04<00:18, 1.33it/s]\n 23%|██▎ | 7/30 [00:05<00:17, 1.32it/s]\n 27%|██▋ | 8/30 [00:05<00:16, 1.32it/s]\n 30%|███ | 9/30 [00:06<00:15, 1.32it/s]\n 33%|███▎ | 10/30 [00:07<00:15, 1.32it/s]\n 37%|███▋ | 11/30 [00:08<00:14, 1.31it/s]\n 40%|████ | 12/30 [00:08<00:13, 1.31it/s]\n 43%|████▎ | 13/30 [00:09<00:12, 1.31it/s]\n 47%|████▋ | 14/30 [00:10<00:12, 1.31it/s]\n 50%|█████ | 15/30 [00:11<00:11, 1.31it/s]\n 53%|█████▎ | 16/30 [00:12<00:10, 1.31it/s]\n 57%|█████▋ | 17/30 [00:12<00:09, 1.31it/s]\n 60%|██████ | 18/30 [00:13<00:09, 1.31it/s]\n 63%|██████▎ | 19/30 [00:14<00:08, 1.31it/s]\n 67%|██████▋ | 20/30 [00:15<00:07, 1.31it/s]\n 70%|███████ | 21/30 [00:15<00:06, 1.31it/s]\n 73%|███████▎ | 22/30 [00:16<00:06, 1.31it/s]\n 77%|███████▋ | 23/30 [00:17<00:05, 1.31it/s]\n 80%|████████ | 24/30 [00:18<00:04, 1.31it/s]\n 83%|████████▎ | 25/30 [00:18<00:03, 1.31it/s]\n 87%|████████▋ | 26/30 [00:19<00:03, 1.31it/s]\n 90%|█████████ | 27/30 [00:20<00:02, 1.31it/s]\n 93%|█████████▎| 28/30 [00:21<00:01, 1.31it/s]\n 97%|█████████▋| 29/30 [00:21<00:00, 1.31it/s]\n100%|██████████| 30/30 [00:22<00:00, 1.31it/s]\n100%|██████████| 30/30 [00:22<00:00, 1.32it/s]\n 0%| | 0/8 [00:00<?, ?it/s]\n 25%|██▌ | 2/8 [00:00<00:00, 12.16it/s]\n 50%|█████ | 4/8 [00:00<00:00, 13.96it/s]\n 75%|███████▌ | 6/8 [00:00<00:00, 14.22it/s]\n100%|██████████| 8/8 [00:00<00:00, 14.34it/s]\n100%|██████████| 8/8 [00:00<00:00, 14.08it/s]", "metrics": { "predict_time": 37.895191, "total_time": 37.912384 }, "output": "https://pbxt.replicate.delivery/ZLI9qg2tMbIWDhEfabfxZAoi8024GmKAXkcg0VOF5F7VmOrRA/tmp.gif", "started_at": "2023-10-05T22:23:44.917254Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/ve5nvsdbvn4fk2cp6iorzzcj3i", "cancel": "https://api.replicate.com/v1/predictions/ve5nvsdbvn4fk2cp6iorzzcj3i/cancel" }, "version": "75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b" }
Generated inLoading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s] Loading pipeline components...: 14%|█▍ | 1/7 [00:02<00:15, 2.63s/it] Loading pipeline components...: 29%|██▊ | 2/7 [00:03<00:07, 1.45s/it] Loading pipeline components...: 86%|████████▌ | 6/7 [00:03<00:00, 2.69it/s] Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 2.90it/s] Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 1.89it/s] 0%| | 0/30 [00:00<?, ?it/s] 3%|▎ | 1/30 [00:00<00:16, 1.72it/s] 7%|▋ | 2/30 [00:01<00:19, 1.45it/s] 10%|█ | 3/30 [00:02<00:19, 1.38it/s] 13%|█▎ | 4/30 [00:02<00:19, 1.36it/s] 17%|█▋ | 5/30 [00:03<00:18, 1.34it/s] 20%|██ | 6/30 [00:04<00:18, 1.33it/s] 23%|██▎ | 7/30 [00:05<00:17, 1.32it/s] 27%|██▋ | 8/30 [00:05<00:16, 1.32it/s] 30%|███ | 9/30 [00:06<00:15, 1.32it/s] 33%|███▎ | 10/30 [00:07<00:15, 1.32it/s] 37%|███▋ | 11/30 [00:08<00:14, 1.31it/s] 40%|████ | 12/30 [00:08<00:13, 1.31it/s] 43%|████▎ | 13/30 [00:09<00:12, 1.31it/s] 47%|████▋ | 14/30 [00:10<00:12, 1.31it/s] 50%|█████ | 15/30 [00:11<00:11, 1.31it/s] 53%|█████▎ | 16/30 [00:12<00:10, 1.31it/s] 57%|█████▋ | 17/30 [00:12<00:09, 1.31it/s] 60%|██████ | 18/30 [00:13<00:09, 1.31it/s] 63%|██████▎ | 19/30 [00:14<00:08, 1.31it/s] 67%|██████▋ | 20/30 [00:15<00:07, 1.31it/s] 70%|███████ | 21/30 [00:15<00:06, 1.31it/s] 73%|███████▎ | 22/30 [00:16<00:06, 1.31it/s] 77%|███████▋ | 23/30 [00:17<00:05, 1.31it/s] 80%|████████ | 24/30 [00:18<00:04, 1.31it/s] 83%|████████▎ | 25/30 [00:18<00:03, 1.31it/s] 87%|████████▋ | 26/30 [00:19<00:03, 1.31it/s] 90%|█████████ | 27/30 [00:20<00:02, 1.31it/s] 93%|█████████▎| 28/30 [00:21<00:01, 1.31it/s] 97%|█████████▋| 29/30 [00:21<00:00, 1.31it/s] 100%|██████████| 30/30 [00:22<00:00, 1.31it/s] 100%|██████████| 30/30 [00:22<00:00, 1.32it/s] 0%| | 0/8 [00:00<?, ?it/s] 25%|██▌ | 2/8 [00:00<00:00, 12.16it/s] 50%|█████ | 4/8 [00:00<00:00, 13.96it/s] 75%|███████▌ | 6/8 [00:00<00:00, 14.22it/s] 100%|██████████| 8/8 [00:00<00:00, 14.34it/s] 100%|██████████| 8/8 [00:00<00:00, 14.08it/s]
Prediction
cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568bID2kx22etbicnss47wjmi6zlessyStatusSucceededSourceWebHardwareA40 (Large)Total durationCreatedInput
- steps
- 30
- width
- 672
- height
- 384
- prompt
- SHRMI dog dancing in mars
- control_type
- depth
- target_width
- 512
- video_length
- 8
- target_height
- 512
- original_width
- 1920
- video_duration
- 1000
- negative_prompt
- original_height
- 1080
- control_guidance_end
- 1
- replicate_weights_url
- https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?
- control_guidance_start
- 0
- controlnet_conditioning_scale
- 1
{ "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in mars", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 }
Install Replicate’s Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", { input: { gif: "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", steps: 30, width: 672, height: 384, prompt: "SHRMI dog dancing in mars", control_type: "depth", target_width: 512, video_length: 8, target_height: 512, original_width: 1920, video_duration: 1000, negative_prompt: "", original_height: 1080, control_guidance_end: 1, replicate_weights_url: "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", control_guidance_start: 0, controlnet_conditioning_scale: 1 } } ); // To access the file URL: console.log(output.url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:pip install replicate
Import the client:import replicate
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", input={ "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in mars", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 } ) print(output)
To learn more, take a look at the guide on getting started with Python.
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", "input": { "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in mars", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
Output
{ "completed_at": "2023-10-06T02:31:59.944042Z", "created_at": "2023-10-06T02:31:22.999709Z", "data_removed": false, "error": null, "id": "2kx22etbicnss47wjmi6zlessy", "input": { "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in mars", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 }, "logs": "Loading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s]\nLoading pipeline components...: 14%|█▍ | 1/7 [00:02<00:15, 2.55s/it]\nLoading pipeline components...: 71%|███████▏ | 5/7 [00:03<00:01, 1.87it/s]\nLoading pipeline components...: 86%|████████▌ | 6/7 [00:03<00:00, 2.27it/s]\nLoading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 2.59it/s]\nLoading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 1.93it/s]\nLoading Unet LoRA\nYou have saved the LoRA weights using the old format. To convert the old LoRA weights to the new format, you can first load them in a dictionary and then create a new dictionary like the following: `new_state_dict = {f'unet.{module_name}': params for module_name, params in old_state_dict.items()}`.\n 0%| | 0/30 [00:00<?, ?it/s]\n 3%|▎ | 1/30 [00:00<00:21, 1.32it/s]\n 7%|▋ | 2/30 [00:01<00:23, 1.17it/s]\n 10%|█ | 3/30 [00:02<00:23, 1.13it/s]\n 13%|█▎ | 4/30 [00:03<00:23, 1.11it/s]\n 17%|█▋ | 5/30 [00:04<00:22, 1.10it/s]\n 20%|██ | 6/30 [00:05<00:21, 1.09it/s]\n 23%|██▎ | 7/30 [00:06<00:21, 1.09it/s]\n 27%|██▋ | 8/30 [00:07<00:20, 1.09it/s]\n 30%|███ | 9/30 [00:08<00:19, 1.08it/s]\n 33%|███▎ | 10/30 [00:09<00:18, 1.08it/s]\n 37%|███▋ | 11/30 [00:10<00:17, 1.08it/s]\n 40%|████ | 12/30 [00:10<00:16, 1.08it/s]\n 43%|████▎ | 13/30 [00:11<00:15, 1.08it/s]\n 47%|████▋ | 14/30 [00:12<00:14, 1.08it/s]\n 50%|█████ | 15/30 [00:13<00:13, 1.08it/s]\n 53%|█████▎ | 16/30 [00:14<00:12, 1.08it/s]\n 57%|█████▋ | 17/30 [00:15<00:12, 1.08it/s]\n 60%|██████ | 18/30 [00:16<00:11, 1.08it/s]\n 63%|██████▎ | 19/30 [00:17<00:10, 1.08it/s]\n 67%|██████▋ | 20/30 [00:18<00:09, 1.08it/s]\n 70%|███████ | 21/30 [00:19<00:08, 1.08it/s]\n 73%|███████▎ | 22/30 [00:20<00:07, 1.08it/s]\n 77%|███████▋ | 23/30 [00:21<00:06, 1.08it/s]\n 80%|████████ | 24/30 [00:22<00:05, 1.08it/s]\n 83%|████████▎ | 25/30 [00:22<00:04, 1.08it/s]\n 87%|████████▋ | 26/30 [00:23<00:03, 1.08it/s]\n 90%|█████████ | 27/30 [00:24<00:02, 1.08it/s]\n 93%|█████████▎| 28/30 [00:25<00:01, 1.08it/s]\n 97%|█████████▋| 29/30 [00:26<00:00, 1.08it/s]\n100%|██████████| 30/30 [00:27<00:00, 1.08it/s]\n100%|██████████| 30/30 [00:27<00:00, 1.09it/s]\n 0%| | 0/8 [00:00<?, ?it/s]\n 25%|██▌ | 2/8 [00:00<00:00, 13.42it/s]\n 50%|█████ | 4/8 [00:00<00:00, 14.81it/s]\n 75%|███████▌ | 6/8 [00:00<00:00, 14.86it/s]\n100%|██████████| 8/8 [00:00<00:00, 14.88it/s]\n100%|██████████| 8/8 [00:00<00:00, 14.74it/s]", "metrics": { "predict_time": 36.958822, "total_time": 36.944333 }, "output": "https://pbxt.replicate.delivery/No1zxXpqtQZdEtael3C9RRH12aLRorQEg9AhmrV5gWVPHp1IA/tmp.gif", "started_at": "2023-10-06T02:31:22.985220Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/2kx22etbicnss47wjmi6zlessy", "cancel": "https://api.replicate.com/v1/predictions/2kx22etbicnss47wjmi6zlessy/cancel" }, "version": "75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b" }
Generated inLoading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s] Loading pipeline components...: 14%|█▍ | 1/7 [00:02<00:15, 2.55s/it] Loading pipeline components...: 71%|███████▏ | 5/7 [00:03<00:01, 1.87it/s] Loading pipeline components...: 86%|████████▌ | 6/7 [00:03<00:00, 2.27it/s] Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 2.59it/s] Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 1.93it/s] Loading Unet LoRA You have saved the LoRA weights using the old format. To convert the old LoRA weights to the new format, you can first load them in a dictionary and then create a new dictionary like the following: `new_state_dict = {f'unet.{module_name}': params for module_name, params in old_state_dict.items()}`. 0%| | 0/30 [00:00<?, ?it/s] 3%|▎ | 1/30 [00:00<00:21, 1.32it/s] 7%|▋ | 2/30 [00:01<00:23, 1.17it/s] 10%|█ | 3/30 [00:02<00:23, 1.13it/s] 13%|█▎ | 4/30 [00:03<00:23, 1.11it/s] 17%|█▋ | 5/30 [00:04<00:22, 1.10it/s] 20%|██ | 6/30 [00:05<00:21, 1.09it/s] 23%|██▎ | 7/30 [00:06<00:21, 1.09it/s] 27%|██▋ | 8/30 [00:07<00:20, 1.09it/s] 30%|███ | 9/30 [00:08<00:19, 1.08it/s] 33%|███▎ | 10/30 [00:09<00:18, 1.08it/s] 37%|███▋ | 11/30 [00:10<00:17, 1.08it/s] 40%|████ | 12/30 [00:10<00:16, 1.08it/s] 43%|████▎ | 13/30 [00:11<00:15, 1.08it/s] 47%|████▋ | 14/30 [00:12<00:14, 1.08it/s] 50%|█████ | 15/30 [00:13<00:13, 1.08it/s] 53%|█████▎ | 16/30 [00:14<00:12, 1.08it/s] 57%|█████▋ | 17/30 [00:15<00:12, 1.08it/s] 60%|██████ | 18/30 [00:16<00:11, 1.08it/s] 63%|██████▎ | 19/30 [00:17<00:10, 1.08it/s] 67%|██████▋ | 20/30 [00:18<00:09, 1.08it/s] 70%|███████ | 21/30 [00:19<00:08, 1.08it/s] 73%|███████▎ | 22/30 [00:20<00:07, 1.08it/s] 77%|███████▋ | 23/30 [00:21<00:06, 1.08it/s] 80%|████████ | 24/30 [00:22<00:05, 1.08it/s] 83%|████████▎ | 25/30 [00:22<00:04, 1.08it/s] 87%|████████▋ | 26/30 [00:23<00:03, 1.08it/s] 90%|█████████ | 27/30 [00:24<00:02, 1.08it/s] 93%|█████████▎| 28/30 [00:25<00:01, 1.08it/s] 97%|█████████▋| 29/30 [00:26<00:00, 1.08it/s] 100%|██████████| 30/30 [00:27<00:00, 1.08it/s] 100%|██████████| 30/30 [00:27<00:00, 1.09it/s] 0%| | 0/8 [00:00<?, ?it/s] 25%|██▌ | 2/8 [00:00<00:00, 13.42it/s] 50%|█████ | 4/8 [00:00<00:00, 14.81it/s] 75%|███████▌ | 6/8 [00:00<00:00, 14.86it/s] 100%|██████████| 8/8 [00:00<00:00, 14.88it/s] 100%|██████████| 8/8 [00:00<00:00, 14.74it/s]
Prediction
cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568bIDvhiozkdbpzryxt3fkq63oakgiaStatusSucceededSourceWebHardwareA40 (Large)Total durationCreatedInput
- steps
- 30
- width
- 672
- height
- 384
- prompt
- SHRMI dog dancing in space, colorful galaxies on background
- control_type
- depth
- target_width
- 512
- video_length
- 8
- target_height
- 512
- original_width
- 1920
- video_duration
- 1000
- negative_prompt
- original_height
- 1080
- control_guidance_end
- 1
- replicate_weights_url
- https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?
- control_guidance_start
- 0
- controlnet_conditioning_scale
- 1
{ "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in space, colorful galaxies on background", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 }
Install Replicate’s Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", { input: { gif: "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", steps: 30, width: 672, height: 384, prompt: "SHRMI dog dancing in space, colorful galaxies on background", control_type: "depth", target_width: 512, video_length: 8, target_height: 512, original_width: 1920, video_duration: 1000, negative_prompt: "", original_height: 1080, control_guidance_end: 1, replicate_weights_url: "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", control_guidance_start: 0, controlnet_conditioning_scale: 1 } } ); // To access the file URL: console.log(output.url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:pip install replicate
Import the client:import replicate
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", input={ "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in space, colorful galaxies on background", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 } ) print(output)
To learn more, take a look at the guide on getting started with Python.
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "cloneofsimo/hotshot-xl-lora-controlnet:75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b", "input": { "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in space, colorful galaxies on background", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
Output
{ "completed_at": "2023-10-06T02:32:59.788561Z", "created_at": "2023-10-06T02:32:22.723245Z", "data_removed": false, "error": null, "id": "vhiozkdbpzryxt3fkq63oakgia", "input": { "gif": "https://replicate.delivery/pbxt/JeGg0LkufHuNpTwb8fVpXPAW7M6B6GYB1WfFIuy8LuJWwBNp/tmp2.gif", "steps": 30, "width": 672, "height": 384, "prompt": "SHRMI dog dancing in space, colorful galaxies on background", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 1, "replicate_weights_url": "https://pbxt.replicate.delivery/i2HfjZWkJC1yIqHPDaPHOx9eJsqFqZ2tnWMgn5FFd9evSGWjA/trained_model.tar?", "control_guidance_start": 0, "controlnet_conditioning_scale": 1 }, "logs": "Loading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s]\nLoading pipeline components...: 14%|█▍ | 1/7 [00:02<00:15, 2.63s/it]\nLoading pipeline components...: 71%|███████▏ | 5/7 [00:03<00:01, 1.87it/s]\nLoading pipeline components...: 86%|████████▌ | 6/7 [00:03<00:00, 2.27it/s]\nLoading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 2.59it/s]\nLoading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 1.91it/s]\nLoading Unet LoRA\nYou have saved the LoRA weights using the old format. To convert the old LoRA weights to the new format, you can first load them in a dictionary and then create a new dictionary like the following: `new_state_dict = {f'unet.{module_name}': params for module_name, params in old_state_dict.items()}`.\n 0%| | 0/30 [00:00<?, ?it/s]\n 3%|▎ | 1/30 [00:00<00:22, 1.31it/s]\n 7%|▋ | 2/30 [00:01<00:24, 1.17it/s]\n 10%|█ | 3/30 [00:02<00:24, 1.12it/s]\n 13%|█▎ | 4/30 [00:03<00:23, 1.11it/s]\n 17%|█▋ | 5/30 [00:04<00:22, 1.10it/s]\n 20%|██ | 6/30 [00:05<00:22, 1.09it/s]\n 23%|██▎ | 7/30 [00:06<00:21, 1.09it/s]\n 27%|██▋ | 8/30 [00:07<00:20, 1.09it/s]\n 30%|███ | 9/30 [00:08<00:19, 1.08it/s]\n 33%|███▎ | 10/30 [00:09<00:18, 1.08it/s]\n 37%|███▋ | 11/30 [00:10<00:17, 1.08it/s]\n 40%|████ | 12/30 [00:10<00:16, 1.08it/s]\n 43%|████▎ | 13/30 [00:11<00:15, 1.08it/s]\n 47%|████▋ | 14/30 [00:12<00:14, 1.08it/s]\n 50%|█████ | 15/30 [00:13<00:13, 1.08it/s]\n 53%|█████▎ | 16/30 [00:14<00:12, 1.08it/s]\n 57%|█████▋ | 17/30 [00:15<00:12, 1.08it/s]\n 60%|██████ | 18/30 [00:16<00:11, 1.08it/s]\n 63%|██████▎ | 19/30 [00:17<00:10, 1.08it/s]\n 67%|██████▋ | 20/30 [00:18<00:09, 1.08it/s]\n 70%|███████ | 21/30 [00:19<00:08, 1.08it/s]\n 73%|███████▎ | 22/30 [00:20<00:07, 1.08it/s]\n 77%|███████▋ | 23/30 [00:21<00:06, 1.08it/s]\n 80%|████████ | 24/30 [00:22<00:05, 1.08it/s]\n 83%|████████▎ | 25/30 [00:22<00:04, 1.08it/s]\n 87%|████████▋ | 26/30 [00:23<00:03, 1.08it/s]\n 90%|█████████ | 27/30 [00:24<00:02, 1.08it/s]\n 93%|█████████▎| 28/30 [00:25<00:01, 1.08it/s]\n 97%|█████████▋| 29/30 [00:26<00:00, 1.08it/s]\n100%|██████████| 30/30 [00:27<00:00, 1.08it/s]\n100%|██████████| 30/30 [00:27<00:00, 1.09it/s]\n 0%| | 0/8 [00:00<?, ?it/s]\n 25%|██▌ | 2/8 [00:00<00:00, 13.40it/s]\n 50%|█████ | 4/8 [00:00<00:00, 14.77it/s]\n 75%|███████▌ | 6/8 [00:00<00:00, 14.81it/s]\n100%|██████████| 8/8 [00:00<00:00, 14.84it/s]\n100%|██████████| 8/8 [00:00<00:00, 14.70it/s]", "metrics": { "predict_time": 37.093352, "total_time": 37.065316 }, "output": "https://pbxt.replicate.delivery/sGBhVb69oZYaA1qG5D5R2tOZUNwhgH7FdbeQjJn8n0GtHp1IA/tmp.gif", "started_at": "2023-10-06T02:32:22.695209Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/vhiozkdbpzryxt3fkq63oakgia", "cancel": "https://api.replicate.com/v1/predictions/vhiozkdbpzryxt3fkq63oakgia/cancel" }, "version": "75e26ffd033a59a78954a3d675632f47f7f8470402aec51c255b9f9b7b62568b" }
Generated inLoading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s] Loading pipeline components...: 14%|█▍ | 1/7 [00:02<00:15, 2.63s/it] Loading pipeline components...: 71%|███████▏ | 5/7 [00:03<00:01, 1.87it/s] Loading pipeline components...: 86%|████████▌ | 6/7 [00:03<00:00, 2.27it/s] Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 2.59it/s] Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00, 1.91it/s] Loading Unet LoRA You have saved the LoRA weights using the old format. To convert the old LoRA weights to the new format, you can first load them in a dictionary and then create a new dictionary like the following: `new_state_dict = {f'unet.{module_name}': params for module_name, params in old_state_dict.items()}`. 0%| | 0/30 [00:00<?, ?it/s] 3%|▎ | 1/30 [00:00<00:22, 1.31it/s] 7%|▋ | 2/30 [00:01<00:24, 1.17it/s] 10%|█ | 3/30 [00:02<00:24, 1.12it/s] 13%|█▎ | 4/30 [00:03<00:23, 1.11it/s] 17%|█▋ | 5/30 [00:04<00:22, 1.10it/s] 20%|██ | 6/30 [00:05<00:22, 1.09it/s] 23%|██▎ | 7/30 [00:06<00:21, 1.09it/s] 27%|██▋ | 8/30 [00:07<00:20, 1.09it/s] 30%|███ | 9/30 [00:08<00:19, 1.08it/s] 33%|███▎ | 10/30 [00:09<00:18, 1.08it/s] 37%|███▋ | 11/30 [00:10<00:17, 1.08it/s] 40%|████ | 12/30 [00:10<00:16, 1.08it/s] 43%|████▎ | 13/30 [00:11<00:15, 1.08it/s] 47%|████▋ | 14/30 [00:12<00:14, 1.08it/s] 50%|█████ | 15/30 [00:13<00:13, 1.08it/s] 53%|█████▎ | 16/30 [00:14<00:12, 1.08it/s] 57%|█████▋ | 17/30 [00:15<00:12, 1.08it/s] 60%|██████ | 18/30 [00:16<00:11, 1.08it/s] 63%|██████▎ | 19/30 [00:17<00:10, 1.08it/s] 67%|██████▋ | 20/30 [00:18<00:09, 1.08it/s] 70%|███████ | 21/30 [00:19<00:08, 1.08it/s] 73%|███████▎ | 22/30 [00:20<00:07, 1.08it/s] 77%|███████▋ | 23/30 [00:21<00:06, 1.08it/s] 80%|████████ | 24/30 [00:22<00:05, 1.08it/s] 83%|████████▎ | 25/30 [00:22<00:04, 1.08it/s] 87%|████████▋ | 26/30 [00:23<00:03, 1.08it/s] 90%|█████████ | 27/30 [00:24<00:02, 1.08it/s] 93%|█████████▎| 28/30 [00:25<00:01, 1.08it/s] 97%|█████████▋| 29/30 [00:26<00:00, 1.08it/s] 100%|██████████| 30/30 [00:27<00:00, 1.08it/s] 100%|██████████| 30/30 [00:27<00:00, 1.09it/s] 0%| | 0/8 [00:00<?, ?it/s] 25%|██▌ | 2/8 [00:00<00:00, 13.40it/s] 50%|█████ | 4/8 [00:00<00:00, 14.77it/s] 75%|███████▌ | 6/8 [00:00<00:00, 14.81it/s] 100%|██████████| 8/8 [00:00<00:00, 14.84it/s] 100%|██████████| 8/8 [00:00<00:00, 14.70it/s]
Prediction
cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63IDfgfofdlbkrol52zhx3nvzzm4nqStatusSucceededSourceWebHardwareA40 (Large)Total durationCreatedInput
- gif
- null
- seed
- 1001
- steps
- 30
- width
- 672
- height
- 384
- prompt
- a close up of an anime character casting a spell
- hf_lora_url
- null
- control_type
- depth
- target_width
- 512
- video_length
- 8
- target_height
- 512
- original_width
- 1920
- video_duration
- 1000
- negative_prompt
- original_height
- 1080
- control_guidance_end
- 0.7
- replicate_weights_url
- control_guidance_start
- 0
- controlnet_conditioning_scale
- 0.8
{ "gif": null, "seed": 1001, "steps": 30, "width": 672, "height": 384, "prompt": "a close up of an anime character casting a spell", "hf_lora_url": null, "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 }
Install Replicate’s Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63", { input: { seed: 1001, steps: 30, width: 672, height: 384, prompt: "a close up of an anime character casting a spell", control_type: "depth", target_width: 512, video_length: 8, target_height: 512, original_width: 1920, video_duration: 1000, negative_prompt: "", original_height: 1080, control_guidance_end: 0.7, replicate_weights_url: "", control_guidance_start: 0, controlnet_conditioning_scale: 0.8 } } ); // To access the file URL: console.log(output.url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:pip install replicate
Import the client:import replicate
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63", input={ "seed": 1001, "steps": 30, "width": 672, "height": 384, "prompt": "a close up of an anime character casting a spell", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 } ) print(output)
To learn more, take a look at the guide on getting started with Python.
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63", "input": { "seed": 1001, "steps": 30, "width": 672, "height": 384, "prompt": "a close up of an anime character casting a spell", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
Output
{ "completed_at": "2023-10-10T09:47:37.788732Z", "created_at": "2023-10-10T09:47:14.486650Z", "data_removed": false, "error": null, "id": "fgfofdlbkrol52zhx3nvzzm4nq", "input": { "gif": null, "seed": 1001, "steps": 30, "width": 672, "height": 384, "prompt": "a close up of an anime character casting a spell", "hf_lora_url": null, "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 }, "logs": "Warning - setting num_images_per_prompt = 1 because video_length = 8\n 0%| | 0/30 [00:00<?, ?it/s]\n 3%|▎ | 1/30 [00:00<00:19, 1.45it/s]\n 7%|▋ | 2/30 [00:01<00:19, 1.46it/s]\n 10%|█ | 3/30 [00:02<00:18, 1.46it/s]\n 13%|█▎ | 4/30 [00:02<00:17, 1.46it/s]\n 17%|█▋ | 5/30 [00:03<00:17, 1.45it/s]\n 20%|██ | 6/30 [00:04<00:16, 1.45it/s]\n 23%|██▎ | 7/30 [00:04<00:15, 1.45it/s]\n 27%|██▋ | 8/30 [00:05<00:15, 1.45it/s]\n 30%|███ | 9/30 [00:06<00:14, 1.45it/s]\n 33%|███▎ | 10/30 [00:06<00:13, 1.45it/s]\n 37%|███▋ | 11/30 [00:07<00:13, 1.45it/s]\n 40%|████ | 12/30 [00:08<00:12, 1.45it/s]\n 43%|████▎ | 13/30 [00:08<00:11, 1.45it/s]\n 47%|████▋ | 14/30 [00:09<00:11, 1.45it/s]\n 50%|█████ | 15/30 [00:10<00:10, 1.45it/s]\n 53%|█████▎ | 16/30 [00:11<00:09, 1.45it/s]\n 57%|█████▋ | 17/30 [00:11<00:08, 1.45it/s]\n 60%|██████ | 18/30 [00:12<00:08, 1.45it/s]\n 63%|██████▎ | 19/30 [00:13<00:07, 1.45it/s]\n 67%|██████▋ | 20/30 [00:13<00:06, 1.45it/s]\n 70%|███████ | 21/30 [00:14<00:06, 1.45it/s]\n 73%|███████▎ | 22/30 [00:15<00:05, 1.45it/s]\n 77%|███████▋ | 23/30 [00:15<00:04, 1.45it/s]\n 80%|████████ | 24/30 [00:16<00:04, 1.45it/s]\n 83%|████████▎ | 25/30 [00:17<00:03, 1.45it/s]\n 87%|████████▋ | 26/30 [00:17<00:02, 1.45it/s]\n 90%|█████████ | 27/30 [00:18<00:02, 1.45it/s]\n 93%|█████████▎| 28/30 [00:19<00:01, 1.45it/s]\n 97%|█████████▋| 29/30 [00:19<00:00, 1.45it/s]\n100%|██████████| 30/30 [00:20<00:00, 1.45it/s]\n100%|██████████| 30/30 [00:20<00:00, 1.45it/s]\n 0%| | 0/8 [00:00<?, ?it/s]\n 50%|█████ | 4/8 [00:00<00:00, 35.63it/s]\n100%|██████████| 8/8 [00:00<00:00, 19.59it/s]\n100%|██████████| 8/8 [00:00<00:00, 21.00it/s]", "metrics": { "predict_time": 23.278045, "total_time": 23.302082 }, "output": "https://replicate.delivery/pbxt/uGxehLs9cfqYd0c6aHyfHjv0eA4VDzXOfL79rogAwopG3nlNC/tmp.gif", "started_at": "2023-10-10T09:47:14.510687Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/fgfofdlbkrol52zhx3nvzzm4nq", "cancel": "https://api.replicate.com/v1/predictions/fgfofdlbkrol52zhx3nvzzm4nq/cancel" }, "version": "c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63" }
Generated inWarning - setting num_images_per_prompt = 1 because video_length = 8 0%| | 0/30 [00:00<?, ?it/s] 3%|▎ | 1/30 [00:00<00:19, 1.45it/s] 7%|▋ | 2/30 [00:01<00:19, 1.46it/s] 10%|█ | 3/30 [00:02<00:18, 1.46it/s] 13%|█▎ | 4/30 [00:02<00:17, 1.46it/s] 17%|█▋ | 5/30 [00:03<00:17, 1.45it/s] 20%|██ | 6/30 [00:04<00:16, 1.45it/s] 23%|██▎ | 7/30 [00:04<00:15, 1.45it/s] 27%|██▋ | 8/30 [00:05<00:15, 1.45it/s] 30%|███ | 9/30 [00:06<00:14, 1.45it/s] 33%|███▎ | 10/30 [00:06<00:13, 1.45it/s] 37%|███▋ | 11/30 [00:07<00:13, 1.45it/s] 40%|████ | 12/30 [00:08<00:12, 1.45it/s] 43%|████▎ | 13/30 [00:08<00:11, 1.45it/s] 47%|████▋ | 14/30 [00:09<00:11, 1.45it/s] 50%|█████ | 15/30 [00:10<00:10, 1.45it/s] 53%|█████▎ | 16/30 [00:11<00:09, 1.45it/s] 57%|█████▋ | 17/30 [00:11<00:08, 1.45it/s] 60%|██████ | 18/30 [00:12<00:08, 1.45it/s] 63%|██████▎ | 19/30 [00:13<00:07, 1.45it/s] 67%|██████▋ | 20/30 [00:13<00:06, 1.45it/s] 70%|███████ | 21/30 [00:14<00:06, 1.45it/s] 73%|███████▎ | 22/30 [00:15<00:05, 1.45it/s] 77%|███████▋ | 23/30 [00:15<00:04, 1.45it/s] 80%|████████ | 24/30 [00:16<00:04, 1.45it/s] 83%|████████▎ | 25/30 [00:17<00:03, 1.45it/s] 87%|████████▋ | 26/30 [00:17<00:02, 1.45it/s] 90%|█████████ | 27/30 [00:18<00:02, 1.45it/s] 93%|█████████▎| 28/30 [00:19<00:01, 1.45it/s] 97%|█████████▋| 29/30 [00:19<00:00, 1.45it/s] 100%|██████████| 30/30 [00:20<00:00, 1.45it/s] 100%|██████████| 30/30 [00:20<00:00, 1.45it/s] 0%| | 0/8 [00:00<?, ?it/s] 50%|█████ | 4/8 [00:00<00:00, 35.63it/s] 100%|██████████| 8/8 [00:00<00:00, 19.59it/s] 100%|██████████| 8/8 [00:00<00:00, 21.00it/s]
Prediction
cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63IDirlqj5lb2ilnbbdmrpxmjtutxuStatusSucceededSourceWebHardwareA40 (Large)Total durationCreatedInput
- gif
- null
- seed
- 1002
- steps
- 30
- width
- 672
- height
- 384
- prompt
- anime, animated establishing shot of a volcano erupting, bright sunshine and snow
- hf_lora_url
- null
- control_type
- depth
- target_width
- 512
- video_length
- 8
- target_height
- 512
- original_width
- 1920
- video_duration
- 1000
- negative_prompt
- dark, underexposed
- original_height
- 1080
- control_guidance_end
- 0.7
- replicate_weights_url
- control_guidance_start
- 0
- controlnet_conditioning_scale
- 0.8
{ "gif": null, "seed": 1002, "steps": 30, "width": 672, "height": 384, "prompt": "anime, animated establishing shot of a volcano erupting, bright sunshine and snow", "hf_lora_url": null, "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "dark, underexposed", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 }
Install Replicate’s Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63", { input: { seed: 1002, steps: 30, width: 672, height: 384, prompt: "anime, animated establishing shot of a volcano erupting, bright sunshine and snow", control_type: "depth", target_width: 512, video_length: 8, target_height: 512, original_width: 1920, video_duration: 1000, negative_prompt: "dark, underexposed", original_height: 1080, control_guidance_end: 0.7, replicate_weights_url: "", control_guidance_start: 0, controlnet_conditioning_scale: 0.8 } } ); // To access the file URL: console.log(output.url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:pip install replicate
Import the client:import replicate
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63", input={ "seed": 1002, "steps": 30, "width": 672, "height": 384, "prompt": "anime, animated establishing shot of a volcano erupting, bright sunshine and snow", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "dark, underexposed", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 } ) print(output)
To learn more, take a look at the guide on getting started with Python.
Run cloneofsimo/hotshot-xl-lora-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "cloneofsimo/hotshot-xl-lora-controlnet:c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63", "input": { "seed": 1002, "steps": 30, "width": 672, "height": 384, "prompt": "anime, animated establishing shot of a volcano erupting, bright sunshine and snow", "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "dark, underexposed", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
Output
{ "completed_at": "2023-10-10T09:52:09.874760Z", "created_at": "2023-10-10T09:51:46.452704Z", "data_removed": false, "error": null, "id": "irlqj5lb2ilnbbdmrpxmjtutxu", "input": { "gif": null, "seed": 1002, "steps": 30, "width": 672, "height": 384, "prompt": "anime, animated establishing shot of a volcano erupting, bright sunshine and snow", "hf_lora_url": null, "control_type": "depth", "target_width": 512, "video_length": 8, "target_height": 512, "original_width": 1920, "video_duration": 1000, "negative_prompt": "dark, underexposed", "original_height": 1080, "control_guidance_end": 0.7, "replicate_weights_url": "", "control_guidance_start": 0, "controlnet_conditioning_scale": 0.8 }, "logs": "Warning - setting num_images_per_prompt = 1 because video_length = 8\n 0%| | 0/30 [00:00<?, ?it/s]\n 3%|▎ | 1/30 [00:00<00:19, 1.45it/s]\n 7%|▋ | 2/30 [00:01<00:19, 1.45it/s]\n 10%|█ | 3/30 [00:02<00:18, 1.45it/s]\n 13%|█▎ | 4/30 [00:02<00:17, 1.45it/s]\n 17%|█▋ | 5/30 [00:03<00:17, 1.45it/s]\n 20%|██ | 6/30 [00:04<00:16, 1.45it/s]\n 23%|██▎ | 7/30 [00:04<00:15, 1.46it/s]\n 27%|██▋ | 8/30 [00:05<00:15, 1.46it/s]\n 30%|███ | 9/30 [00:06<00:14, 1.46it/s]\n 33%|███▎ | 10/30 [00:06<00:13, 1.46it/s]\n 37%|███▋ | 11/30 [00:07<00:13, 1.45it/s]\n 40%|████ | 12/30 [00:08<00:12, 1.45it/s]\n 43%|████▎ | 13/30 [00:08<00:11, 1.45it/s]\n 47%|████▋ | 14/30 [00:09<00:10, 1.45it/s]\n 50%|█████ | 15/30 [00:10<00:10, 1.45it/s]\n 53%|█████▎ | 16/30 [00:11<00:09, 1.45it/s]\n 57%|█████▋ | 17/30 [00:11<00:08, 1.45it/s]\n 60%|██████ | 18/30 [00:12<00:08, 1.45it/s]\n 63%|██████▎ | 19/30 [00:13<00:07, 1.45it/s]\n 67%|██████▋ | 20/30 [00:13<00:06, 1.45it/s]\n 70%|███████ | 21/30 [00:14<00:06, 1.45it/s]\n 73%|███████▎ | 22/30 [00:15<00:05, 1.45it/s]\n 77%|███████▋ | 23/30 [00:15<00:04, 1.45it/s]\n 80%|████████ | 24/30 [00:16<00:04, 1.45it/s]\n 83%|████████▎ | 25/30 [00:17<00:03, 1.45it/s]\n 87%|████████▋ | 26/30 [00:17<00:02, 1.45it/s]\n 90%|█████████ | 27/30 [00:18<00:02, 1.45it/s]\n 93%|█████████▎| 28/30 [00:19<00:01, 1.45it/s]\n 97%|█████████▋| 29/30 [00:19<00:00, 1.45it/s]\n100%|██████████| 30/30 [00:20<00:00, 1.45it/s]\n100%|██████████| 30/30 [00:20<00:00, 1.45it/s]\n 0%| | 0/8 [00:00<?, ?it/s]\n 50%|█████ | 4/8 [00:00<00:00, 35.64it/s]\n100%|██████████| 8/8 [00:00<00:00, 19.58it/s]\n100%|██████████| 8/8 [00:00<00:00, 20.99it/s]", "metrics": { "predict_time": 23.417482, "total_time": 23.422056 }, "output": "https://replicate.delivery/pbxt/RU9CI33SMCKMFBFQplELLexGPsOGNIU42VpauosBZZLkhW2IA/tmp.gif", "started_at": "2023-10-10T09:51:46.457278Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/irlqj5lb2ilnbbdmrpxmjtutxu", "cancel": "https://api.replicate.com/v1/predictions/irlqj5lb2ilnbbdmrpxmjtutxu/cancel" }, "version": "c447ef9fc621af091e2c06d08fd2a22d9f5906389a2f8103c851a2f7cf9c4e63" }
Generated inWarning - setting num_images_per_prompt = 1 because video_length = 8 0%| | 0/30 [00:00<?, ?it/s] 3%|▎ | 1/30 [00:00<00:19, 1.45it/s] 7%|▋ | 2/30 [00:01<00:19, 1.45it/s] 10%|█ | 3/30 [00:02<00:18, 1.45it/s] 13%|█▎ | 4/30 [00:02<00:17, 1.45it/s] 17%|█▋ | 5/30 [00:03<00:17, 1.45it/s] 20%|██ | 6/30 [00:04<00:16, 1.45it/s] 23%|██▎ | 7/30 [00:04<00:15, 1.46it/s] 27%|██▋ | 8/30 [00:05<00:15, 1.46it/s] 30%|███ | 9/30 [00:06<00:14, 1.46it/s] 33%|███▎ | 10/30 [00:06<00:13, 1.46it/s] 37%|███▋ | 11/30 [00:07<00:13, 1.45it/s] 40%|████ | 12/30 [00:08<00:12, 1.45it/s] 43%|████▎ | 13/30 [00:08<00:11, 1.45it/s] 47%|████▋ | 14/30 [00:09<00:10, 1.45it/s] 50%|█████ | 15/30 [00:10<00:10, 1.45it/s] 53%|█████▎ | 16/30 [00:11<00:09, 1.45it/s] 57%|█████▋ | 17/30 [00:11<00:08, 1.45it/s] 60%|██████ | 18/30 [00:12<00:08, 1.45it/s] 63%|██████▎ | 19/30 [00:13<00:07, 1.45it/s] 67%|██████▋ | 20/30 [00:13<00:06, 1.45it/s] 70%|███████ | 21/30 [00:14<00:06, 1.45it/s] 73%|███████▎ | 22/30 [00:15<00:05, 1.45it/s] 77%|███████▋ | 23/30 [00:15<00:04, 1.45it/s] 80%|████████ | 24/30 [00:16<00:04, 1.45it/s] 83%|████████▎ | 25/30 [00:17<00:03, 1.45it/s] 87%|████████▋ | 26/30 [00:17<00:02, 1.45it/s] 90%|█████████ | 27/30 [00:18<00:02, 1.45it/s] 93%|█████████▎| 28/30 [00:19<00:01, 1.45it/s] 97%|█████████▋| 29/30 [00:19<00:00, 1.45it/s] 100%|██████████| 30/30 [00:20<00:00, 1.45it/s] 100%|██████████| 30/30 [00:20<00:00, 1.45it/s] 0%| | 0/8 [00:00<?, ?it/s] 50%|█████ | 4/8 [00:00<00:00, 35.64it/s] 100%|██████████| 8/8 [00:00<00:00, 19.58it/s] 100%|██████████| 8/8 [00:00<00:00, 20.99it/s]
Want to make some of these yourself?
Run this model