cloneofsimo / hotshot-xl-lora-controlnet

Text-to-gif using SDXL, with controlnet and lora support

  • Public
  • 3.7K runs
  • GitHub
  • Paper
  • License
Run with an API
Playground API Examples README Versions

Examples

View more

Run time and cost

This model costs approximately $0.027 to run on Replicate, or 37 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A40 (Large) GPU hardware. Predictions typically complete within 37 seconds. The predict time for this model varies significantly based on the inputs.

Readme

✨ Hotshot-XL: State-of-the-art AI text-to-GIF model trained to work alongside Stable Diffusion XL

hotshot.co

Hotshot-XL can generate GIFs with any fine-tuned SDXL model. This means two things:

  1. You’ll be able to make GIFs with any existing or newly fine-tuned SDXL model you may want to use.
  2. If you’d like to make GIFs of personalized subjects, you can load your own SDXL based LORAs, and not have to worry about fine-tuning Hotshot-XL. This is awesome because it’s usually much easier to find suitable images for training data than it is to find videos. It also hopefully fits into everyone’s existing LORA usage/workflows :)

Hotshot-XL is compatible with SDXL ControlNet to make GIFs in the composition/layout you’d like. More information about controlnet

Hotshot-XL was trained to generate 1 second GIFs at 8 FPS.

Hotshot-XL was trained on various aspect ratios. For best results with the base Hotshot-XL model, we recommend using it with an SDXL model that has been fine-tuned with 512x512 images. You can find an SDXL model we fine-tuned for 512x512 resolutions:

https://huggingface.co/hotshotco/SDXL-512