Readme
HunyuanVideo with LoRA Support 🎬
⚠️ This model supports LoRA training (click on the train tab)
Turn your text descriptions into videos using HunyuanVideo, now with support for custom LoRA files! LoRAs are like style plugins that help you customize how your videos look without changing the main model.
What’s This All About? ✨
This is a text-to-video AI model that lets you:
-
Create videos just by describing what you want to see
-
Use custom LoRA files to add your own style or characters
-
Train your own custom models that will be saved as destination models
-
Control various aspects of how your video turns out
Think of it like having an AI video creator that you can teach new styles!
How It Works 🎥
Under the hood, this uses HunyuanVideo - a powerful AI model that turns text into videos. We’ve added support for LoRA files, which are like special add-ons that can make the videos look more like what you want. For example, you could use a LoRA trained on anime art to make your videos look more animated!
The cool part is that even if your LoRA was only trained on still images, it can still create smooth-moving videos. You can also train your own models using your dataset, and once training is complete, it’ll automatically create a destination model ready for use. Pretty neat, right?
What You Can Control 🎮
When creating your video, you can adjust things like:
-
Your text description of what you want to see
-
Your custom LoRA file (must be .safetensors format)
-
How strongly your LoRA affects the final video
-
Video size (width and height)
-
How many frames you want
-
Video speed (frames per second)
-
Video quality settings
-
Training parameters when creating your own models
Current Limits ⚠️
Since this is a work in progress, there are some limitations:
-
Videos can’t be bigger than 1536x1024
-
Maximum length is 300 frames
-
You need to use .safetensors format for LoRA files
-
Bigger videos take longer to make
-
Might need a beefy computer for larger videos
Coming Soon! 🚀
I’m (zsxkib) working on adding LoRA training directly to Replicate! This means you’ll be able to:
-
Train your own LoRAs right here on Replicate
-
Use them immediately for video generation
-
Share them with others
-
Get an automatically created destination model after training completes
Stay tuned for updates!
Credits and Thanks 📚
This builds on the amazing work by Tencent’s HunyuanVideo team:
@misc{kong2024hunyuanvideo,
title={HunyuanVideo: A Systematic Framework For Large Video Generative Models},
author={Weijie Kong, et al.},
year={2024},
archivePrefix={arXiv},
primaryClass={cs.CV}
}Special thanks to Jukka Seppänen (@Kijaidesign) for creating the fantastic ComfyUI implementation that makes this all possible. His ComfyUI nodes are the backbone of this project!
Follow me on Twitter/X @zsakib_ for updates on LoRA training and other cool features!