lucataco / musubi-tuner

Finetune HunyuanVideo LoRAs with kohya-ss/musibi-tuner

  • Public
  • 76 runs
  • GitHub
  • Weights
  • Paper
  • License
Run with an API
Playground API Examples README Versions

Run time and cost

This model runs on Nvidia H100 GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

About

Cog implementation of Kohya-ss/musubi-tuner, a fine tuning script for training HunyuanVideo LoRA models

Follow me on Twitter/X @lucataco93

How to use

  • This model expects a zip file that contains at least 8 video and caption file pairs. Example file pairing: segment1.mp4 & segment1.txt
  • Videos should be around 544x960 and 2 seconds in length each. Captions should be more than 50 words each
  • For help with captioning videos see our collection here: video-to-text
  • For help with splitting up videos to desired width height and duration, see this model: lucataco/video-split

Run your Trained LoRA

Use the model zsxkib/hunyuan-video-lora to run your HunyuanVideo LoRA.

Convert to use with ComfyUI

After training a LoRA (ex: lucataco/hunyuan-musubi-rose-6), if you want to use the LoRA locally you’ll need to convert it to a ComfyUI compatible format(lucataco/hunyuan-musubi-rose-6-comfyui).

Convert your Musubi LoRA to a ComfyUI compatible format with the following model: lucataco/musubi-tuner-lora-converter

Train & Create a new Replicate Model

If you’re already familiar with the Replicate Flux Trainer: ostris/flux-dev-lora-trainer which creates a new Replicate model using the trained LoRA, you might be interested in a similar model for HunyuanVideo LoRAs: hunyuan-video-lora