genmoai / mochi-1-lora-trainer

a-r-r-o-w/cogvideox-factory for Mochi-1 LoRA Training

  • Public
  • 16 runs
  • GitHub
  • Weights
  • License
Run with an API
Playground API Examples README Versions

Run time and cost

This model runs on Nvidia H100 GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

About

A Cog implementation of a-r-r-o-w/cogvideox-factory for Mochi-1 LoRA training

How to use

You must include a zip file of mov/mp4 video snippets. It is recommended to include captions for each video (in separate txt files) but not required, note that lack of captions may hurt fine-tuning quality.

Feel free to use any video captioning model to caption your videos. Try using: lucataco/bulk-video-caption , lucataco/apollo-3b , or ChatGPT , etc. Captions should be fairly detailed, ideally more than 50 words.

Make sure each input video is no longer than 2 seconds long each. Use this tool to help you split up video files: mochi1-video-split

Example use case: VHS effect

Lets say I want to train a VHS video effect lora that looks like this: Luis Quintero Noise-Video

  • First steps is to caption the video with apollo-7b, bulk-video-caption, ChatGPT, or Gemini1.5 etc

  • Now that you have a txt file you should split up the large video into smaller 2.5 second snippets with this tool: mochi1-video-split

  • Now train your VHS LoRA using the same settings as this training run: mochi-lora-vhs

  • Finally, test out your LoRA with this model to see the effect! Be sure to use similar words that are found in your caption txt file. Here is an example run of the VHS LoRA

Example Trained LoRAs:

Under the examples tab you will see the following Mochi-1 LoRAs trained with this model:

How to Run your LoRA:

Once you have uploaded your LoRA file to a huggingface space, you can use the Mochi-1 LoRA Explorer Model and try it out.