Train – zsxkib/wan-lora-trainer:9372a2a2

zsxkib / wan-lora-trainer

📽️Fine-tune the Wan2.1 both 14b & 1.3b video model using video datasets🎥

Public
262 runs
Commercial use
GitHub
Weights
License

Create training

Trainings for this model run on Nvidia H100 GPU hardware, which costs $0.001525 per second. Upon creation, you will be redirected to the training detail page where you can monitor your training's progress, and eventually download the weights and run the trained model.

Note: versions of this model with fast booting use the hardware set by the base model they were trained from.

Learn more about training

If you haven't yet trained a model on Replicate, you can read one of the following guides.

Destination

*string

Select a model on Replicate that will be the destination for the trained version. If the model does not exist, select the "Create model" option and a field will appear to enter the name of the new model. We'll create the model for you when you create the training.

input_video_zip

*file

A zip file containing video and caption data for training. The zip should contain at least one video file (e.g., mp4, mov) and optionally caption files (.txt).

trigger_word

string

Shift + Return to add a new line

The trigger word to be associated with all videos during training. This word will help activate the LoRA when used in prompts.

Default: "TOK"

autocaption

boolean

Automatically caption videos using QWEN-VL that don't have matching caption files.

Default: true

autocaption_prefix

string

Shift + Return to add a new line

Optional: Text you want to appear at the beginning of all your generated captions; for example, 'a video of TOK, '. You can include your trigger word in the prefix.

Default: ""

autocaption_suffix

string

Shift + Return to add a new line

Optional: Text you want to appear at the end of all your generated captions; for example, ' in the style of TOK'. You can include your trigger word in suffixes.

Default: ""

max_training_steps

integer

(minimum: 10, maximum: 20000)

Total number of training steps, including warmup. For example, if max_training_steps=1000 and warmup_steps_budget=100, then you have 1000 steps total, with the first 100 for warmup.

Default: 1000

learning_rate

number

(minimum: 0.00001, maximum: 0.01)

Learning rate for training. Higher values may lead to faster convergence but potential instability.

Default: 0.00002

lora_rank

integer

(minimum: 16, maximum: 256)

LoRA rank for training. Higher ranks can capture more complex features but require more training time.

Default: 32

warmup_steps_budget

integer

(minimum: -1, maximum: 2000)

If not provided or set to -1, defaults to 10% of max_training_steps. These steps ramp from a lower LR to the configured LR. They are included within max_training_steps, not added on top.

Default: -1

weight_decay

number

(minimum: 0, maximum: 0.1)

Weight decay for regularization. Controls overfitting - lower values allow more detailed memorization.

Default: 0.0001

seed

integer

Random seed for training reproducibility. Use -1 for a random seed.

Default: -1

hf_repo_id

string

Shift + Return to add a new line

Hugging Face repository ID, if you'd like to upload the trained LoRA to Hugging Face. For example, username/wan-lora. If the given repo does not exist, a new public repo will be created.

hf_token

secret

Hugging Face token, if you'd like to upload the trained LoRA to Hugging Face.

wandb_api_key

secret

Weights and Biases API key for experiment tracking. If provided, training progress will be logged to W&B.

wandb_project

string

Shift + Return to add a new line

Weights and Biases project name. A new project will be created if it doesn't exist.

Default: "wan_train_replicate"

wandb_run

string

Shift + Return to add a new line

Weights and Biases run name. If not provided, a random name will be generated.

model_type

string

An enumeration.

wandb_entity

string

Shift + Return to add a new line

Weights and Biases entity (username or organization). If not provided, will use your default entity.

finetuning_type

string

An enumeration.

video_clip_mode

string

An enumeration.

wandb_sample_prompts

string

Shift + Return to add a new line

Newline-separated list of prompts to use for sample generation. These will be logged to W&B with your trigger word.

wandb_sample_interval

integer

(minimum: 1)

Step interval for sampling output videos to W&B

Default: 250

wandb_save_interval

integer

(minimum: 1)

Step interval for saving checkpoints to W&B

Default: 500

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Train zsxkib/wan-lora-trainer using Replicate’s API:

training = replicate.trainings.create(
  # You need to create a model on Replicate that will be the destination for the trained version.
  destination="/model-name"
  version="zsxkib/wan-lora-trainer:9372a2a2451568214313fdea02a04b62c1378176525ebb1d2cd46974fe87bcf1",
  input={
    "seed": -1,
    "lora_rank": 32,
    "autocaption": True,
    "trigger_word": "TOK",
    "weight_decay": 0.0001,
    "learning_rate": 0.00002,
    "wandb_project": "wan_train_replicate",
    "input_video_zip": "https://",
    "autocaption_prefix": "",
    "autocaption_suffix": "",
    "max_training_steps": 1000,
    "wandb_save_interval": 500,
    "warmup_steps_budget": -1,
    "wandb_sample_interval": 250
  },
)

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Train zsxkib/wan-lora-trainer using Replicate’s API:

const training = await replicate.trainings.create(
  "zsxkib",
  "wan-lora-trainer",
  "9372a2a2451568214313fdea02a04b62c1378176525ebb1d2cd46974fe87bcf1",
  {
    // You need to create a model on Replicate that will be the destination for the trained version.
    destination: "/model-name",
    input: {
      seed: -1,
      lora_rank: 32,
      autocaption: true,
      trigger_word: "TOK",
      weight_decay: 0.0001,
      learning_rate: 0.00002,
      wandb_project: "wan_train_replicate",
      input_video_zip: "https://",
      autocaption_prefix: "",
      autocaption_suffix: "",
      max_training_steps: 1000,
      wandb_save_interval: 500,
      warmup_steps_budget: -1,
      wandb_sample_interval: 250
    }
  }
);