zhouzhengjun/lora_train | Run with an API on Replicate

zhouzhengjun

lora_train

Public

5 runs

Run with an API

Pricing

Playground API Examples README Versions

Input

instance_data

*file

A ZIP file containing your training images (JPG, PNG, etc. size not restricted). These images contain your 'subject' that you want the trained model to embed in the output domain for later generating customized scenes beyond the training images. For best results, use images without noise or unrelated objects in the background.

seed

integer

A seed for reproducible training

Default: 1337

resolution

integer

The resolution for input images. All the images in the train/validation dataset will be resized to this resolution.

Default: 512

train_text_encoder

boolean

Whether to train the text encoder

Default: true

train_batch_size

integer

Batch size (per device) for the training dataloader.

Default: 1

gradient_accumulation_steps

integer

Number of updates steps to accumulate before performing a backward/update pass.

Default: 4

gradient_checkpointing

boolean

Whether or not to use gradient checkpointing to save memory at the expense of slower backward pass.

Default: false

scale_lr

boolean

Scale the learning rate by the number of GPUs, gradient accumulation steps, and batch size.

Default: true

lr_scheduler

string

The scheduler type to use

Default: "constant"

lr_warmup_steps

integer

Number of steps for the warmup in the lr scheduler.

Default: 0

clip_ti_decay

boolean

Whether or not to perform Bayesian Learning Rule on norm of the CLIP latent.

Default: true

color_jitter

boolean

Whether or not to use color jitter at augmentation.

Default: true

continue_inversion

boolean

Whether or not to continue inversion.

Default: false

continue_inversion_lr

number

The learning rate for continuing an inversion.

Default: 0.0001

initializer_tokens

string

Shift + Return to add a new line

The tokens to use for the initializer. If not provided, will randomly initialize from gaussian N(0,0.017^2)

learning_rate_text

number

The learning rate for the text encoder.

Default: 0.00001

learning_rate_ti

number

The learning rate for the TI.

Default: 0.0005

learning_rate_unet

number

The learning rate for the unet.

Default: 0.0001

lora_rank

integer

Rank of the LoRA. Larger it is, more likely to capture fidelity but less likely to be editable. Larger rank will make the end result larger.

Default: 4

lora_dropout_p

number

Dropout for the LoRA layer. Reference LoRA paper for more details.

Default: 0.1

lora_scale

number

Scaling parameter at the end of the LoRA layer.

Default: 1

lr_scheduler_lora

string

The scheduler type to use

Default: "constant"

lr_warmup_steps_lora

integer

Number of steps for the warmup in the lr scheduler.

Default: 0

max_train_steps_ti

integer

The maximum number of training steps for the TI.

Default: 500

max_train_steps_tuning

integer

The maximum number of training steps for the tuning.

Default: 1000

placeholder_token_at_data

string

Shift + Return to add a new line

If this value is provided as 'X|Y', it will transform target word X into Y at caption. You are required to provide caption as filename (not regarding extension), and Y has to contain placeholder token below. You are also required to set `None` for `use_template` argument to use this feature.

placeholder_tokens

string

Shift + Return to add a new line

The placeholder tokens to use for the initializer. If not provided, will use the first tokens of the data.

Default: "<s1>|<s2>"

use_face_segmentation_condition

boolean

Whether or not to use the face segmentation condition.

Default: false

use_template

string

The template to use for the inversion.

Default: "object"

weight_decay_lora

number

The weight decay for the LORA loss.

Default: 0.001

weight_decay_ti

number

The weight decay for the TI.

Default: 0

Run this model in Node.js with one line of code:

npx create-replicate --model=zhouzhengjun/lora_train

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zhouzhengjun/lora_train using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zhouzhengjun/lora_train:490c2d30a2996a69f617ea9a8e5fd17d31bbc032291b2097c324ac1670abb851",
  {
    input: {
      seed: 1337,
      scale_lr: true,
      lora_rank: 4,
      lora_scale: 1,
      resolution: 512,
      color_jitter: true,
      lr_scheduler: "constant",
      use_template: "object",
      clip_ti_decay: true,
      lora_dropout_p: 0.1,
      lr_warmup_steps: 0,
      weight_decay_ti: 0,
      learning_rate_ti: 0.0005,
      train_batch_size: 1,
      lr_scheduler_lora: "constant",
      weight_decay_lora: 0.001,
      continue_inversion: false,
      learning_rate_text: 0.00001,
      learning_rate_unet: 0.0001,
      max_train_steps_ti: 500,
      placeholder_tokens: "<s1>|<s2>",
      train_text_encoder: true,
      lr_warmup_steps_lora: 0,
      continue_inversion_lr: 0.0001,
      gradient_checkpointing: false,
      max_train_steps_tuning: 1000,
      gradient_accumulation_steps: 4,
      use_face_segmentation_condition: false
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zhouzhengjun/lora_train using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zhouzhengjun/lora_train:490c2d30a2996a69f617ea9a8e5fd17d31bbc032291b2097c324ac1670abb851",
    input={
        "seed": 1337,
        "scale_lr": True,
        "lora_rank": 4,
        "lora_scale": 1,
        "resolution": 512,
        "color_jitter": True,
        "lr_scheduler": "constant",
        "use_template": "object",
        "clip_ti_decay": True,
        "lora_dropout_p": 0.1,
        "lr_warmup_steps": 0,
        "weight_decay_ti": 0,
        "learning_rate_ti": 0.0005,
        "train_batch_size": 1,
        "lr_scheduler_lora": "constant",
        "weight_decay_lora": 0.001,
        "continue_inversion": False,
        "learning_rate_text": 0.00001,
        "learning_rate_unet": 0.0001,
        "max_train_steps_ti": 500,
        "placeholder_tokens": "<s1>|<s2>",
        "train_text_encoder": True,
        "lr_warmup_steps_lora": 0,
        "continue_inversion_lr": 0.0001,
        "gradient_checkpointing": False,
        "max_train_steps_tuning": 1000,
        "gradient_accumulation_steps": 4,
        "use_face_segmentation_condition": False
    }
)

# To access the file URL:
print(output.url())
#=> "http://example.com"

# To write the file to disk:
with open("my-image.png", "wb") as file:
    file.write(output.read())

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zhouzhengjun/lora_train using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zhouzhengjun/lora_train:490c2d30a2996a69f617ea9a8e5fd17d31bbc032291b2097c324ac1670abb851",
    "input": {
      "seed": 1337,
      "scale_lr": true,
      "lora_rank": 4,
      "lora_scale": 1,
      "resolution": 512,
      "color_jitter": true,
      "lr_scheduler": "constant",
      "use_template": "object",
      "clip_ti_decay": true,
      "lora_dropout_p": 0.1,
      "lr_warmup_steps": 0,
      "weight_decay_ti": 0,
      "learning_rate_ti": 0.0005,
      "train_batch_size": 1,
      "lr_scheduler_lora": "constant",
      "weight_decay_lora": 0.001,
      "continue_inversion": false,
      "learning_rate_text": 0.00001,
      "learning_rate_unet": 0.0001,
      "max_train_steps_ti": 500,
      "placeholder_tokens": "<s1>|<s2>",
      "train_text_encoder": true,
      "lr_warmup_steps_lora": 0,
      "continue_inversion_lr": 0.0001,
      "gradient_checkpointing": false,
      "max_train_steps_tuning": 1000,
      "gradient_accumulation_steps": 4,
      "use_face_segmentation_condition": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/zhouzhengjun/lora_train@sha256:490c2d30a2996a69f617ea9a8e5fd17d31bbc032291b2097c324ac1670abb851 \
  -i 'seed=1337' \
  -i 'scale_lr=true' \
  -i 'lora_rank=4' \
  -i 'lora_scale=1' \
  -i 'resolution=512' \
  -i 'color_jitter=true' \
  -i 'lr_scheduler="constant"' \
  -i 'use_template="object"' \
  -i 'clip_ti_decay=true' \
  -i 'lora_dropout_p=0.1' \
  -i 'lr_warmup_steps=0' \
  -i 'weight_decay_ti=0' \
  -i 'learning_rate_ti=0.0005' \
  -i 'train_batch_size=1' \
  -i 'lr_scheduler_lora="constant"' \
  -i 'weight_decay_lora=0.001' \
  -i 'continue_inversion=false' \
  -i 'learning_rate_text=0.00001' \
  -i 'learning_rate_unet=0.0001' \
  -i 'max_train_steps_ti=500' \
  -i 'placeholder_tokens="<s1>|<s2>"' \
  -i 'train_text_encoder=true' \
  -i 'lr_warmup_steps_lora=0' \
  -i 'continue_inversion_lr=0.0001' \
  -i 'gradient_checkpointing=false' \
  -i 'max_train_steps_tuning=1000' \
  -i 'gradient_accumulation_steps=4' \
  -i 'use_face_segmentation_condition=false'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/zhouzhengjun/lora_train@sha256:490c2d30a2996a69f617ea9a8e5fd17d31bbc032291b2097c324ac1670abb851
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "seed": 1337,
      "scale_lr": true,
      "lora_rank": 4,
      "lora_scale": 1,
      "resolution": 512,
      "color_jitter": true,
      "lr_scheduler": "constant",
      "use_template": "object",
      "clip_ti_decay": true,
      "lora_dropout_p": 0.1,
      "lr_warmup_steps": 0,
      "weight_decay_ti": 0,
      "learning_rate_ti": 0.0005,
      "train_batch_size": 1,
      "lr_scheduler_lora": "constant",
      "weight_decay_lora": 0.001,
      "continue_inversion": false,
      "learning_rate_text": 0.00001,
      "learning_rate_unet": 0.0001,
      "max_train_steps_ti": 500,
      "placeholder_tokens": "<s1>|<s2>",
      "train_text_encoder": true,
      "lr_warmup_steps_lora": 0,
      "continue_inversion_lr": 0.0001,
      "gradient_checkpointing": false,
      "max_train_steps_tuning": 1000,
      "gradient_accumulation_steps": 4,
      "use_face_segmentation_condition": false
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

No output yet! Press "Submit" to start a prediction.

Run time and cost

This model runs on Nvidia T4 GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

This model doesn't have a readme.