cloneofsimo/lora-advanced-training:8252738e – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

cloneofsimo /lora-advanced-training:8252738e

Input

instance_data

*file

A ZIP file containing your training images (JPG, PNG, etc. size not restricted). These images contain your 'subject' that you want the trained model to embed in the output domain for later generating customized scenes beyond the training images. For best results, use images without noise or unrelated objects in the background.

class_data

file

An optional ZIP file containing the training data of class images. This corresponds to `class_prompt` above, also with the purpose of keeping the model generalizable. By default, the pretrained stable-diffusion model will generate N images (determined by the `num_class_images` you set) based on the `class_prompt` provided. But to save time or to have your preferred specific set of `class_data`, you can also provide them in a ZIP file.

seed

integer

A seed for reproducible training

Default: 1337

resolution

integer

The resolution for input images. All the images in the train/validation dataset will be resized to this resolution.

Default: 512

train_text_encoder

boolean

Whether to train the text encoder

Default: true

train_batch_size

integer

Batch size (per device) for the training dataloader.

Default: 1

gradient_accumulation_steps

integer

Number of updates steps to accumulate before performing a backward/update pass.

Default: 4

gradient_checkpointing

boolean

Whether or not to use gradient checkpointing to save memory at the expense of slower backward pass.

Default: false

scale_lr

boolean

Scale the learning rate by the number of GPUs, gradient accumulation steps, and batch size.

Default: true

lr_scheduler

string

The scheduler type to use

Default: "constant"

lr_warmup_steps

integer

Number of steps for the warmup in the lr scheduler.

Default: 0

clip_ti_decay

boolean

Whether or not to clip the TI decay to be between 0 and 1.

Default: true

color_jitter

boolean

Whether or not to use color jitter.

Default: true

continue_inversion

boolean

Whether or not to continue an inversion.

Default: false

continue_inversion_lr

number

The learning rate for continuing an inversion.

Default: 0.0001

device

string

Shift + Return to add a new line

The device to use. Can be 'cuda' or 'cpu'.

Default: "cuda:0"

initializer_tokens

string

Shift + Return to add a new line

The tokens to use for the initializer. If not provided, will randomly initialize from gaussian N(0,0.017^2)

learning_rate_text

number

The learning rate for the text encoder.

Default: 0.00001

learning_rate_ti

number

The learning rate for the TI.

Default: 0.0005

learning_rate_unet

number

The learning rate for the unet.

Default: 0.0001

lora_rank

integer

The rank for the LORA.

Default: 4

lora_dropout_p

number

Dropout at lora

Default: 0.1

lora_scale

number

Scale for the LORA.

Default: 1

lr_scheduler_lora

string

The scheduler type to use

Default: "constant"

lr_warmup_steps_lora

integer

Number of steps for the warmup in the lr scheduler.

Default: 0

max_train_steps_ti

integer

The maximum number of training steps for the TI.

Default: 500

max_train_steps_tuning

integer

The maximum number of training steps for the tuning.

Default: 1000

perform_inversion

boolean

Whether to perform inversion during training.

Default: true

placeholder_token_at_data

string

Shift + Return to add a new line

Whether or not to use a placeholder token at the data.

placeholder_tokens

string

Shift + Return to add a new line

The placeholder tokens to use for the initializer. If not provided, will use the first tokens of the data.

Default: "<s1>|<s2>"

use_face_segmentation_condition

boolean

Whether or not to use the face segmentation condition.

Default: false

use_template

string

The template to use for the inversion.

Default: "object"

weight_decay_lora

number

The weight decay for the LORA loss.

Default: 0.001

weight_decay_ti

number

The weight decay for the TI.

Default: 0

Run this model in Node.js with one line of code:

npx create-replicate --model=cloneofsimo/lora-advanced-training

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run cloneofsimo/lora-advanced-training using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "cloneofsimo/lora-advanced-training:8252738e3dc457e439f446c567e45668d6fb2a86e2c3a873e582009e7c87e1ba",
  {
    input: {
      seed: 1337,
      device: "cuda:0",
      scale_lr: true,
      lora_rank: 4,
      lora_scale: 1,
      resolution: 512,
      color_jitter: true,
      lr_scheduler: "constant",
      use_template: "object",
      clip_ti_decay: true,
      lora_dropout_p: 0.1,
      lr_warmup_steps: 0,
      weight_decay_ti: 0,
      learning_rate_ti: 0.0005,
      train_batch_size: 1,
      lr_scheduler_lora: "constant",
      perform_inversion: true,
      weight_decay_lora: 0.001,
      continue_inversion: false,
      learning_rate_text: 0.00001,
      learning_rate_unet: 0.0001,
      max_train_steps_ti: 500,
      placeholder_tokens: "<s1>|<s2>",
      train_text_encoder: true,
      lr_warmup_steps_lora: 0,
      continue_inversion_lr: 0.0001,
      gradient_checkpointing: false,
      max_train_steps_tuning: 1000,
      gradient_accumulation_steps: 4,
      use_face_segmentation_condition: false
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run cloneofsimo/lora-advanced-training using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "cloneofsimo/lora-advanced-training:8252738e3dc457e439f446c567e45668d6fb2a86e2c3a873e582009e7c87e1ba",
    input={
        "seed": 1337,
        "device": "cuda:0",
        "scale_lr": True,
        "lora_rank": 4,
        "lora_scale": 1,
        "resolution": 512,
        "color_jitter": True,
        "lr_scheduler": "constant",
        "use_template": "object",
        "clip_ti_decay": True,
        "lora_dropout_p": 0.1,
        "lr_warmup_steps": 0,
        "weight_decay_ti": 0,
        "learning_rate_ti": 0.0005,
        "train_batch_size": 1,
        "lr_scheduler_lora": "constant",
        "perform_inversion": True,
        "weight_decay_lora": 0.001,
        "continue_inversion": False,
        "learning_rate_text": 0.00001,
        "learning_rate_unet": 0.0001,
        "max_train_steps_ti": 500,
        "placeholder_tokens": "<s1>|<s2>",
        "train_text_encoder": True,
        "lr_warmup_steps_lora": 0,
        "continue_inversion_lr": 0.0001,
        "gradient_checkpointing": False,
        "max_train_steps_tuning": 1000,
        "gradient_accumulation_steps": 4,
        "use_face_segmentation_condition": False
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run cloneofsimo/lora-advanced-training using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "cloneofsimo/lora-advanced-training:8252738e3dc457e439f446c567e45668d6fb2a86e2c3a873e582009e7c87e1ba",
    "input": {
      "seed": 1337,
      "device": "cuda:0",
      "scale_lr": true,
      "lora_rank": 4,
      "lora_scale": 1,
      "resolution": 512,
      "color_jitter": true,
      "lr_scheduler": "constant",
      "use_template": "object",
      "clip_ti_decay": true,
      "lora_dropout_p": 0.1,
      "lr_warmup_steps": 0,
      "weight_decay_ti": 0,
      "learning_rate_ti": 0.0005,
      "train_batch_size": 1,
      "lr_scheduler_lora": "constant",
      "perform_inversion": true,
      "weight_decay_lora": 0.001,
      "continue_inversion": false,
      "learning_rate_text": 0.00001,
      "learning_rate_unet": 0.0001,
      "max_train_steps_ti": 500,
      "placeholder_tokens": "<s1>|<s2>",
      "train_text_encoder": true,
      "lr_warmup_steps_lora": 0,
      "continue_inversion_lr": 0.0001,
      "gradient_checkpointing": false,
      "max_train_steps_tuning": 1000,
      "gradient_accumulation_steps": 4,
      "use_face_segmentation_condition": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.