zylim0702/sdxl-lora-customize-training:a4eefb9b

Version

You're looking at a specific version of this model. Jump to the model overview.

zylim0702 /sdxl-lora-customize-training:a4eefb9b

Playground API

Input

input_images

*file

A .zip or .tar file containing the image files that will be used for fine-tuning

seed

integer

Random seed for reproducible training. Leave empty to use a random seed

resolution

integer

Square pixel resolution which your images will be resized to for training

Default: 768

train_batch_size

integer

Batch size (per device) for training

Default: 4

num_train_epochs

integer

Number of epochs to loop through your training dataset

Default: 4000

max_train_steps

integer

Number of individual training steps. Takes precedence over num_train_epochs

Default: 1000

is_lora

boolean

Whether to use LoRA training. If set to False, will use Full fine tuning

Default: true

unet_learning_rate

number

Learning rate for the U-Net. We recommend this value to be somewhere between `1e-6` to `1e-5`.

Default: 0.000001

ti_lr

number

Scaling of learning rate for training textual inversion embeddings. Don't alter unless you know what you're doing.

Default: 0.0003

lora_lr

number

Scaling of learning rate for training LoRA embeddings. Don't alter unless you know what you're doing.

Default: 0.0001

lora_rank

integer

Rank of LoRA embeddings. Don't alter unless you know what you're doing.

Default: 32

lr_scheduler

string

Learning rate scheduler to use for training

Default: "constant"

lr_warmup_steps

integer

Number of warmup steps for lr schedulers with warmups.

Default: 100

token_string

string

Shift + Return to add a new line

A unique string that will be trained to refer to the concept in the input images. Can be anything, but TOK works well

Default: "TOK"

caption_prefix

string

Shift + Return to add a new line

Text which will be used as prefix during automatic captioning. Must contain the `token_string`. For example, if caption text is 'a photo of TOK', automatic captioning will expand to 'a photo of TOK under a bridge', 'a photo of TOK holding a cup', etc.

Default: "a photo of TOK, "

mask_target_prompts

string

Shift + Return to add a new line

Prompt that describes part of the image that you will find important. For example, if you are fine-tuning your pet, `photo of a dog` will be a good prompt. Prompt-based masking is used to focus the fine-tuning process on the important/salient parts of the image

crop_based_on_salience

boolean

If you want to crop the image to `target_size` based on the important parts of the image, set this to True. If you want to crop the image based on face detection, set this to False

Default: true

use_face_detection_instead

boolean

If you want to use face detection instead of CLIPSeg for masking. For face applications, we recommend using this option.

Default: false

clipseg_temperature

number

How blurry you want the CLIPSeg mask to be. We recommend this value be something between `0.5` to `1.0`. If you want to have more sharp mask (but thus more errorful), you can decrease this value.

Default: 1

verbose

boolean

verbose output

Default: true

checkpointing_steps

integer

Number of steps between saving checkpoints. Set to very very high number to disable checkpointing, because you don't need one.

Default: 999999

input_images_filetype

string

Filetype of the input images. Can be either `zip` or `tar`. By default its `infer`, and it will be inferred from the ext of input file.

Default: "infer"

Run this model in Node.js with one line of code:

npx create-replicate --model=zylim0702/sdxl-lora-customize-training

or set up a project from scratch

Install Replicate’s Node.js client library

npm install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zylim0702/sdxl-lora-customize-training using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zylim0702/sdxl-lora-customize-training:a4eefb9bbc944eaefc56d67acba982ab73d85ec449de6606cf48b13f1483a495",
  {
    input: {
      ti_lr: 0.0003,
      is_lora: true,
      lora_lr: 0.0001,
      verbose: true,
      lora_rank: 32,
      resolution: 768,
      lr_scheduler: "constant",
      token_string: "TOK",
      caption_prefix: "a photo of TOK, ",
      lr_warmup_steps: 100,
      max_train_steps: 1000,
      num_train_epochs: 4000,
      train_batch_size: 4,
      unet_learning_rate: 0.000001,
      checkpointing_steps: 999999,
      clipseg_temperature: 1,
      input_images_filetype: "infer",
      crop_based_on_salience: true,
      use_face_detection_instead: false
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library

pip install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client

import replicate

Run zylim0702/sdxl-lora-customize-training using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zylim0702/sdxl-lora-customize-training:a4eefb9bbc944eaefc56d67acba982ab73d85ec449de6606cf48b13f1483a495",
    input={
        "ti_lr": 0.0003,
        "is_lora": True,
        "lora_lr": 0.0001,
        "verbose": True,
        "lora_rank": 32,
        "resolution": 768,
        "lr_scheduler": "constant",
        "token_string": "TOK",
        "caption_prefix": "a photo of TOK, ",
        "lr_warmup_steps": 100,
        "max_train_steps": 1000,
        "num_train_epochs": 4000,
        "train_batch_size": 4,
        "unet_learning_rate": 0.000001,
        "checkpointing_steps": 999999,
        "clipseg_temperature": 1,
        "input_images_filetype": "infer",
        "crop_based_on_salience": True,
        "use_face_detection_instead": False
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zylim0702/sdxl-lora-customize-training using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "a4eefb9bbc944eaefc56d67acba982ab73d85ec449de6606cf48b13f1483a495",
    "input": {
      "ti_lr": 0.0003,
      "is_lora": true,
      "lora_lr": 0.0001,
      "verbose": true,
      "lora_rank": 32,
      "resolution": 768,
      "lr_scheduler": "constant",
      "token_string": "TOK",
      "caption_prefix": "a photo of TOK, ",
      "lr_warmup_steps": 100,
      "max_train_steps": 1000,
      "num_train_epochs": 4000,
      "train_batch_size": 4,
      "unet_learning_rate": 0.000001,
      "checkpointing_steps": 999999,
      "clipseg_temperature": 1,
      "input_images_filetype": "infer",
      "crop_based_on_salience": true,
      "use_face_detection_instead": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Install Cog

brew install cog

If you don’t have Homebrew, there are other installation options available.

Pull and run zylim0702/sdxl-lora-customize-training using Cog (this will download the full model and run it in your local environment):

cog predict r8.im/zylim0702/sdxl-lora-customize-training@sha256:a4eefb9bbc944eaefc56d67acba982ab73d85ec449de6606cf48b13f1483a495 \
  -i 'ti_lr=0.0003' \
  -i 'is_lora=true' \
  -i 'lora_lr=0.0001' \
  -i 'verbose=true' \
  -i 'lora_rank=32' \
  -i 'resolution=768' \
  -i 'lr_scheduler="constant"' \
  -i 'token_string="TOK"' \
  -i 'caption_prefix="a photo of TOK, "' \
  -i 'lr_warmup_steps=100' \
  -i 'max_train_steps=1000' \
  -i 'num_train_epochs=4000' \
  -i 'train_batch_size=4' \
  -i 'unet_learning_rate=0.000001' \
  -i 'checkpointing_steps=999999' \
  -i 'clipseg_temperature=1' \
  -i 'input_images_filetype="infer"' \
  -i 'crop_based_on_salience=true' \
  -i 'use_face_detection_instead=false'

To learn more, take a look at the Cog documentation.

Pull and run zylim0702/sdxl-lora-customize-training using Docker (this will download the full model and run it in your local environment):

docker run -d -p 5000:5000 --gpus=all r8.im/zylim0702/sdxl-lora-customize-training@sha256:a4eefb9bbc944eaefc56d67acba982ab73d85ec449de6606cf48b13f1483a495
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "ti_lr": 0.0003,
      "is_lora": true,
      "lora_lr": 0.0001,
      "verbose": true,
      "lora_rank": 32,
      "resolution": 768,
      "lr_scheduler": "constant",
      "token_string": "TOK",
      "caption_prefix": "a photo of TOK, ",
      "lr_warmup_steps": 100,
      "max_train_steps": 1000,
      "num_train_epochs": 4000,
      "train_batch_size": 4,
      "unet_learning_rate": 0.000001,
      "checkpointing_steps": 999999,
      "clipseg_temperature": 1,
      "input_images_filetype": "infer",
      "crop_based_on_salience": true,
      "use_face_detection_instead": false
    }
  }' \
  http://localhost:5000/predictions

Output

No output yet! Press "Submit" to start a prediction.