anotherjesse/dreambooth – Run with an API on Replicate

anotherjesse / dreambooth

this is where I test new dreambooths

Cold

Public
276.5K runs
A100 (80GB)

Run with an API

Playground API Examples README Versions

Input

pretrained_model

string

Model identifier from huggingface.co/models

Default: "stabilityai/stable-diffusion-2-1"

instance_prompt

*string

Shift + Return to add a new line

The prompt you use to describe your training images, in the format: `a [identifier] [class noun]`, where the `[identifier]` should be a rare token. Relatively short sequences with 1-3 letters work the best (e.g. `sks`, `xjy`). `[class noun]` is a coarse class descriptor of the subject (e.g. cat, dog, watch, etc.). For example, your `instance_prompt` can be: `a sks dog`, or with some extra description `a photo of a sks dog`. The trained model will learn to bind a unique identifier with your specific subject in the `instance_data`.

class_prompt

*string

Shift + Return to add a new line

The prompt or description of the coarse class of your training images, in the format of `a [class noun]`, optionally with some extra description. `class_prompt` is used to alleviate overfitting to your customised images (the trained model should still keep the learnt prior so that it can still generate different dogs when the `[identifier]` is not in the prompt). Corresponding to the examples of the `instant_prompt` above, the `class_prompt` can be `a dog` or `a photo of a dog`.

with_prior_preservation

boolean

Flag to add prior preservation loss.

Default: true

prior_loss_weight

number

The weight of prior preservation loss.

Default: 1

instance_data

*file

A ZIP file containing your training images (JPG, PNG, etc. size not restricted). These images contain your 'subject' that you want the trained model to embed in the output domain for later generating customized scenes beyond the training images. For best results, use images without noise or unrelated objects in the background.

class_data

file

An optional ZIP file containing the training data of class images. This corresponds to `class_prompt` above, also with the purpose of keeping the model generalizable. By default, the pretrained stable-diffusion model will generate N images (determined by the `num_class_images` you set) based on the `class_prompt` provided. But to save time or to have your preferred specific set of `class_data`, you can also provide them in a ZIP file.

num_class_images

integer

Minimal class images for prior preservation loss. If not enough images are provided in class_data, additional images will be sampled with class_prompt.

Default: 50

seed

integer

A seed for reproducible training.

Default: 1337

resolution

integer

The resolution for input images. All the images in the train/validation dataset will be resized to this resolution.

Default: 768

center_crop

boolean

Whether to center crop the input images to the resolution. If not set, the images will be randomly cropped. The images will be resized to the resolution first before cropping.

Default: false

train_text_encoder

boolean

Whether to train the text encoder. If set, the text encoder should be float32 precision.

Default: true

train_batch_size

integer

Batch size (per device) for the training dataloader.

Default: 1

sample_batch_size

integer

Batch size (per device) for sampling images.

Default: 4

num_train_epochs

integer

Default: 1

max_train_steps

integer

Total number of training steps to perform. If provided, overrides num_train_epochs.

Default: 2000

gradient_accumulation_steps

integer

Number of updates steps to accumulate before performing a backward/update pass.

Default: 1

gradient_checkpointing

boolean

Whether or not to use gradient checkpointing to save memory at the expense of slower backward pass.

Default: false

learning_rate

number

Initial learning rate (after the potential warmup period) to use.

Default: 0.000005

scale_lr

boolean

Scale the learning rate by the number of GPUs, gradient accumulation steps, and batch size.

Default: false

lr_scheduler

string

The scheduler type to use

Default: "constant"

lr_warmup_steps

integer

Number of steps for the warmup in the lr scheduler.

Default: 0

lr_num_cycles

integer

Number of hard resets of the lr in cosine_with_restarts scheduler.

Default: 1

lr_power

number

Power factor of the polynomial scheduler.

Default: 1

use_8bit_adam

boolean

Whether or not to use 8-bit Adam from bitsandbytes.

Default: false

adam_beta1

number

The beta1 parameter for the Adam optimizer.

Default: 0.9

adam_beta2

number

The beta2 parameter for the Adam optimizer.

Default: 0.999

adam_weight_decay

number

Weight decay to use

Default: 0.01

adam_epsilon

number

Epsilon value for the Adam optimizer

Default: 1e-8

max_grad_norm

number

Max gradient norm.

Default: 1

hflip

boolean

Whether or not to horizontally flip training images 50 percent of the time.

Default: false

generate_images

*string

Shift + Return to add a new line

json of samples to generate: [{"name": "sample_name", "input": {"prompt": "a sks dog", "num_samples": 4, "save_guidance_scale": 7.5, "save_infer_steps": 50}}]

Run this model in Node.js with one line of code:

npx create-replicate --model=anotherjesse/dreambooth

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run anotherjesse/dreambooth using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "anotherjesse/dreambooth:926e61c5948d452a942e258d8a0a0f09a2c1749d2e2f153b3cccef6f18d6043f",
  {
    input: {
      seed: 1337,
      hflip: false,
      lr_power: 1,
      scale_lr: false,
      adam_beta1: 0.9,
      adam_beta2: 0.999,
      resolution: 768,
      center_crop: false,
      adam_epsilon: 1e-8,
      lr_scheduler: "constant",
      learning_rate: 0.000005,
      lr_num_cycles: 1,
      max_grad_norm: 1,
      use_8bit_adam: false,
      lr_warmup_steps: 0,
      max_train_steps: 2000,
      num_class_images: 50,
      num_train_epochs: 1,
      pretrained_model: "stabilityai/stable-diffusion-2-1",
      train_batch_size: 1,
      adam_weight_decay: 0.01,
      prior_loss_weight: 1,
      sample_batch_size: 4,
      train_text_encoder: true,
      gradient_checkpointing: false,
      with_prior_preservation: true,
      gradient_accumulation_steps: 1
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run anotherjesse/dreambooth using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "anotherjesse/dreambooth:926e61c5948d452a942e258d8a0a0f09a2c1749d2e2f153b3cccef6f18d6043f",
    input={
        "seed": 1337,
        "hflip": False,
        "lr_power": 1,
        "scale_lr": False,
        "adam_beta1": 0.9,
        "adam_beta2": 0.999,
        "resolution": 768,
        "center_crop": False,
        "adam_epsilon": 1e-8,
        "lr_scheduler": "constant",
        "learning_rate": 0.000005,
        "lr_num_cycles": 1,
        "max_grad_norm": 1,
        "use_8bit_adam": False,
        "lr_warmup_steps": 0,
        "max_train_steps": 2000,
        "num_class_images": 50,
        "num_train_epochs": 1,
        "pretrained_model": "stabilityai/stable-diffusion-2-1",
        "train_batch_size": 1,
        "adam_weight_decay": 0.01,
        "prior_loss_weight": 1,
        "sample_batch_size": 4,
        "train_text_encoder": True,
        "gradient_checkpointing": False,
        "with_prior_preservation": True,
        "gradient_accumulation_steps": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run anotherjesse/dreambooth using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "926e61c5948d452a942e258d8a0a0f09a2c1749d2e2f153b3cccef6f18d6043f",
    "input": {
      "seed": 1337,
      "hflip": false,
      "lr_power": 1,
      "scale_lr": false,
      "adam_beta1": 0.9,
      "adam_beta2": 0.999,
      "resolution": 768,
      "center_crop": false,
      "adam_epsilon": 1e-8,
      "lr_scheduler": "constant",
      "learning_rate": 0.000005,
      "lr_num_cycles": 1,
      "max_grad_norm": 1,
      "use_8bit_adam": false,
      "lr_warmup_steps": 0,
      "max_train_steps": 2000,
      "num_class_images": 50,
      "num_train_epochs": 1,
      "pretrained_model": "stabilityai/stable-diffusion-2-1",
      "train_batch_size": 1,
      "adam_weight_decay": 0.01,
      "prior_loss_weight": 1,
      "sample_batch_size": 4,
      "train_text_encoder": true,
      "gradient_checkpointing": false,
      "with_prior_preservation": true,
      "gradient_accumulation_steps": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/anotherjesse/dreambooth@sha256:926e61c5948d452a942e258d8a0a0f09a2c1749d2e2f153b3cccef6f18d6043f \
  -i 'seed=1337' \
  -i 'hflip=false' \
  -i 'lr_power=1' \
  -i 'scale_lr=false' \
  -i 'adam_beta1=0.9' \
  -i 'adam_beta2=0.999' \
  -i 'resolution=768' \
  -i 'center_crop=false' \
  -i 'adam_epsilon=1e-8' \
  -i 'lr_scheduler="constant"' \
  -i 'learning_rate=0.000005' \
  -i 'lr_num_cycles=1' \
  -i 'max_grad_norm=1' \
  -i 'use_8bit_adam=false' \
  -i 'lr_warmup_steps=0' \
  -i 'max_train_steps=2000' \
  -i 'num_class_images=50' \
  -i 'num_train_epochs=1' \
  -i 'pretrained_model="stabilityai/stable-diffusion-2-1"' \
  -i 'train_batch_size=1' \
  -i 'adam_weight_decay=0.01' \
  -i 'prior_loss_weight=1' \
  -i 'sample_batch_size=4' \
  -i 'train_text_encoder=true' \
  -i 'gradient_checkpointing=false' \
  -i 'with_prior_preservation=true' \
  -i 'gradient_accumulation_steps=1'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/anotherjesse/dreambooth@sha256:926e61c5948d452a942e258d8a0a0f09a2c1749d2e2f153b3cccef6f18d6043f
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "seed": 1337,
      "hflip": false,
      "lr_power": 1,
      "scale_lr": false,
      "adam_beta1": 0.9,
      "adam_beta2": 0.999,
      "resolution": 768,
      "center_crop": false,
      "adam_epsilon": 1e-8,
      "lr_scheduler": "constant",
      "learning_rate": 0.000005,
      "lr_num_cycles": 1,
      "max_grad_norm": 1,
      "use_8bit_adam": false,
      "lr_warmup_steps": 0,
      "max_train_steps": 2000,
      "num_class_images": 50,
      "num_train_epochs": 1,
      "pretrained_model": "stabilityai/stable-diffusion-2-1",
      "train_batch_size": 1,
      "adam_weight_decay": 0.01,
      "prior_loss_weight": 1,
      "sample_batch_size": 4,
      "train_text_encoder": true,
      "gradient_checkpointing": false,
      "with_prior_preservation": true,
      "gradient_accumulation_steps": 1
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

No output yet! Press "Submit" to start a prediction.

Run time and cost

This model runs on Nvidia A100 (80GB) GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

Please ignore this - use dreambooth api