adirik/leditsplusplus | Run with an API on Replicate

adirik / leditsplusplus

LEdits++ for image editing (Updated 1 year, 2 months ago)

Cold

Public
776 runs
A100 (80GB)
GitHub
Paper
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

image

*file

Input image to edit.

num_inversion_steps

integer

(minimum: 1, maximum: 200)

Number of image inversion steps.

Default: 50

source_prompt

string

Shift + Return to add a new line

Prompt describing the input image that will be used for guidance during inversion. Guidance is disabled if the `source_prompt` is ``.

Default: ""

source_guidance_scale

number

(minimum: 1, maximum: 25)

Strength of guidance during inversion.

Default: 3.5

skip

number

(minimum: 0, maximum: 1)

Portion of initial steps that will be ignored for inversion and subsequent generation. Lower values will lead to stronger changes to the input image.

Default: 0.15

negative_prompt

string

Shift + Return to add a new line

Negative prompt for the first text encoder to guide the image generation. *optional*, defaults to None.

negative_prompt2

string

Shift + Return to add a new line

Negative prompt for the second text encoder to guide the image generation. *optional*, defaults to None if *negative_prompt* is also left empty, alternatively defaults to *negative_prompt* otherwise.

editing_prompts

string

Shift + Return to add a new line

Comma separated objects to add, remove or edit. Defaults to None, which inverts and reconstructs the input image.

reverse_editing_directions

string

Shift + Return to add a new line

Comma separated True or False boolean values indicating whether the corresponding prompt in `editing_prompts` should be increased or decreased to add, remove or edit. *optional*, defaults to `False`

edit_guidance_scale

string

Shift + Return to add a new line

Comma separated float values for each change specified in editing prompts list. *optional*, defaults to 5 if left empty.

edit_warmup_steps

integer

(minimum: 0, maximum: 100)

Number of diffusion steps (for each prompt) for which guidance is not applied

Default: 0

edit_threshold

string

Shift + Return to add a new line

Comma separated edit threshold float values for each editing prompt, threshold values should be proportional to the image region that is modified. *optional*, defaults to 0.9 if left empty.

Run this model in Node.js with one line of code:

npx create-replicate --model=adirik/leditsplusplus

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run adirik/leditsplusplus using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "adirik/leditsplusplus:ea5b4a96d43c51d9b5a579177b9044e631fbb4dbfaae51367a226a65663dfebe",
  {
    input: {
      skip: 0.3,
      image: "https://replicate.delivery/pbxt/Kdrtkd4IdmtW53B6l9tG1upqxL6xFMhXobvcQ27qayMQFAIA/girl_with_a_pearl_earring.jpeg",
      source_prompt: "",
      edit_threshold: "0.75",
      editing_prompts: "glasses",
      edit_warmup_steps: 8,
      edit_guidance_scale: "3.0",
      num_inversion_steps: 50,
      source_guidance_scale: 3.5,
      reverse_editing_directions: "False"
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run adirik/leditsplusplus using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "adirik/leditsplusplus:ea5b4a96d43c51d9b5a579177b9044e631fbb4dbfaae51367a226a65663dfebe",
    input={
        "skip": 0.3,
        "image": "https://replicate.delivery/pbxt/Kdrtkd4IdmtW53B6l9tG1upqxL6xFMhXobvcQ27qayMQFAIA/girl_with_a_pearl_earring.jpeg",
        "source_prompt": "",
        "edit_threshold": "0.75",
        "editing_prompts": "glasses",
        "edit_warmup_steps": 8,
        "edit_guidance_scale": "3.0",
        "num_inversion_steps": 50,
        "source_guidance_scale": 3.5,
        "reverse_editing_directions": "False"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run adirik/leditsplusplus using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "adirik/leditsplusplus:ea5b4a96d43c51d9b5a579177b9044e631fbb4dbfaae51367a226a65663dfebe",
    "input": {
      "skip": 0.3,
      "image": "https://replicate.delivery/pbxt/Kdrtkd4IdmtW53B6l9tG1upqxL6xFMhXobvcQ27qayMQFAIA/girl_with_a_pearl_earring.jpeg",
      "source_prompt": "",
      "edit_threshold": "0.75",
      "editing_prompts": "glasses",
      "edit_warmup_steps": 8,
      "edit_guidance_scale": "3.0",
      "num_inversion_steps": 50,
      "source_guidance_scale": 3.5,
      "reverse_editing_directions": "False"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-03-27T09:58:44.144141Z",
  "created_at": "2024-03-27T09:58:32.337830Z",
  "data_removed": false,
  "error": null,
  "id": "iots6ptb7k45tak4fwusfgatla",
  "input": {
    "skip": 0.3,
    "image": "https://replicate.delivery/pbxt/Kdrtkd4IdmtW53B6l9tG1upqxL6xFMhXobvcQ27qayMQFAIA/girl_with_a_pearl_earring.jpeg",
    "edit_threshold": "0.75",
    "editing_prompts": "glasses",
    "edit_warmup_steps": 8,
    "edit_guidance_scale": "3.0",
    "num_inversion_steps": 50,
    "source_guidance_scale": 3.5,
    "reverse_editing_directions": "False"
  },
  "logs": "Your input images far exceed the default resolution of the underlying diffusion model. The output images may contain severe artifacts! Consider down-sampling the input using the `height` and `width` parameters\n  0%|          | 0/50 [00:00<?, ?it/s]\n  4%|▍         | 2/50 [00:00<00:03, 13.15it/s]\n  8%|▊         | 4/50 [00:00<00:03, 12.97it/s]\n 12%|█▏        | 6/50 [00:00<00:03, 12.77it/s]\n 16%|█▌        | 8/50 [00:00<00:03, 12.73it/s]\n 20%|██        | 10/50 [00:00<00:03, 12.76it/s]\n 24%|██▍       | 12/50 [00:00<00:02, 12.75it/s]\n 28%|██▊       | 14/50 [00:01<00:02, 12.78it/s]\n 32%|███▏      | 16/50 [00:01<00:02, 12.85it/s]\n 36%|███▌      | 18/50 [00:01<00:02, 12.82it/s]\n 40%|████      | 20/50 [00:01<00:02, 12.79it/s]\n 44%|████▍     | 22/50 [00:01<00:02, 12.79it/s]\n 48%|████▊     | 24/50 [00:01<00:02, 12.70it/s]\n 52%|█████▏    | 26/50 [00:02<00:01, 12.67it/s]\n 56%|█████▌    | 28/50 [00:02<00:01, 12.70it/s]\n 60%|██████    | 30/50 [00:02<00:01, 12.70it/s]\n 64%|██████▍   | 32/50 [00:02<00:01, 12.63it/s]\n 68%|██████▊   | 34/50 [00:02<00:01, 12.60it/s]\n 72%|███████▏  | 36/50 [00:02<00:01, 12.62it/s]\n 76%|███████▌  | 38/50 [00:02<00:00, 12.60it/s]\n 80%|████████  | 40/50 [00:03<00:00, 12.63it/s]\n 84%|████████▍ | 42/50 [00:03<00:00, 12.67it/s]\n 88%|████████▊ | 44/50 [00:03<00:00, 12.63it/s]\n 92%|█████████▏| 46/50 [00:03<00:00, 12.60it/s]\n 96%|█████████▌| 48/50 [00:03<00:00, 12.56it/s]\n100%|██████████| 50/50 [00:03<00:00, 12.36it/s]\n100%|██████████| 50/50 [00:03<00:00, 12.66it/s]\n  0%|          | 0/50 [00:00<?, ?it/s]\n  2%|▏         | 1/50 [00:00<00:06,  7.98it/s]\n  4%|▍         | 2/50 [00:00<00:06,  7.96it/s]\n  6%|▌         | 3/50 [00:00<00:05,  7.95it/s]\n  8%|▊         | 4/50 [00:00<00:05,  7.95it/s]\n 10%|█         | 5/50 [00:00<00:05,  7.95it/s]\n 12%|█▏        | 6/50 [00:00<00:05,  7.93it/s]\n 14%|█▍        | 7/50 [00:00<00:05,  7.94it/s]\n 16%|█▌        | 8/50 [00:01<00:05,  7.94it/s]\n 18%|█▊        | 9/50 [00:01<00:05,  7.93it/s]\n 20%|██        | 10/50 [00:01<00:05,  7.93it/s]\n 22%|██▏       | 11/50 [00:01<00:04,  7.93it/s]\n 24%|██▍       | 12/50 [00:01<00:04,  7.93it/s]\n 26%|██▌       | 13/50 [00:01<00:04,  7.93it/s]\n 28%|██▊       | 14/50 [00:01<00:04,  7.74it/s]\n 30%|███       | 15/50 [00:01<00:04,  7.80it/s]\n 32%|███▏      | 16/50 [00:02<00:04,  7.84it/s]\n 34%|███▍      | 17/50 [00:02<00:04,  7.87it/s]\n 36%|███▌      | 18/50 [00:02<00:04,  7.89it/s]\n 38%|███▊      | 19/50 [00:02<00:03,  7.90it/s]\n 40%|████      | 20/50 [00:02<00:03,  7.91it/s]\n 42%|████▏     | 21/50 [00:02<00:03,  7.92it/s]\n 44%|████▍     | 22/50 [00:02<00:03,  7.92it/s]\n 46%|████▌     | 23/50 [00:02<00:03,  7.92it/s]\n 48%|████▊     | 24/50 [00:03<00:03,  7.93it/s]\n 50%|█████     | 25/50 [00:03<00:03,  7.93it/s]\n 52%|█████▏    | 26/50 [00:03<00:03,  7.93it/s]\n 54%|█████▍    | 27/50 [00:03<00:02,  7.92it/s]\n 56%|█████▌    | 28/50 [00:03<00:02,  7.93it/s]\n 58%|█████▊    | 29/50 [00:03<00:02,  7.91it/s]\n 60%|██████    | 30/50 [00:03<00:02,  7.91it/s]\n 62%|██████▏   | 31/50 [00:03<00:02,  7.91it/s]\n 64%|██████▍   | 32/50 [00:04<00:02,  7.92it/s]\n 66%|██████▌   | 33/50 [00:04<00:02,  7.92it/s]\n 68%|██████▊   | 34/50 [00:04<00:02,  7.92it/s]\n 70%|███████   | 35/50 [00:04<00:01,  7.92it/s]\n 72%|███████▏  | 36/50 [00:04<00:01,  7.92it/s]\n 74%|███████▍  | 37/50 [00:04<00:01,  7.76it/s]\n 76%|███████▌  | 38/50 [00:04<00:01,  7.80it/s]\n 78%|███████▊  | 39/50 [00:04<00:01,  7.84it/s]\n 80%|████████  | 40/50 [00:05<00:01,  7.87it/s]\n 82%|████████▏ | 41/50 [00:05<00:01,  7.87it/s]\n 84%|████████▍ | 42/50 [00:05<00:01,  7.89it/s]\n 86%|████████▌ | 43/50 [00:05<00:00,  7.90it/s]\n 88%|████████▊ | 44/50 [00:05<00:00,  7.92it/s]\n 90%|█████████ | 45/50 [00:05<00:00,  7.92it/s]\n 92%|█████████▏| 46/50 [00:05<00:00,  7.92it/s]\n 94%|█████████▍| 47/50 [00:05<00:00,  7.92it/s]\n 96%|█████████▌| 48/50 [00:06<00:00,  7.92it/s]\n 98%|█████████▊| 49/50 [00:06<00:00,  7.93it/s]\n100%|██████████| 50/50 [00:06<00:00,  7.93it/s]\n100%|██████████| 50/50 [00:06<00:00,  7.90it/s]",
  "metrics": {
    "predict_time": 11.797109,
    "total_time": 11.806311
  },
  "output": "https://replicate.delivery/pbxt/cyvUwrQY1KLzMJEODz13q4fcg0NpAcqAbeRGrPStsvPTfzIlA/output.png",
  "started_at": "2024-03-27T09:58:32.347032Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/iots6ptb7k45tak4fwusfgatla",
    "cancel": "https://api.replicate.com/v1/predictions/iots6ptb7k45tak4fwusfgatla/cancel"
  },
  "version": "18916a9500f503aa4aa92ec0b2dbf3cecfa1995ee2280b2033e80d50973af9f2"
}

Generated in

11.8 seconds

Tweak it Report View full prediction

Your input images far exceed the default resolution of the underlying diffusion model. The output images may contain severe artifacts! Consider down-sampling the input using the `height` and `width` parameters
  0%|          | 0/50 [00:00<?, ?it/s]
  4%|▍         | 2/50 [00:00<00:03, 13.15it/s]
  8%|▊         | 4/50 [00:00<00:03, 12.97it/s]
 12%|█▏        | 6/50 [00:00<00:03, 12.77it/s]
 16%|█▌        | 8/50 [00:00<00:03, 12.73it/s]
 20%|██        | 10/50 [00:00<00:03, 12.76it/s]
 24%|██▍       | 12/50 [00:00<00:02, 12.75it/s]
 28%|██▊       | 14/50 [00:01<00:02, 12.78it/s]
 32%|███▏      | 16/50 [00:01<00:02, 12.85it/s]
 36%|███▌      | 18/50 [00:01<00:02, 12.82it/s]
 40%|████      | 20/50 [00:01<00:02, 12.79it/s]
 44%|████▍     | 22/50 [00:01<00:02, 12.79it/s]
 48%|████▊     | 24/50 [00:01<00:02, 12.70it/s]
 52%|█████▏    | 26/50 [00:02<00:01, 12.67it/s]
 56%|█████▌    | 28/50 [00:02<00:01, 12.70it/s]
 60%|██████    | 30/50 [00:02<00:01, 12.70it/s]
 64%|██████▍   | 32/50 [00:02<00:01, 12.63it/s]
 68%|██████▊   | 34/50 [00:02<00:01, 12.60it/s]
 72%|███████▏  | 36/50 [00:02<00:01, 12.62it/s]
 76%|███████▌  | 38/50 [00:02<00:00, 12.60it/s]
 80%|████████  | 40/50 [00:03<00:00, 12.63it/s]
 84%|████████▍ | 42/50 [00:03<00:00, 12.67it/s]
 88%|████████▊ | 44/50 [00:03<00:00, 12.63it/s]
 92%|█████████▏| 46/50 [00:03<00:00, 12.60it/s]
 96%|█████████▌| 48/50 [00:03<00:00, 12.56it/s]
100%|██████████| 50/50 [00:03<00:00, 12.36it/s]
100%|██████████| 50/50 [00:03<00:00, 12.66it/s]
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:00<00:06,  7.98it/s]
  4%|▍         | 2/50 [00:00<00:06,  7.96it/s]
  6%|▌         | 3/50 [00:00<00:05,  7.95it/s]
  8%|▊         | 4/50 [00:00<00:05,  7.95it/s]
 10%|█         | 5/50 [00:00<00:05,  7.95it/s]
 12%|█▏        | 6/50 [00:00<00:05,  7.93it/s]
 14%|█▍        | 7/50 [00:00<00:05,  7.94it/s]
 16%|█▌        | 8/50 [00:01<00:05,  7.94it/s]
 18%|█▊        | 9/50 [00:01<00:05,  7.93it/s]
 20%|██        | 10/50 [00:01<00:05,  7.93it/s]
 22%|██▏       | 11/50 [00:01<00:04,  7.93it/s]
 24%|██▍       | 12/50 [00:01<00:04,  7.93it/s]
 26%|██▌       | 13/50 [00:01<00:04,  7.93it/s]
 28%|██▊       | 14/50 [00:01<00:04,  7.74it/s]
 30%|███       | 15/50 [00:01<00:04,  7.80it/s]
 32%|███▏      | 16/50 [00:02<00:04,  7.84it/s]
 34%|███▍      | 17/50 [00:02<00:04,  7.87it/s]
 36%|███▌      | 18/50 [00:02<00:04,  7.89it/s]
 38%|███▊      | 19/50 [00:02<00:03,  7.90it/s]
 40%|████      | 20/50 [00:02<00:03,  7.91it/s]
 42%|████▏     | 21/50 [00:02<00:03,  7.92it/s]
 44%|████▍     | 22/50 [00:02<00:03,  7.92it/s]
 46%|████▌     | 23/50 [00:02<00:03,  7.92it/s]
 48%|████▊     | 24/50 [00:03<00:03,  7.93it/s]
 50%|█████     | 25/50 [00:03<00:03,  7.93it/s]
 52%|█████▏    | 26/50 [00:03<00:03,  7.93it/s]
 54%|█████▍    | 27/50 [00:03<00:02,  7.92it/s]
 56%|█████▌    | 28/50 [00:03<00:02,  7.93it/s]
 58%|█████▊    | 29/50 [00:03<00:02,  7.91it/s]
 60%|██████    | 30/50 [00:03<00:02,  7.91it/s]
 62%|██████▏   | 31/50 [00:03<00:02,  7.91it/s]
 64%|██████▍   | 32/50 [00:04<00:02,  7.92it/s]
 66%|██████▌   | 33/50 [00:04<00:02,  7.92it/s]
 68%|██████▊   | 34/50 [00:04<00:02,  7.92it/s]
 70%|███████   | 35/50 [00:04<00:01,  7.92it/s]
 72%|███████▏  | 36/50 [00:04<00:01,  7.92it/s]
 74%|███████▍  | 37/50 [00:04<00:01,  7.76it/s]
 76%|███████▌  | 38/50 [00:04<00:01,  7.80it/s]
 78%|███████▊  | 39/50 [00:04<00:01,  7.84it/s]
 80%|████████  | 40/50 [00:05<00:01,  7.87it/s]
 82%|████████▏ | 41/50 [00:05<00:01,  7.87it/s]
 84%|████████▍ | 42/50 [00:05<00:01,  7.89it/s]
 86%|████████▌ | 43/50 [00:05<00:00,  7.90it/s]
 88%|████████▊ | 44/50 [00:05<00:00,  7.92it/s]
 90%|█████████ | 45/50 [00:05<00:00,  7.92it/s]
 92%|█████████▏| 46/50 [00:05<00:00,  7.92it/s]
 94%|█████████▍| 47/50 [00:05<00:00,  7.92it/s]
 96%|█████████▌| 48/50 [00:06<00:00,  7.92it/s]
 98%|█████████▊| 49/50 [00:06<00:00,  7.93it/s]
100%|██████████| 50/50 [00:06<00:00,  7.93it/s]
100%|██████████| 50/50 [00:06<00:00,  7.90it/s]

This output was created using a different version of the model, adirik/leditsplusplus:18916a95.

Examples

View more examples

Run time and cost

This model costs approximately $0.039 to run on Replicate, or 25 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 28 seconds. The predict time for this model varies significantly based on the inputs.

Readme

LEdits++

LEdits++ is a textual image editing method for Stable Diffusion XL and variants. See the paper, Hugging Face demo and docs for details.

How to use the API

To edit an image with LEdits++, upload an image and specify the objects you would like to remove or add as a comma separated string. In order to edit an object in place (e.g. changing an apple to an orange), both nouns need to included in the editing_prompts as removal and addition targets respectively. The full list of API arguments are as follows:

image: Input image to edit.
num_inversion_steps: Number of image inversion steps to retrieve the image latent code.
negative_prompt: Negative prompt for the first text encoder to guide the image generation. optional, defaults to None.
source_prompt: Prompt describing the input image that will be used for guidance during inversion. Guidance is disabled if the source_prompt is "".
source_guidance_scale: Strength of guidance during inversion.
skip: Portion of initial steps that will be ignored for inversion and subsequent generation. Lower values will lead to stronger changes to the input image.
negative_prompt2: Negative prompt for the second text encoder to guide the image generation. optional, defaults to None if negative_prompt is also left empty, alternatively defaults to negative_prompt otherwise.
editing_prompts: Comma separated objects to add, remove or edit. Defaults to None, which inverts and reconstructs the input image.
reverse_editing_directions: Comma separated True or False boolean values indicating whether the corresponding prompt in editing_prompts should be increased or decreased to add, remove or edit. optional, defaults to False.
edit_guidance_scale: Comma separated float values for each change specified in editing prompts list. optional, defaults to 5 if left empty.
edit_warmup_steps: Number of diffusion steps (for each prompt) for which guidance is not applied.
edit_threshold: Comma separated edit threshold float values for each editing prompt, threshold values should be proportional to the image region that is modified. optional, defaults to 0.9 if left empty.

LEdits++ only supports DPMSolverMultistepScheduler (default) and DDIMScheduler. Images that are larger than 1024x1024 are downsampled while preserving the aspect ratio.

References

@article{Brack2023LEDITSLI,
  title={LEDITS++: Limitless Image Editing using Text-to-Image Models},
  author={Manuel Brack and Felix Friedrich and Katharina Kornmeier and Linoy Tsaban and Patrick Schramowski and Kristian Kersting and Apolin'ario Passos},
  journal={ArXiv},
  year={2023},
  volume={abs/2311.16711},
  url={https://api.semanticscholar.org/CorpusID:265466786}
}