Readme
DiffEdit with Stable Diffusion
Diffuser implementation of “DiffEdit: Diffusion-based semantic image editing with mask guidance” with Stable Diffusion, for better sample efficiency.
https://github.com/huggingface/diffusers/releases/tag/v0.17.0
cjwbw
/
diffedit-stable-diffusion
Diffusion-based semantic image editing with mask guidance
Input
Run this model in Node.js with one line of code:
npx create-replicate --model=cjwbw/diffedit-stable-diffusion
or set up a project from scratch
Install Replicate’s Node.js client library:
npm install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import and set up the client:
import Replicate from "replicate";
const replicate = new Replicate({
auth: process.env.REPLICATE_API_TOKEN,
});Run cjwbw/diffedit-stable-diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run(
"cjwbw/diffedit-stable-diffusion:8c51da11578f75df306db64e27fd337b8b18de902f49b4404b0b340dced96f48",
{
input: {
image: "https://replicate.delivery/pbxt/IyIBdgYw3DehQHUxK1t0dkX5vmS7QYqzgHmeEYGaxSv4uDUE/origin.png",
source_prompt: "a bowl of fruits",
target_prompt: "a basket of fruits",
guidance_scale: 7.5,
num_inference_steps: 50
}
}
);
// To access the file URL:
console.log(output.url()); //=> "http://example.com"
// To write the file to disk:
fs.writeFile("my-image.png", output);To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:
pip install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import the client:
import replicateRun cjwbw/diffedit-stable-diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run(
"cjwbw/diffedit-stable-diffusion:8c51da11578f75df306db64e27fd337b8b18de902f49b4404b0b340dced96f48",
input={
"image": "https://replicate.delivery/pbxt/IyIBdgYw3DehQHUxK1t0dkX5vmS7QYqzgHmeEYGaxSv4uDUE/origin.png",
"source_prompt": "a bowl of fruits",
"target_prompt": "a basket of fruits",
"guidance_scale": 7.5,
"num_inference_steps": 50
}
)
print(output)To learn more, take a look at the guide on getting started with Python.
Set the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Run cjwbw/diffedit-stable-diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
-H "Content-Type: application/json" \
-H "Prefer: wait" \
-d $'{
"version": "8c51da11578f75df306db64e27fd337b8b18de902f49b4404b0b340dced96f48",
"input": {
"image": "https://replicate.delivery/pbxt/IyIBdgYw3DehQHUxK1t0dkX5vmS7QYqzgHmeEYGaxSv4uDUE/origin.png",
"source_prompt": "a bowl of fruits",
"target_prompt": "a basket of fruits",
"guidance_scale": 7.5,
"num_inference_steps": 50
}
}' \
https://api.replicate.com/v1/predictionsTo learn more, take a look at Replicate’s HTTP API reference docs.
Add a payment method to run this model.
Sign in with GitHub
By signing in, you agree to our
terms of service and privacy policy
Output
{
"completed_at": "2023-06-09T14:47:52.403818Z",
"created_at": "2023-06-09T14:43:56.305702Z",
"data_removed": false,
"error": null,
"id": "vra43nyuunhdxeb7jhipbjux6m",
"input": {
"image": "https://replicate.delivery/pbxt/IyIBdgYw3DehQHUxK1t0dkX5vmS7QYqzgHmeEYGaxSv4uDUE/origin.png",
"source_prompt": "a bowl of fruits",
"target_prompt": "a basket of fruits",
"guidance_scale": 7.5,
"num_inference_steps": 50
},
"logs": "Using seed: 43695\n/root/.pyenv/versions/3.10.12/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_diffedit.py:162: FutureWarning: The preprocess method is deprecated and will be removed in a future version. Please use VaeImageProcessor.preprocess instead\nwarnings.warn(\n 0%| | 0/39 [00:00<?, ?it/s]\n 3%|▎ | 1/39 [00:01<00:49, 1.31s/it]\n 5%|▌ | 2/39 [00:01<00:27, 1.33it/s]\n 8%|▊ | 3/39 [00:01<00:20, 1.80it/s]\n 10%|█ | 4/39 [00:02<00:16, 2.15it/s]\n 13%|█▎ | 5/39 [00:02<00:14, 2.42it/s]\n 15%|█▌ | 6/39 [00:02<00:12, 2.62it/s]\n 18%|█▊ | 7/39 [00:03<00:11, 2.77it/s]\n 21%|██ | 8/39 [00:03<00:10, 2.87it/s]\n 23%|██▎ | 9/39 [00:03<00:10, 2.94it/s]\n 26%|██▌ | 10/39 [00:04<00:09, 2.98it/s]\n 28%|██▊ | 11/39 [00:04<00:09, 3.04it/s]\n 31%|███ | 12/39 [00:04<00:08, 3.06it/s]\n 33%|███▎ | 13/39 [00:05<00:08, 3.07it/s]\n 36%|███▌ | 14/39 [00:05<00:08, 3.09it/s]\n 38%|███▊ | 15/39 [00:05<00:07, 3.09it/s]\n 41%|████ | 16/39 [00:06<00:07, 3.05it/s]\n 44%|████▎ | 17/39 [00:06<00:07, 3.07it/s]\n 46%|████▌ | 18/39 [00:06<00:06, 3.07it/s]\n 49%|████▊ | 19/39 [00:07<00:06, 3.08it/s]\n 51%|█████▏ | 20/39 [00:07<00:06, 3.07it/s]\n 54%|█████▍ | 21/39 [00:07<00:05, 3.08it/s]\n 56%|█████▋ | 22/39 [00:08<00:05, 3.08it/s]\n 59%|█████▉ | 23/39 [00:08<00:05, 3.09it/s]\n 62%|██████▏ | 24/39 [00:08<00:04, 3.08it/s]\n 64%|██████▍ | 25/39 [00:09<00:04, 3.09it/s]\n 67%|██████▋ | 26/39 [00:09<00:04, 3.09it/s]\n 69%|██████▉ | 27/39 [00:09<00:03, 3.09it/s]\n 72%|███████▏ | 28/39 [00:10<00:03, 3.09it/s]\n 74%|███████▍ | 29/39 [00:10<00:03, 3.09it/s]\n 77%|███████▋ | 30/39 [00:10<00:02, 3.08it/s]\n 79%|███████▉ | 31/39 [00:11<00:02, 3.10it/s]\n 82%|████████▏ | 32/39 [00:11<00:02, 3.11it/s]\n 85%|████████▍ | 33/39 [00:11<00:01, 3.10it/s]\n 87%|████████▋ | 34/39 [00:12<00:01, 3.10it/s]\n 90%|████████▉ | 35/39 [00:12<00:01, 3.09it/s]\n 92%|█████████▏| 36/39 [00:12<00:00, 3.09it/s]\n 95%|█████████▍| 37/39 [00:12<00:00, 3.10it/s]\n 97%|█████████▋| 38/39 [00:13<00:00, 3.09it/s]\n100%|██████████| 39/39 [00:13<00:00, 3.08it/s]\n100%|██████████| 39/39 [00:13<00:00, 2.86it/s]\n 0%| | 0/40 [00:00<?, ?it/s]\n 2%|▎ | 1/40 [00:00<00:23, 1.69it/s]\n 5%|▌ | 2/40 [00:00<00:16, 2.30it/s]\n 8%|▊ | 3/40 [00:01<00:14, 2.58it/s]\n 10%|█ | 4/40 [00:01<00:13, 2.75it/s]\n 12%|█▎ | 5/40 [00:01<00:12, 2.85it/s]\n 15%|█▌ | 6/40 [00:02<00:11, 2.93it/s]\n 18%|█▊ | 7/40 [00:02<00:10, 3.01it/s]\n 20%|██ | 8/40 [00:02<00:10, 3.00it/s]\n 22%|██▎ | 9/40 [00:03<00:10, 3.03it/s]\n 25%|██▌ | 10/40 [00:03<00:09, 3.05it/s]\n 28%|██▊ | 11/40 [00:03<00:09, 3.06it/s]\n 30%|███ | 12/40 [00:04<00:09, 3.07it/s]\n 32%|███▎ | 13/40 [00:04<00:08, 3.08it/s]\n 35%|███▌ | 14/40 [00:04<00:08, 3.09it/s]\n 38%|███▊ | 15/40 [00:05<00:08, 3.09it/s]\n 40%|████ | 16/40 [00:05<00:07, 3.08it/s]\n 42%|████▎ | 17/40 [00:05<00:07, 3.08it/s]\n 45%|████▌ | 18/40 [00:06<00:07, 3.07it/s]\n 48%|████▊ | 19/40 [00:06<00:06, 3.07it/s]\n 50%|█████ | 20/40 [00:06<00:06, 3.08it/s]\n 52%|█████▎ | 21/40 [00:07<00:06, 3.07it/s]\n 55%|█████▌ | 22/40 [00:07<00:05, 3.06it/s]\n 57%|█████▊ | 23/40 [00:07<00:05, 3.06it/s]\n 60%|██████ | 24/40 [00:08<00:05, 3.06it/s]\n 62%|██████▎ | 25/40 [00:08<00:04, 3.01it/s]\n 65%|██████▌ | 26/40 [00:08<00:04, 3.03it/s]\n 68%|██████▊ | 27/40 [00:09<00:04, 3.04it/s]\n 70%|███████ | 28/40 [00:09<00:03, 3.04it/s]\n 72%|███████▎ | 29/40 [00:09<00:03, 3.05it/s]\n 75%|███████▌ | 30/40 [00:10<00:03, 3.04it/s]\n 78%|███████▊ | 31/40 [00:10<00:02, 3.05it/s]\n 80%|████████ | 32/40 [00:10<00:02, 3.06it/s]\n 82%|████████▎ | 33/40 [00:11<00:02, 3.07it/s]\n 85%|████████▌ | 34/40 [00:11<00:01, 3.08it/s]\n 88%|████████▊ | 35/40 [00:11<00:01, 3.08it/s]\n 90%|█████████ | 36/40 [00:12<00:01, 3.08it/s]\n 92%|█████████▎| 37/40 [00:12<00:00, 3.06it/s]\n 95%|█████████▌| 38/40 [00:12<00:00, 3.04it/s]\n 98%|█████████▊| 39/40 [00:13<00:00, 3.03it/s]\n100%|██████████| 40/40 [00:13<00:00, 3.03it/s]\n100%|██████████| 40/40 [00:13<00:00, 3.00it/s]",
"metrics": {
"predict_time": 44.264369,
"total_time": 236.098116
},
"output": "https://replicate.delivery/pbxt/uQL969AiMJ6GIdI3iO2sxFik9amSKgpC3e0Tef4O1ohvsdIiA/out.png",
"started_at": "2023-06-09T14:47:08.139449Z",
"status": "succeeded",
"urls": {
"get": "https://api.replicate.com/v1/predictions/vra43nyuunhdxeb7jhipbjux6m",
"cancel": "https://api.replicate.com/v1/predictions/vra43nyuunhdxeb7jhipbjux6m/cancel"
},
"version": "8c51da11578f75df306db64e27fd337b8b18de902f49b4404b0b340dced96f48"
}Generated in
Using seed: 43695
/root/.pyenv/versions/3.10.12/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_diffedit.py:162: FutureWarning: The preprocess method is deprecated and will be removed in a future version. Please use VaeImageProcessor.preprocess instead
warnings.warn(
0%| | 0/39 [00:00<?, ?it/s]
3%|▎ | 1/39 [00:01<00:49, 1.31s/it]
5%|▌ | 2/39 [00:01<00:27, 1.33it/s]
8%|▊ | 3/39 [00:01<00:20, 1.80it/s]
10%|█ | 4/39 [00:02<00:16, 2.15it/s]
13%|█▎ | 5/39 [00:02<00:14, 2.42it/s]
15%|█▌ | 6/39 [00:02<00:12, 2.62it/s]
18%|█▊ | 7/39 [00:03<00:11, 2.77it/s]
21%|██ | 8/39 [00:03<00:10, 2.87it/s]
23%|██▎ | 9/39 [00:03<00:10, 2.94it/s]
26%|██▌ | 10/39 [00:04<00:09, 2.98it/s]
28%|██▊ | 11/39 [00:04<00:09, 3.04it/s]
31%|███ | 12/39 [00:04<00:08, 3.06it/s]
33%|███▎ | 13/39 [00:05<00:08, 3.07it/s]
36%|███▌ | 14/39 [00:05<00:08, 3.09it/s]
38%|███▊ | 15/39 [00:05<00:07, 3.09it/s]
41%|████ | 16/39 [00:06<00:07, 3.05it/s]
44%|████▎ | 17/39 [00:06<00:07, 3.07it/s]
46%|████▌ | 18/39 [00:06<00:06, 3.07it/s]
49%|████▊ | 19/39 [00:07<00:06, 3.08it/s]
51%|█████▏ | 20/39 [00:07<00:06, 3.07it/s]
54%|█████▍ | 21/39 [00:07<00:05, 3.08it/s]
56%|█████▋ | 22/39 [00:08<00:05, 3.08it/s]
59%|█████▉ | 23/39 [00:08<00:05, 3.09it/s]
62%|██████▏ | 24/39 [00:08<00:04, 3.08it/s]
64%|██████▍ | 25/39 [00:09<00:04, 3.09it/s]
67%|██████▋ | 26/39 [00:09<00:04, 3.09it/s]
69%|██████▉ | 27/39 [00:09<00:03, 3.09it/s]
72%|███████▏ | 28/39 [00:10<00:03, 3.09it/s]
74%|███████▍ | 29/39 [00:10<00:03, 3.09it/s]
77%|███████▋ | 30/39 [00:10<00:02, 3.08it/s]
79%|███████▉ | 31/39 [00:11<00:02, 3.10it/s]
82%|████████▏ | 32/39 [00:11<00:02, 3.11it/s]
85%|████████▍ | 33/39 [00:11<00:01, 3.10it/s]
87%|████████▋ | 34/39 [00:12<00:01, 3.10it/s]
90%|████████▉ | 35/39 [00:12<00:01, 3.09it/s]
92%|█████████▏| 36/39 [00:12<00:00, 3.09it/s]
95%|█████████▍| 37/39 [00:12<00:00, 3.10it/s]
97%|█████████▋| 38/39 [00:13<00:00, 3.09it/s]
100%|██████████| 39/39 [00:13<00:00, 3.08it/s]
100%|██████████| 39/39 [00:13<00:00, 2.86it/s]
0%| | 0/40 [00:00<?, ?it/s]
2%|▎ | 1/40 [00:00<00:23, 1.69it/s]
5%|▌ | 2/40 [00:00<00:16, 2.30it/s]
8%|▊ | 3/40 [00:01<00:14, 2.58it/s]
10%|█ | 4/40 [00:01<00:13, 2.75it/s]
12%|█▎ | 5/40 [00:01<00:12, 2.85it/s]
15%|█▌ | 6/40 [00:02<00:11, 2.93it/s]
18%|█▊ | 7/40 [00:02<00:10, 3.01it/s]
20%|██ | 8/40 [00:02<00:10, 3.00it/s]
22%|██▎ | 9/40 [00:03<00:10, 3.03it/s]
25%|██▌ | 10/40 [00:03<00:09, 3.05it/s]
28%|██▊ | 11/40 [00:03<00:09, 3.06it/s]
30%|███ | 12/40 [00:04<00:09, 3.07it/s]
32%|███▎ | 13/40 [00:04<00:08, 3.08it/s]
35%|███▌ | 14/40 [00:04<00:08, 3.09it/s]
38%|███▊ | 15/40 [00:05<00:08, 3.09it/s]
40%|████ | 16/40 [00:05<00:07, 3.08it/s]
42%|████▎ | 17/40 [00:05<00:07, 3.08it/s]
45%|████▌ | 18/40 [00:06<00:07, 3.07it/s]
48%|████▊ | 19/40 [00:06<00:06, 3.07it/s]
50%|█████ | 20/40 [00:06<00:06, 3.08it/s]
52%|█████▎ | 21/40 [00:07<00:06, 3.07it/s]
55%|█████▌ | 22/40 [00:07<00:05, 3.06it/s]
57%|█████▊ | 23/40 [00:07<00:05, 3.06it/s]
60%|██████ | 24/40 [00:08<00:05, 3.06it/s]
62%|██████▎ | 25/40 [00:08<00:04, 3.01it/s]
65%|██████▌ | 26/40 [00:08<00:04, 3.03it/s]
68%|██████▊ | 27/40 [00:09<00:04, 3.04it/s]
70%|███████ | 28/40 [00:09<00:03, 3.04it/s]
72%|███████▎ | 29/40 [00:09<00:03, 3.05it/s]
75%|███████▌ | 30/40 [00:10<00:03, 3.04it/s]
78%|███████▊ | 31/40 [00:10<00:02, 3.05it/s]
80%|████████ | 32/40 [00:10<00:02, 3.06it/s]
82%|████████▎ | 33/40 [00:11<00:02, 3.07it/s]
85%|████████▌ | 34/40 [00:11<00:01, 3.08it/s]
88%|████████▊ | 35/40 [00:11<00:01, 3.08it/s]
90%|█████████ | 36/40 [00:12<00:01, 3.08it/s]
92%|█████████▎| 37/40 [00:12<00:00, 3.06it/s]
95%|█████████▌| 38/40 [00:12<00:00, 3.04it/s]
98%|█████████▊| 39/40 [00:13<00:00, 3.03it/s]
100%|██████████| 40/40 [00:13<00:00, 3.03it/s]
100%|██████████| 40/40 [00:13<00:00, 3.00it/s]Examples
Run time and cost
This model costs approximately $0.063 to run on Replicate, or 15 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.
This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 5 minutes. The predict time for this model varies significantly based on the inputs.