philz1337x/controlnet-deliberate – Run with an API on Replicate

philz1337x / controlnet-deliberate

Modify images with canny edge detection and Deliberate model twitter: @philz1337x

Warm

Public
1.1M runs
A100 (80GB)

Run with an API

Playground API Examples README Versions

Input

image

*file

Input image

prompt

*string

Shift + Return to add a new line

RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3

Prompt for the model

num_samples

string

Number of samples (higher values may OOM)

Default: "1"

image_resolution

string

Image resolution to be generated

Default: "512"

weight

number

(minimum: 0.01, maximum: 2)

Weight of ControlNet

Default: 1

low_threshold

integer

(minimum: 1, maximum: 255)

Canny line detection low threshold

Default: 100

high_threshold

integer

(minimum: 1, maximum: 255)

Canny line detection high threshold

Default: 200

ddim_steps

integer

Steps

Default: 20

scale

number

(minimum: 0.1, maximum: 30)

Scale for classifier-free guidance

Default: 9

seed

integer

Seed

eta

number

Controls the amount of noise that is added to the input data during the denoising diffusion process. Higher value -> more noise

Default: 0

a_prompt

string

Shift + Return to add a new line

Additional text to be appended to prompt

Default: "best quality, extremely detailed"

n_prompt

string

Shift + Return to add a new line

longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low qualitylongbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality

Negative Prompt

Default: "longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality"

detect_resolution

integer

(minimum: 128, maximum: 1024)

Resolution at which detection method will be applied)

Default: 512

Run this model in Node.js with one line of code:

npx create-replicate --model=philz1337x/controlnet-deliberate

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run philz1337x/controlnet-deliberate using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "philz1337x/controlnet-deliberate:57d86bd78018d138449fda45bfcafb8b10888379a600034cc2c7186faab98c66",
  {
    input: {
      eta: 0,
      seed: -67,
      image: "https://replicate.delivery/pbxt/IW3cOsyRN9tZv2fRDZa8XmtQJ9ni2f1mm14dQyhKgHjGQfuN/Nofretete_Neues_Museum.jpg",
      scale: 9,
      prompt: "RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3",
      weight: 0.6,
      a_prompt: "best quality, extremely detailed",
      n_prompt: "longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
      ddim_steps: 20,
      num_samples: "1",
      low_threshold: 100,
      high_threshold: 200,
      image_resolution: "512",
      detect_resolution: 512
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run philz1337x/controlnet-deliberate using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "philz1337x/controlnet-deliberate:57d86bd78018d138449fda45bfcafb8b10888379a600034cc2c7186faab98c66",
    input={
        "eta": 0,
        "seed": -67,
        "image": "https://replicate.delivery/pbxt/IW3cOsyRN9tZv2fRDZa8XmtQJ9ni2f1mm14dQyhKgHjGQfuN/Nofretete_Neues_Museum.jpg",
        "scale": 9,
        "prompt": "RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3",
        "weight": 0.6,
        "a_prompt": "best quality, extremely detailed",
        "n_prompt": "longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
        "ddim_steps": 20,
        "num_samples": "1",
        "low_threshold": 100,
        "high_threshold": 200,
        "image_resolution": "512",
        "detect_resolution": 512
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run philz1337x/controlnet-deliberate using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "57d86bd78018d138449fda45bfcafb8b10888379a600034cc2c7186faab98c66",
    "input": {
      "eta": 0,
      "seed": -67,
      "image": "https://replicate.delivery/pbxt/IW3cOsyRN9tZv2fRDZa8XmtQJ9ni2f1mm14dQyhKgHjGQfuN/Nofretete_Neues_Museum.jpg",
      "scale": 9,
      "prompt": "RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3",
      "weight": 0.6,
      "a_prompt": "best quality, extremely detailed",
      "n_prompt": "longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
      "ddim_steps": 20,
      "num_samples": "1",
      "low_threshold": 100,
      "high_threshold": 200,
      "image_resolution": "512",
      "detect_resolution": 512
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/philz1337/controlnet-deliberate@sha256:57d86bd78018d138449fda45bfcafb8b10888379a600034cc2c7186faab98c66 \
  -i 'eta=0' \
  -i 'seed=-67' \
  -i 'image="https://replicate.delivery/pbxt/IW3cOsyRN9tZv2fRDZa8XmtQJ9ni2f1mm14dQyhKgHjGQfuN/Nofretete_Neues_Museum.jpg"' \
  -i 'scale=9' \
  -i 'prompt="RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3"' \
  -i 'weight=0.6' \
  -i 'a_prompt="best quality, extremely detailed"' \
  -i 'n_prompt="longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality"' \
  -i 'ddim_steps=20' \
  -i 'num_samples="1"' \
  -i 'low_threshold=100' \
  -i 'high_threshold=200' \
  -i 'image_resolution="512"' \
  -i 'detect_resolution=512'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/philz1337/controlnet-deliberate@sha256:57d86bd78018d138449fda45bfcafb8b10888379a600034cc2c7186faab98c66
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "eta": 0,
      "seed": -67,
      "image": "https://replicate.delivery/pbxt/IW3cOsyRN9tZv2fRDZa8XmtQJ9ni2f1mm14dQyhKgHjGQfuN/Nofretete_Neues_Museum.jpg",
      "scale": 9,
      "prompt": "RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3",
      "weight": 0.6,
      "a_prompt": "best quality, extremely detailed",
      "n_prompt": "longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
      "ddim_steps": 20,
      "num_samples": "1",
      "low_threshold": 100,
      "high_threshold": 200,
      "image_resolution": "512",
      "detect_resolution": 512
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2023-03-22T01:39:22.553294Z",
  "created_at": "2023-03-22T01:39:15.403386Z",
  "data_removed": false,
  "error": null,
  "id": "7siltbdylnbghgqbxffj4tkkmm",
  "input": {
    "seed": -67,
    "image": "https://replicate.delivery/pbxt/IW3cOsyRN9tZv2fRDZa8XmtQJ9ni2f1mm14dQyhKgHjGQfuN/Nofretete_Neues_Museum.jpg",
    "scale": 9,
    "prompt": "RAW photo, portrait photo of 30 y.o woman queen, pale skin, slim body, (high detailed skin:1.2), background ocean, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3",
    "weight": 0.6,
    "a_prompt": "best quality, extremely detailed",
    "n_prompt": "longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
    "ddim_steps": 20,
    "num_samples": "1",
    "low_threshold": 100,
    "high_threshold": 200,
    "image_resolution": "512",
    "detect_resolution": 512
  },
  "logs": "this is the weight value:  0.6\nthis is the control value:  tensor([[[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.],\n...,\n[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.]],\n[[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.],\n...,\n[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.]],\n[[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.],\n...,\n[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.]],\n...,\n[[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.],\n...,\n[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.]],\n[[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.],\n...,\n[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.]],\n[[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.],\n...,\n[0., 0., 0.],\n[0., 0., 0.],\n[0., 0., 0.]]], device='cuda:0')\nGlobal seed set to 3536601535\nData shape for DDIM sampling is (1, 4, 96, 64), eta 0.0\nRunning DDIM Sampling with 20 timesteps\nDDIM Sampler:   0%|          | 0/20 [00:00<?, ?it/s]\nDDIM Sampler:   5%|▌         | 1/20 [00:00<00:05,  3.37it/s]\nDDIM Sampler:  10%|█         | 2/20 [00:00<00:05,  3.57it/s]\nDDIM Sampler:  15%|█▌        | 3/20 [00:00<00:04,  3.65it/s]\nDDIM Sampler:  20%|██        | 4/20 [00:01<00:04,  3.68it/s]\nDDIM Sampler:  25%|██▌       | 5/20 [00:01<00:04,  3.71it/s]\nDDIM Sampler:  30%|███       | 6/20 [00:01<00:03,  3.71it/s]\nDDIM Sampler:  35%|███▌      | 7/20 [00:01<00:03,  3.72it/s]\nDDIM Sampler:  40%|████      | 8/20 [00:02<00:03,  3.73it/s]\nDDIM Sampler:  45%|████▌     | 9/20 [00:02<00:02,  3.73it/s]\nDDIM Sampler:  50%|█████     | 10/20 [00:02<00:02,  3.73it/s]\nDDIM Sampler:  55%|█████▌    | 11/20 [00:02<00:02,  3.73it/s]\nDDIM Sampler:  60%|██████    | 12/20 [00:03<00:02,  3.73it/s]\nDDIM Sampler:  65%|██████▌   | 13/20 [00:03<00:01,  3.74it/s]\nDDIM Sampler:  70%|███████   | 14/20 [00:03<00:01,  3.74it/s]\nDDIM Sampler:  75%|███████▌  | 15/20 [00:04<00:01,  3.74it/s]\nDDIM Sampler:  80%|████████  | 16/20 [00:04<00:01,  3.74it/s]\nDDIM Sampler:  85%|████████▌ | 17/20 [00:04<00:00,  3.74it/s]\nDDIM Sampler:  90%|█████████ | 18/20 [00:04<00:00,  3.74it/s]\nDDIM Sampler:  95%|█████████▌| 19/20 [00:05<00:00,  3.74it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:05<00:00,  3.75it/s]\nDDIM Sampler: 100%|██████████| 20/20 [00:05<00:00,  3.72it/s]",
  "metrics": {
    "predict_time": 7.07353,
    "total_time": 7.149908
  },
  "output": [
    "https://replicate.delivery/pbxt/cxCQfEx0yFwXQqgef7ePwxISiOu3Bi9g0qA1MjkY32ikkDoCB/output_0.png",
    "https://replicate.delivery/pbxt/0RfnrNN7i50qLqJtwmOn2HDOwQO0cEKMABPBf76qECGJ5AqQA/output_1.png"
  ],
  "started_at": "2023-03-22T01:39:15.479764Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/7siltbdylnbghgqbxffj4tkkmm",
    "cancel": "https://api.replicate.com/v1/predictions/7siltbdylnbghgqbxffj4tkkmm/cancel"
  },
  "version": "57d86bd78018d138449fda45bfcafb8b10888379a600034cc2c7186faab98c66"
}

Generated in

7.1 seconds

Tweak itReport

this is the weight value:  0.6
this is the control value:  tensor([[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
...,
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
...,
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
...,
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
...,
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
...,
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
...,
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
...,
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]]], device='cuda:0')
Global seed set to 3536601535
Data shape for DDIM sampling is (1, 4, 96, 64), eta 0.0
Running DDIM Sampling with 20 timesteps
DDIM Sampler:   0%|          | 0/20 [00:00<?, ?it/s]
DDIM Sampler:   5%|▌         | 1/20 [00:00<00:05,  3.37it/s]
DDIM Sampler:  10%|█         | 2/20 [00:00<00:05,  3.57it/s]
DDIM Sampler:  15%|█▌        | 3/20 [00:00<00:04,  3.65it/s]
DDIM Sampler:  20%|██        | 4/20 [00:01<00:04,  3.68it/s]
DDIM Sampler:  25%|██▌       | 5/20 [00:01<00:04,  3.71it/s]
DDIM Sampler:  30%|███       | 6/20 [00:01<00:03,  3.71it/s]
DDIM Sampler:  35%|███▌      | 7/20 [00:01<00:03,  3.72it/s]
DDIM Sampler:  40%|████      | 8/20 [00:02<00:03,  3.73it/s]
DDIM Sampler:  45%|████▌     | 9/20 [00:02<00:02,  3.73it/s]
DDIM Sampler:  50%|█████     | 10/20 [00:02<00:02,  3.73it/s]
DDIM Sampler:  55%|█████▌    | 11/20 [00:02<00:02,  3.73it/s]
DDIM Sampler:  60%|██████    | 12/20 [00:03<00:02,  3.73it/s]
DDIM Sampler:  65%|██████▌   | 13/20 [00:03<00:01,  3.74it/s]
DDIM Sampler:  70%|███████   | 14/20 [00:03<00:01,  3.74it/s]
DDIM Sampler:  75%|███████▌  | 15/20 [00:04<00:01,  3.74it/s]
DDIM Sampler:  80%|████████  | 16/20 [00:04<00:01,  3.74it/s]
DDIM Sampler:  85%|████████▌ | 17/20 [00:04<00:00,  3.74it/s]
DDIM Sampler:  90%|█████████ | 18/20 [00:04<00:00,  3.74it/s]
DDIM Sampler:  95%|█████████▌| 19/20 [00:05<00:00,  3.74it/s]
DDIM Sampler: 100%|██████████| 20/20 [00:05<00:00,  3.75it/s]
DDIM Sampler: 100%|██████████| 20/20 [00:05<00:00,  3.72it/s]

Examples

View more examples

Run time and cost

This model costs approximately $0.0075 to run on Replicate, or 133 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 6 seconds.

Readme

X/Twitter: @philz1337x

🦄 This model uses ControlNet to control image generation with SD.

It’s based on https://github.com/lllyasviel/ControlNet but I made two modifications:

It uses the community model Deliberate based on SD 1.5 (https://civitai.com/models/4823/deliberate)
I added weight to adjust the strength of canny influence on image control.