usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5

usamaehsan / controlnet-x-ip-adapter-realistic-vision-v5

Inpainting || multi-controlnet || single-controlnet || ip-adapter || ip adapter face || ip adapter plus || No ip adapter

Warm

Public
583.3K runs
L40S

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

a bookmark showcasing rippling water in a forest, depicted in a detailed illustration style. The artwork captures the moonlit water, accentuated by the reflection of the moon above. Rich and vibrant colors intensify the moonlit ambiance. The lighting casts dramatic shadows, enhancing the depth and texture of the scene. Pro vector, perfect MINIMALISTIC art HIGH QUALITY details, Ultra high details, full design, victoria, vibrant vector, deep lines, heavy strokesa bookmark showcasing rippling water in a forest, depicted in a detailed illustration style. The artwork captures the moonlit water, accentuated by the reflection of the moon above. Rich and vibrant colors intensify the moonlit ambiance. The lighting casts dramatic shadows, enhancing the depth and texture of the scene. Pro vector, perfect MINIMALISTIC art HIGH QUALITY details, Ultra high details, full design, victoria, vibrant vector, deep lines, heavy strokes

Prompt - using compel, use +++ to increase words weight:: doc: https://github.com/damian0815/compel/tree/main/doc || https://invoke-ai.github.io/InvokeAI/features/PROMPTS/#attention-weighting

negative_prompt

string

Shift + Return to add a new line

(worst quality)++ ,(low quality)++ ,(normal quality)++ , lowres ,watermark ,(worst quality)++ ,(low quality)++ ,(normal quality)++ , lowres ,watermark ,

Negative prompt - using compel, use +++ to increase words weight//// negative-embeddings available ///// FastNegativeV2 , boring_e621_v4 , verybadimagenegative_v1 || to use them, write their keyword in negative prompt

Default: "Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality"

num_inference_steps

integer

Steps to run denoising

Default: 20

guidance_scale

number

(minimum: 0.1, maximum: 30)

Scale for classifier-free guidance

Default: 7

seed

integer

Seed

disable_safety_check

boolean

Disable safety check. Use at your own risk!

Default: false

num_outputs

integer

(minimum: 1, maximum: 10)

Number of images to generate

Default: 1

max_width

integer

Max width/Resolution of image

Default: 512

max_height

integer

Max height/Resolution of image

Default: 512

scheduler

string

Choose a scheduler.

Default: "DDIM"

lineart_image

file

Control image for canny controlnet

lineart_conditioning_scale

number

Conditioning scale for canny controlnet

Default: 1

scribble_image

file

Preview

Control image for scribble controlnet

scribble_conditioning_scale

number

Conditioning scale for scribble controlnet

Default: 1

tile_image

file

Control image for tile controlnet

tile_conditioning_scale

number

Conditioning scale for tile controlnet

Default: 1

inpainting_image

file

Control image for inpainting controlnet

mask_image

file

mask image for inpainting controlnet

positive_auto_mask_text

string

Shift + Return to add a new line

// seperated list of objects for mask, AI will auto create mask of these objects, if mask text is given, mask image will not work - 'hairs // eyes // cloth'

negative_auto_mask_text

string

Shift + Return to add a new line

// seperated list of objects you dont want to mask - 'hairs // eyes // cloth'

inpainting_conditioning_scale

number

Conditioning scale for inpaint controlnet

Default: 1

inpainting_strength

number

inpainting strength

Default: 1

ip_adapter_ckpt

string

IP Adapter checkpoint

Default: "ip-adapter_sd15.bin"

ip_adapter_image

file

IP Adapter image

ip_adapter_weight

number

IP Adapter weight

Default: 1

Including eta and 12 more...

eta

number

Controls the amount of noise that is added to the input data during the denoising diffusion process. Higher value -> more noise

Default: 0

guess_mode

boolean

In this mode, the ControlNet encoder will try best to recognize the content of the input image even if you remove all prompts. The `guidance_scale` between 3.0 and 5.0 is recommended.

Default: false

brightness_image

file

Control image for brightness controlnet

brightness_conditioning_scale

number

Conditioning scale for brightness controlnet

Default: 1

sorted_controlnets

string

Shift + Return to add a new line

Comma seperated string of controlnet names, list of names: tile, inpainting, lineart,depth ,scribble , brightness /// example value: tile, inpainting, lineart

Default: "lineart, tile, inpainting"

img2img_image

file

Image2image image

img2img_strength

number

img2img strength, does not work when inpainting image is given, 0.1-same image, 0.99-complete destruction of image

Default: 0.5

add_more_detail_lora_scale

number

Scale/ weight of more_details lora, more scale = more details, disabled on 0

Default: 0.5

detail_tweaker_lora_weight

number

disabled on 0

Default: 0

film_grain_lora_weight

number

disabled on 0

Default: 0

epi_noise_offset_lora_weight

number

disabled on 0

Default: 0

color_temprature_slider_lora_weight

number

disabled on 0

Default: 0

int_kwargs

string

Shift + Return to add a new line

Default: ""

Run this model in Node.js with one line of code:

npx create-replicate --model=usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5:50ac06bb9bcf30e7b5dc66d3fe6e67262059a11ade572a35afa0ef686f55db82",
  {
    input: {
      eta: 0,
      prompt: "a bookmark showcasing rippling water in a forest, depicted in a detailed illustration style. The artwork captures the moonlit water, accentuated by the reflection of the moon above. Rich and vibrant colors intensify the moonlit ambiance. The lighting casts dramatic shadows, enhancing the depth and texture of the scene. Pro vector, perfect MINIMALISTIC art HIGH QUALITY details, Ultra high details, full design, victoria, vibrant vector, deep lines, heavy strokes",
      max_width: 512,
      scheduler: "K_EULER_ANCESTRAL",
      guess_mode: false,
      int_kwargs: "",
      max_height: 1024,
      num_outputs: 1,
      guidance_scale: 7,
      scribble_image: "https://replicate.delivery/pbxt/K7J0YI1KOZ88qUL7eT6HORXPIlXTXvuSkZv99Z1MIEAP5Lq6/Screenshot_20231226-060658.jpg",
      ip_adapter_ckpt: "ip-adapter-plus-face_sd15.bin",
      negative_prompt: "(worst quality)++ ,(low quality)++ ,(normal quality)++ , lowres ,watermark ,",
      img2img_strength: 0.5,
      ip_adapter_weight: 0.7,
      sorted_controlnets: "tile, inpainting, lineart",
      inpainting_strength: 1,
      num_inference_steps: 20,
      disable_safety_check: false,
      film_grain_lora_weight: 0,
      tile_conditioning_scale: 1,
      add_more_detail_lora_scale: 0,
      detail_tweaker_lora_weight: 0,
      lineart_conditioning_scale: 0.6,
      scribble_conditioning_scale: 0,
      epi_noise_offset_lora_weight: 0,
      brightness_conditioning_scale: 1,
      inpainting_conditioning_scale: 1,
      color_temprature_slider_lora_weight: 0
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5:50ac06bb9bcf30e7b5dc66d3fe6e67262059a11ade572a35afa0ef686f55db82",
    input={
        "eta": 0,
        "prompt": "a bookmark showcasing rippling water in a forest, depicted in a detailed illustration style. The artwork captures the moonlit water, accentuated by the reflection of the moon above. Rich and vibrant colors intensify the moonlit ambiance. The lighting casts dramatic shadows, enhancing the depth and texture of the scene. Pro vector, perfect MINIMALISTIC art HIGH QUALITY details, Ultra high details, full design, victoria, vibrant vector, deep lines, heavy strokes",
        "max_width": 512,
        "scheduler": "K_EULER_ANCESTRAL",
        "guess_mode": False,
        "int_kwargs": "",
        "max_height": 1024,
        "num_outputs": 1,
        "guidance_scale": 7,
        "scribble_image": "https://replicate.delivery/pbxt/K7J0YI1KOZ88qUL7eT6HORXPIlXTXvuSkZv99Z1MIEAP5Lq6/Screenshot_20231226-060658.jpg",
        "ip_adapter_ckpt": "ip-adapter-plus-face_sd15.bin",
        "negative_prompt": "(worst quality)++ ,(low quality)++ ,(normal quality)++ , lowres ,watermark ,",
        "img2img_strength": 0.5,
        "ip_adapter_weight": 0.7,
        "sorted_controlnets": "tile, inpainting, lineart",
        "inpainting_strength": 1,
        "num_inference_steps": 20,
        "disable_safety_check": False,
        "film_grain_lora_weight": 0,
        "tile_conditioning_scale": 1,
        "add_more_detail_lora_scale": 0,
        "detail_tweaker_lora_weight": 0,
        "lineart_conditioning_scale": 0.6,
        "scribble_conditioning_scale": 0,
        "epi_noise_offset_lora_weight": 0,
        "brightness_conditioning_scale": 1,
        "inpainting_conditioning_scale": 1,
        "color_temprature_slider_lora_weight": 0
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "50ac06bb9bcf30e7b5dc66d3fe6e67262059a11ade572a35afa0ef686f55db82",
    "input": {
      "eta": 0,
      "prompt": "a bookmark showcasing rippling water in a forest, depicted in a detailed illustration style. The artwork captures the moonlit water, accentuated by the reflection of the moon above. Rich and vibrant colors intensify the moonlit ambiance. The lighting casts dramatic shadows, enhancing the depth and texture of the scene. Pro vector, perfect MINIMALISTIC art HIGH QUALITY details, Ultra high details, full design, victoria, vibrant vector, deep lines, heavy strokes",
      "max_width": 512,
      "scheduler": "K_EULER_ANCESTRAL",
      "guess_mode": false,
      "int_kwargs": "",
      "max_height": 1024,
      "num_outputs": 1,
      "guidance_scale": 7,
      "scribble_image": "https://replicate.delivery/pbxt/K7J0YI1KOZ88qUL7eT6HORXPIlXTXvuSkZv99Z1MIEAP5Lq6/Screenshot_20231226-060658.jpg",
      "ip_adapter_ckpt": "ip-adapter-plus-face_sd15.bin",
      "negative_prompt": "(worst quality)++ ,(low quality)++ ,(normal quality)++ , lowres ,watermark ,",
      "img2img_strength": 0.5,
      "ip_adapter_weight": 0.7,
      "sorted_controlnets": "tile, inpainting, lineart",
      "inpainting_strength": 1,
      "num_inference_steps": 20,
      "disable_safety_check": false,
      "film_grain_lora_weight": 0,
      "tile_conditioning_scale": 1,
      "add_more_detail_lora_scale": 0,
      "detail_tweaker_lora_weight": 0,
      "lineart_conditioning_scale": 0.6,
      "scribble_conditioning_scale": 0,
      "epi_noise_offset_lora_weight": 0,
      "brightness_conditioning_scale": 1,
      "inpainting_conditioning_scale": 1,
      "color_temprature_slider_lora_weight": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-02-25T04:13:40.084131Z",
  "created_at": "2024-02-25T04:11:42.137873Z",
  "data_removed": false,
  "error": null,
  "id": "5hstjo3bfutm6u27jnzhbdkabq",
  "input": {
    "eta": 0,
    "prompt": "a bookmark showcasing rippling water in a forest, depicted in a detailed illustration style. The artwork captures the moonlit water, accentuated by the reflection of the moon above. Rich and vibrant colors intensify the moonlit ambiance. The lighting casts dramatic shadows, enhancing the depth and texture of the scene. Pro vector, perfect MINIMALISTIC art HIGH QUALITY details, Ultra high details, full design, victoria, vibrant vector, deep lines, heavy strokes",
    "max_width": 512,
    "scheduler": "K_EULER_ANCESTRAL",
    "guess_mode": false,
    "int_kwargs": "",
    "max_height": 1024,
    "num_outputs": 1,
    "guidance_scale": 7,
    "scribble_image": "https://replicate.delivery/pbxt/K7J0YI1KOZ88qUL7eT6HORXPIlXTXvuSkZv99Z1MIEAP5Lq6/Screenshot_20231226-060658.jpg",
    "ip_adapter_ckpt": "ip-adapter-plus-face_sd15.bin",
    "negative_prompt": "(worst quality)++ ,(low quality)++ ,(normal quality)++ , lowres ,watermark ,",
    "img2img_strength": 0.5,
    "ip_adapter_weight": 0.7,
    "sorted_controlnets": "tile, inpainting, lineart",
    "num_inference_steps": 20,
    "disable_safety_check": false,
    "film_grain_lora_weight": 0,
    "tile_conditioning_scale": 1,
    "add_more_detail_lora_scale": 0,
    "detail_tweaker_lora_weight": 0,
    "lineart_conditioning_scale": 0.6,
    "scribble_conditioning_scale": 0,
    "epi_noise_offset_lora_weight": 0,
    "brightness_conditioning_scale": 1,
    "inpainting_conditioning_scale": 1,
    "color_temprature_slider_lora_weight": 0
  },
  "logs": "Time taken until build pipe: 0.00 seconds\nusing ip adapter:: example/cat.png\nloading ip adapter\nTime taken to load IP adapter model: 0.12 seconds\nTime taken to load image encoder model: 0.00 seconds\nTime taken to build pipe: 1.80 seconds\nTime taken to apply scheduler-- : 0.00 seconds\nTime taken to cuda-- : 0.00 seconds\nUsing seed: 59960\nTime taken until ip -- : 0.04 seconds\nTime taken to load ip-- : 4.80 seconds\n  0%|          | 0/20 [00:00<?, ?it/s]\n  5%|▌         | 1/20 [00:00<00:04,  3.98it/s]\n 15%|█▌        | 3/20 [00:00<00:02,  7.71it/s]\n 20%|██        | 4/20 [00:00<00:01,  8.09it/s]\n 25%|██▌       | 5/20 [00:00<00:01,  8.34it/s]\n 30%|███       | 6/20 [00:00<00:01,  8.51it/s]\n 35%|███▌      | 7/20 [00:00<00:01,  8.63it/s]\n 40%|████      | 8/20 [00:00<00:01,  8.62it/s]\n 45%|████▌     | 9/20 [00:01<00:01,  8.75it/s]\n 50%|█████     | 10/20 [00:01<00:01,  8.79it/s]\n 55%|█████▌    | 11/20 [00:01<00:01,  8.81it/s]\n 60%|██████    | 12/20 [00:01<00:00,  8.87it/s]\n 65%|██████▌   | 13/20 [00:01<00:00,  8.86it/s]\n 70%|███████   | 14/20 [00:01<00:00,  8.85it/s]\n 75%|███████▌  | 15/20 [00:01<00:00,  8.88it/s]\n 80%|████████  | 16/20 [00:01<00:00,  8.88it/s]\n 85%|████████▌ | 17/20 [00:02<00:00,  8.83it/s]\n 90%|█████████ | 18/20 [00:02<00:00,  8.87it/s]\n 95%|█████████▌| 19/20 [00:02<00:00,  8.86it/s]\n100%|██████████| 20/20 [00:02<00:00,  8.78it/s]\n100%|██████████| 20/20 [00:02<00:00,  8.53it/s]\nTime taken to generate image-- : 2.60 seconds\n/root/.pyenv/versions/3.9.18/lib/python3.9/site-packages/peft/tuners/lora/layer.py:595: UserWarning: Already unmerged. Nothing to do.\nwarnings.warn(\"Already unmerged. Nothing to do.\")\n/root/.pyenv/versions/3.9.18/lib/python3.9/site-packages/peft/tuners/lora/layer.py:256: UserWarning: Already unmerged. Nothing to do.\nwarnings.warn(\"Already unmerged. Nothing to do.\")\nnot returning processed",
  "metrics": {
    "predict_time": 9.825593,
    "total_time": 117.946258
  },
  "output": [
    "https://replicate.delivery/pbxt/8I5Ibs22qcKMMh5VuT3abac9kgf91HB33Px5APKVe78zBHaSA/output_0.png"
  ],
  "started_at": "2024-02-25T04:13:30.258538Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/5hstjo3bfutm6u27jnzhbdkabq",
    "cancel": "https://api.replicate.com/v1/predictions/5hstjo3bfutm6u27jnzhbdkabq/cancel"
  },
  "version": "6192c21027f6975c3a87968bf5180a25ec1f7dd1f56293469aa76bd24a2341d0"
}

Generated in

9.8 seconds

Tweak itReport

Time taken until build pipe: 0.00 seconds
using ip adapter:: example/cat.png
loading ip adapter
Time taken to load IP adapter model: 0.12 seconds
Time taken to load image encoder model: 0.00 seconds
Time taken to build pipe: 1.80 seconds
Time taken to apply scheduler-- : 0.00 seconds
Time taken to cuda-- : 0.00 seconds
Using seed: 59960
Time taken until ip -- : 0.04 seconds
Time taken to load ip-- : 4.80 seconds
  0%|          | 0/20 [00:00<?, ?it/s]
  5%|▌         | 1/20 [00:00<00:04,  3.98it/s]
 15%|█▌        | 3/20 [00:00<00:02,  7.71it/s]
 20%|██        | 4/20 [00:00<00:01,  8.09it/s]
 25%|██▌       | 5/20 [00:00<00:01,  8.34it/s]
 30%|███       | 6/20 [00:00<00:01,  8.51it/s]
 35%|███▌      | 7/20 [00:00<00:01,  8.63it/s]
 40%|████      | 8/20 [00:00<00:01,  8.62it/s]
 45%|████▌     | 9/20 [00:01<00:01,  8.75it/s]
 50%|█████     | 10/20 [00:01<00:01,  8.79it/s]
 55%|█████▌    | 11/20 [00:01<00:01,  8.81it/s]
 60%|██████    | 12/20 [00:01<00:00,  8.87it/s]
 65%|██████▌   | 13/20 [00:01<00:00,  8.86it/s]
 70%|███████   | 14/20 [00:01<00:00,  8.85it/s]
 75%|███████▌  | 15/20 [00:01<00:00,  8.88it/s]
 80%|████████  | 16/20 [00:01<00:00,  8.88it/s]
 85%|████████▌ | 17/20 [00:02<00:00,  8.83it/s]
 90%|█████████ | 18/20 [00:02<00:00,  8.87it/s]
 95%|█████████▌| 19/20 [00:02<00:00,  8.86it/s]
100%|██████████| 20/20 [00:02<00:00,  8.78it/s]
100%|██████████| 20/20 [00:02<00:00,  8.53it/s]
Time taken to generate image-- : 2.60 seconds
/root/.pyenv/versions/3.9.18/lib/python3.9/site-packages/peft/tuners/lora/layer.py:595: UserWarning: Already unmerged. Nothing to do.
warnings.warn("Already unmerged. Nothing to do.")
/root/.pyenv/versions/3.9.18/lib/python3.9/site-packages/peft/tuners/lora/layer.py:256: UserWarning: Already unmerged. Nothing to do.
warnings.warn("Already unmerged. Nothing to do.")
not returning processed

This example was created by a different version, usamaehsan/controlnet-x-ip-adapter-realistic-vision-v5:6192c210.

Examples

View more examples

Run time and cost

This model costs approximately $0.0091 to run on Replicate, or 109 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 10 seconds.

Readme

Follow me on x/Twitter please : https://twitter.com/usamaClick