usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5

usamaehsan / multi-controlnet-x-consistency-decoder-x-realestic-vision-v5

Multi-Controlnet + consistency-decoder + INPAINTING + realestic-vision-v5 + Prompt-Weight + Single-Controlnet

Cold

Public
3.4K runs
L40S

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

Prompt - using compel, use +++ to increase words weight:: doc: https://github.com/damian0815/compel/tree/main/doc || https://invoke-ai.github.io/InvokeAI/features/PROMPTS/#attention-weighting

lineart_image

file

Preview

Control image for canny controlnet

lineart_conditioning_scale

number

Conditioning scale for canny controlnet

Default: 1

tile_image

file

Control image for tile controlnet

tile_conditioning_scale

number

Conditioning scale for tile controlnet

Default: 1

brightness_image

file

Control image for brightness controlnet

brightness_conditioning_scale

number

Conditioning scale for brightness controlnet

Default: 1

inpainting_image

file

Control image for inpainting controlnet

mask_image

file

mask image for inpainting controlnet

inpainting_conditioning_scale

number

Conditioning scale for brightness controlnet

Default: 1

num_outputs

integer

(minimum: 1, maximum: 10)

Number of images to generate

Default: 1

max_width

integer

Max width/Resolution of image

Default: 512

max_height

integer

Max height/Resolution of image

Default: 512

scheduler

string

Choose a scheduler.

Default: "DDIM"

num_inference_steps

integer

Steps to run denoising

Default: 20

guidance_scale

number

(minimum: 0.1, maximum: 30)

Scale for classifier-free guidance

Default: 7

seed

integer

Seed

eta

number

Controls the amount of noise that is added to the input data during the denoising diffusion process. Higher value -> more noise

Default: 0

negative_prompt

string

Shift + Return to add a new line

Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low qualityLongbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality

Negative prompt - using compel, use +++ to increase words weight

Default: "Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality"

guess_mode

boolean

In this mode, the ControlNet encoder will try best to recognize the content of the input image even if you remove all prompts. The `guidance_scale` between 3.0 and 5.0 is recommended.

Default: false

disable_safety_check

boolean

Disable safety check. Use at your own risk!

Default: false

sorted_controlnets

string

Shift + Return to add a new line

Comma seperated string of controlnet names, list of names: tile, inpainting, lineart,depth ,scribble , brightness /// example value: tile, inpainting, lineart

Default: "tile, inpainting, lineart"

low_res_fix

boolean

Using controlnet tile- after image generation

Default: false

low_res_fix_resolution

integer

controlnet tile resolution- after image generation

Default: 768

low_res_fix_steps

integer

controlnet tile resolution- after image generation

Default: 10

low_res_fix_prompt

string

Shift + Return to add a new line

Default: "best quality"

low_res_fix_negative_prompt

string

Shift + Return to add a new line

blur, lowres, bad anatomy, bad hands, cropped, worst qualityblur, lowres, bad anatomy, bad hands, cropped, worst quality

Default: "blur, lowres, bad anatomy, bad hands, cropped, worst quality"

low_res_fix_guess_mode

boolean

low res fix - guess mode

Default: false

low_res_fix_conditioning_scale

number

Default: 1

Run this model in Node.js with one line of code:

npx create-replicate --model=usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5:c04024cf5ad3f583b177f69744406027c3d651f8b51452e0eda3ad04a89fc2de",
  {
    input: {
      eta: 0,
      prompt: "underwater++ world",
      max_width: 512,
      scheduler: "DDIM",
      guess_mode: false,
      max_height: 512,
      low_res_fix: false,
      num_outputs: 1,
      lineart_image: "https://replicate.delivery/pbxt/Jqzz9g4kPNdw0IyCqEGIXBHCr33Ai9agnbnZGdNVUUKoJqt3/955ad4a4680988283115264b5b45c211.jpg",
      guidance_scale: 7,
      negative_prompt: "Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
      low_res_fix_steps: 10,
      low_res_fix_prompt: "best quality",
      sorted_controlnets: "tile, inpainting, lineart",
      num_inference_steps: 20,
      disable_safety_check: true,
      low_res_fix_guess_mode: false,
      low_res_fix_resolution: 768,
      tile_conditioning_scale: 1,
      lineart_conditioning_scale: 1,
      low_res_fix_negative_prompt: "blur, lowres, bad anatomy, bad hands, cropped, worst quality",
      brightness_conditioning_scale: 1,
      inpainting_conditioning_scale: 1,
      low_res_fix_conditioning_scale: 1
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5:c04024cf5ad3f583b177f69744406027c3d651f8b51452e0eda3ad04a89fc2de",
    input={
        "eta": 0,
        "prompt": "underwater++ world",
        "max_width": 512,
        "scheduler": "DDIM",
        "guess_mode": False,
        "max_height": 512,
        "low_res_fix": False,
        "num_outputs": 1,
        "lineart_image": "https://replicate.delivery/pbxt/Jqzz9g4kPNdw0IyCqEGIXBHCr33Ai9agnbnZGdNVUUKoJqt3/955ad4a4680988283115264b5b45c211.jpg",
        "guidance_scale": 7,
        "negative_prompt": "Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
        "low_res_fix_steps": 10,
        "low_res_fix_prompt": "best quality",
        "sorted_controlnets": "tile, inpainting, lineart",
        "num_inference_steps": 20,
        "disable_safety_check": True,
        "low_res_fix_guess_mode": False,
        "low_res_fix_resolution": 768,
        "tile_conditioning_scale": 1,
        "lineart_conditioning_scale": 1,
        "low_res_fix_negative_prompt": "blur, lowres, bad anatomy, bad hands, cropped, worst quality",
        "brightness_conditioning_scale": 1,
        "inpainting_conditioning_scale": 1,
        "low_res_fix_conditioning_scale": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5:c04024cf5ad3f583b177f69744406027c3d651f8b51452e0eda3ad04a89fc2de",
    "input": {
      "eta": 0,
      "prompt": "underwater++ world",
      "max_width": 512,
      "scheduler": "DDIM",
      "guess_mode": false,
      "max_height": 512,
      "low_res_fix": false,
      "num_outputs": 1,
      "lineart_image": "https://replicate.delivery/pbxt/Jqzz9g4kPNdw0IyCqEGIXBHCr33Ai9agnbnZGdNVUUKoJqt3/955ad4a4680988283115264b5b45c211.jpg",
      "guidance_scale": 7,
      "negative_prompt": "Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
      "low_res_fix_steps": 10,
      "low_res_fix_prompt": "best quality",
      "sorted_controlnets": "tile, inpainting, lineart",
      "num_inference_steps": 20,
      "disable_safety_check": true,
      "low_res_fix_guess_mode": false,
      "low_res_fix_resolution": 768,
      "tile_conditioning_scale": 1,
      "lineart_conditioning_scale": 1,
      "low_res_fix_negative_prompt": "blur, lowres, bad anatomy, bad hands, cropped, worst quality",
      "brightness_conditioning_scale": 1,
      "inpainting_conditioning_scale": 1,
      "low_res_fix_conditioning_scale": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

seed-44665.png

{
  "completed_at": "2023-11-10T17:42:18.197496Z",
  "created_at": "2023-11-10T17:39:02.361897Z",
  "data_removed": false,
  "error": null,
  "id": "nixvsb3byk2c73cm6hq32v4boq",
  "input": {
    "eta": 0,
    "prompt": "underwater++ world",
    "max_width": 512,
    "scheduler": "DDIM",
    "guess_mode": false,
    "max_height": 512,
    "num_outputs": 1,
    "lineart_image": "https://replicate.delivery/pbxt/Jqzz9g4kPNdw0IyCqEGIXBHCr33Ai9agnbnZGdNVUUKoJqt3/955ad4a4680988283115264b5b45c211.jpg",
    "guidance_scale": 7,
    "negative_prompt": "Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
    "consistency_decoder": true,
    "num_inference_steps": 20,
    "disable_safety_check": true,
    "tile_conditioning_scale": 1,
    "depth_conditioning_scale": 1,
    "lineart_conditioning_scale": 1,
    "scribble_conditioning_scale": 1,
    "brightness_conditioning_scale": 1,
    "inpainting_conditioning_scale": 1
  },
  "logs": "You have disabled the safety checker for <class 'diffusers.pipelines.controlnet.pipeline_controlnet.StableDiffusionControlNetPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .\nUsing seed: 44665\n  0%|          | 0/20 [00:00<?, ?it/s]\n  5%|▌         | 1/20 [00:00<00:03,  4.83it/s]\n 20%|██        | 4/20 [00:00<00:01, 12.98it/s]\n 35%|███▌      | 7/20 [00:00<00:00, 16.54it/s]\n 50%|█████     | 10/20 [00:00<00:00, 18.42it/s]\n 60%|██████    | 12/20 [00:00<00:00, 18.76it/s]\n 70%|███████   | 14/20 [00:00<00:00, 18.86it/s]\n 85%|████████▌ | 17/20 [00:00<00:00, 19.89it/s]\n100%|██████████| 20/20 [00:01<00:00, 20.41it/s]\n100%|██████████| 20/20 [00:01<00:00, 18.04it/s]\nRunning consistency decoder...\nConsistency decoder took 11.307215690612793 seconds",
  "metrics": {
    "predict_time": 15.267783,
    "total_time": 195.835599
  },
  "output": [
    "https://replicate.delivery/pbxt/FMG5rpMM3GrCAVcfnqBFgNDb8mkSfA5mPg5mNJjwgaX51B3RA/seed-44665.png"
  ],
  "started_at": "2023-11-10T17:42:02.929713Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/nixvsb3byk2c73cm6hq32v4boq",
    "cancel": "https://api.replicate.com/v1/predictions/nixvsb3byk2c73cm6hq32v4boq/cancel"
  },
  "version": "59c1f5ce85cf1ea03e07508964ff41968cfad2a61820e3455930120ae2d594f5"
}

Generated in

15.3 seconds

Tweak itReport View full prediction

This output was created using a different version of the model, usamaehsan/multi-controlnet-x-consistency-decoder-x-realestic-vision-v5:59c1f5ce.

Examples

View more examples

Run time and cost

This model costs approximately $0.0099 to run on Replicate, or 101 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 11 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.