usamaehsan/flux-multi-controlnet – Run with an API on Replicate

usamaehsan / flux-multi-controlnet

Fast FLUX DEV -> Flux Controlnet Canny, Controlnet Depth , Controlnet Line Art, Controlnet Upscaler - You can use just one controlnet or All - LORAs: HyperFlex LoRA , Add Details LoRA , Realism LoRA

Warm

Public
158.9K runs
A100 (80GB)

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

A girl in city, 25 years old, cool, futuristicA girl in city, 25 years old, cool, futuristic

The text prompt that guides image generation

Default: "A girl in city, 25 years old, cool, futuristic style"

negative_prompt

string

Shift + Return to add a new line

A girl in city, 25 years old, cool, futuristic styleA girl in city, 25 years old, cool, futuristic style

The negative text prompt that guides image generation

Default: "A girl in city, 25 years old, cool, futuristic style"

canny_image

file

Input image for Canny ControlNet

depth_image

file

Input image for Depth ControlNet

lineart_image

file

Input image for Lineart ControlNet

upscaler_image

file

Input image for Upscaler ControlNet

canny_detector_type

string

Detector type to use with Canny ControlNet

Default: "canny"

depth_detector_type

string

Detector type to use with Depth ControlNet

Default: "depth"

lineart_detector_type

string

Detector type to use with Lineart ControlNet

Default: "lineart"

canny_strength

number

(minimum: 0, maximum: 2)

Strength for Canny ControlNet

Default: 0.6

depth_strength

number

(minimum: 0, maximum: 2)

Strength for Depth ControlNet

Default: 0.6

lineart_strength

number

(minimum: 0, maximum: 2)

Strength for Lineart ControlNet

Default: 0.6

upscaler_strength

number

(minimum: 0, maximum: 2)

Strength for Upscaler ControlNet

Default: 0.6

guidance_scale

number

(minimum: 0, maximum: 50)

Controls how closely the image follows the prompt

Default: 3.5

steps

integer

(minimum: 1, maximum: 50)

Number of denoising steps

Default: 8

seed

integer

Random seed for reproducibility

hyperflex_lora_weight

number

(maximum: 1)

Weight of the HyperFlex LoRA adaptation

Default: 0.125

add_details_lora_weight

number

(maximum: 1)

Weight of the Add Details LoRA adaptation

Default: 0

realism_lora_weight

number

(maximum: 1)

Weight of the Realism LoRA adaptation

Default: 0

visual_identity_design_weight

number

(minimum: 0, maximum: 1)

Weight of visual-identity-design LoRA

Default: 0

couple_profile_weight

number

(minimum: 0, maximum: 1)

Weight of couple-profile LoRA

Default: 0

home_decoration_weight

number

(minimum: 0, maximum: 1)

Weight of home-decoration LoRA

Default: 0

portrait_illustration_weight

number

(minimum: 0, maximum: 1)

Weight of portrait-illustration LoRA

Default: 0

portrait_photography_weight

number

(minimum: 0, maximum: 1)

Weight of portrait-photography LoRA

Default: 0

ppt_templates_weight

number

(minimum: 0, maximum: 1)

Weight of ppt-templates LoRA

Default: 0

consis_lora_weight

number

(minimum: 0, maximum: 1)

Weight of sparklers-visual-effect LoRA

Default: 0

pm_lora_weight

number

(minimum: 0, maximum: 1)

Weight of sparklers-visual-effect LoRA

Default: 0

migrationloracloth_weight

number

(minimum: 0, maximum: 1)

Weight of sparklers-visual-effect LoRA

Default: 0

image

file

Input image for inpainting

mask_image

file

Mask image for inpainting (white pixels will be repainted)

strength

number

(minimum: 0, maximum: 10)

Strength of inpainting (how much to repaint masked area)

Default: 0.6

padding_mask_crop

integer

Size of padding when cropping the mask

control_guidance_start

number

(minimum: 0, maximum: 1)

Percentage of steps at which ControlNet starts applying

Default: 0

control_guidance_end

number

(minimum: 0, maximum: 1)

Percentage of steps at which ControlNet stops applying

Default: 1

use_weighted_embeddings

boolean

Whether to use weighted text embeddings

Default: false

ip_adapter_image

file

Reference image for IP Adapter

ip_adapter_scale

number

(minimum: 0, maximum: 10)

Scale of influence for IP Adapter

Default: 0.5

ip_adapter_image_2

file

Second reference image for IP Adapter

ip_adapter_scale_2

number

(minimum: 0, maximum: 10)

Scale of influence for second IP Adapter image

Default: 0.5

ip_adapter_image_3

file

Third reference image for IP Adapter

ip_adapter_scale_3

number

(minimum: 0, maximum: 10)

Scale of influence for third IP Adapter image

Default: 0.5

negative_ip_adapter_image

file

Negative reference image for IP Adapter

negative_ip_adapter_scale

number

(minimum: 0, maximum: 10)

Scale of influence for negative IP Adapter image

Default: 0.5

auto_mask

boolean

Whether to use automatic masking

Default: false

auto_mask_prompt

string

Shift + Return to add a new line

Text prompt for automatic object detection and masking

auto_mask_box_threshold

number

(minimum: 0, maximum: 1)

Box threshold for automatic masking

Default: 0.3

auto_mask_text_threshold

number

(minimum: 0, maximum: 1)

Text threshold for automatic masking

Default: 0.25

use_background_mask

boolean

Whether to use background mask instead of object mask

Default: false

Including widthh and 1 more...

Run this model in Node.js with one line of code:

npx create-replicate --model=usamaehsan/flux-multi-controlnet

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run usamaehsan/flux-multi-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "usamaehsan/flux-multi-controlnet:40d4c564c30ce12f3d0ebd17584b8460bb61572caf96e7cab1f8d3cf038bbc15",
  {
    input: {
      seed: 55,
      steps: 8,
      prompt: "A girl in city, 25 years old, cool, futuristic",
      widthh: 0,
      heightt: 0,
      strength: 0.6,
      auto_mask: false,
      canny_strength: 0.6,
      depth_strength: 0.6,
      guidance_scale: 3.5,
      pm_lora_weight: 0,
      negative_prompt: "A girl in city, 25 years old, cool, futuristic style",
      ip_adapter_scale: 0.5,
      lineart_strength: 0.6,
      upscaler_strength: 0.6,
      consis_lora_weight: 0,
      ip_adapter_scale_2: 0.5,
      ip_adapter_scale_3: 0.5,
      canny_detector_type: "canny",
      depth_detector_type: "depth",
      realism_lora_weight: 0,
      use_background_mask: false,
      control_guidance_end: 1,
      ppt_templates_weight: 0,
      couple_profile_weight: 0,
      hyperflex_lora_weight: 0.125,
      lineart_detector_type: "lineart",
      control_guidance_start: 0,
      home_decoration_weight: 0,
      add_details_lora_weight: 0,
      auto_mask_box_threshold: 0.3,
      use_weighted_embeddings: false,
      auto_mask_text_threshold: 0.25,
      migrationloracloth_weight: 0,
      negative_ip_adapter_scale: 0.5,
      portrait_photography_weight: 0,
      portrait_illustration_weight: 0,
      visual_identity_design_weight: 0
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run usamaehsan/flux-multi-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "usamaehsan/flux-multi-controlnet:40d4c564c30ce12f3d0ebd17584b8460bb61572caf96e7cab1f8d3cf038bbc15",
    input={
        "seed": 55,
        "steps": 8,
        "prompt": "A girl in city, 25 years old, cool, futuristic",
        "widthh": 0,
        "heightt": 0,
        "strength": 0.6,
        "auto_mask": False,
        "canny_strength": 0.6,
        "depth_strength": 0.6,
        "guidance_scale": 3.5,
        "pm_lora_weight": 0,
        "negative_prompt": "A girl in city, 25 years old, cool, futuristic style",
        "ip_adapter_scale": 0.5,
        "lineart_strength": 0.6,
        "upscaler_strength": 0.6,
        "consis_lora_weight": 0,
        "ip_adapter_scale_2": 0.5,
        "ip_adapter_scale_3": 0.5,
        "canny_detector_type": "canny",
        "depth_detector_type": "depth",
        "realism_lora_weight": 0,
        "use_background_mask": False,
        "control_guidance_end": 1,
        "ppt_templates_weight": 0,
        "couple_profile_weight": 0,
        "hyperflex_lora_weight": 0.125,
        "lineart_detector_type": "lineart",
        "control_guidance_start": 0,
        "home_decoration_weight": 0,
        "add_details_lora_weight": 0,
        "auto_mask_box_threshold": 0.3,
        "use_weighted_embeddings": False,
        "auto_mask_text_threshold": 0.25,
        "migrationloracloth_weight": 0,
        "negative_ip_adapter_scale": 0.5,
        "portrait_photography_weight": 0,
        "portrait_illustration_weight": 0,
        "visual_identity_design_weight": 0
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run usamaehsan/flux-multi-controlnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "usamaehsan/flux-multi-controlnet:40d4c564c30ce12f3d0ebd17584b8460bb61572caf96e7cab1f8d3cf038bbc15",
    "input": {
      "seed": 55,
      "steps": 8,
      "prompt": "A girl in city, 25 years old, cool, futuristic",
      "widthh": 0,
      "heightt": 0,
      "strength": 0.6,
      "auto_mask": false,
      "canny_strength": 0.6,
      "depth_strength": 0.6,
      "guidance_scale": 3.5,
      "pm_lora_weight": 0,
      "negative_prompt": "A girl in city, 25 years old, cool, futuristic style",
      "ip_adapter_scale": 0.5,
      "lineart_strength": 0.6,
      "upscaler_strength": 0.6,
      "consis_lora_weight": 0,
      "ip_adapter_scale_2": 0.5,
      "ip_adapter_scale_3": 0.5,
      "canny_detector_type": "canny",
      "depth_detector_type": "depth",
      "realism_lora_weight": 0,
      "use_background_mask": false,
      "control_guidance_end": 1,
      "ppt_templates_weight": 0,
      "couple_profile_weight": 0,
      "hyperflex_lora_weight": 0.125,
      "lineart_detector_type": "lineart",
      "control_guidance_start": 0,
      "home_decoration_weight": 0,
      "add_details_lora_weight": 0,
      "auto_mask_box_threshold": 0.3,
      "use_weighted_embeddings": false,
      "auto_mask_text_threshold": 0.25,
      "migrationloracloth_weight": 0,
      "negative_ip_adapter_scale": 0.5,
      "portrait_photography_weight": 0,
      "portrait_illustration_weight": 0,
      "visual_identity_design_weight": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Generated in

11.1 seconds

Tweak it ShareReport

This output was created using a different version of the model, usamaehsan/flux-multi-controlnet:85977e12.

Examples

View more examples

Run time and cost

This model costs approximately $0.017 to run on Replicate, or 58 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 13 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.