Input

prompt

string

Shift + Return to add a new line

A bohemian-style female travel blogger with sun-kissed skin and messy beach wavesA bohemian-style female travel blogger with sun-kissed skin and messy beach waves

Input prompt

Default: "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves"

guidance_scale

number

(minimum: 0, maximum: 5)

Guidance scale

Default: 3.5

steps

integer

(minimum: 1, maximum: 50)

Number of steps

Default: 28

control_type

string

Control type

Default: "canny"

control_strength

number

(minimum: 0, maximum: 1)

ControlNet strength, depth works best at 0.2, canny works best at 0.4. Recommended range is 0.3-0.8

Default: 0.2

control_image

file

Control image

Default: ""

seed

integer

Set a seed for reproducibility. Random by default.

Run this model in Node.js with one line of code:

npx create-replicate --model=lucataco/controlnet-union-pro

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run lucataco/controlnet-union-pro using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "lucataco/controlnet-union-pro:bd0dacc60e6247a2a4c28502434a6d6dd5d63f93c39950e5f366775d7a9b114a",
  {
    input: {
      steps: 28,
      prompt: "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves",
      control_type: "pose",
      control_image: "https://replicate.delivery/pbxt/LWGXlTeS6tfDc0J0PqzYhDYePQs7f62kfhc195388gzTikWu/assets_openpose.jpg",
      guidance_scale: 3.5,
      control_strength: 0.3
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run lucataco/controlnet-union-pro using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "lucataco/controlnet-union-pro:bd0dacc60e6247a2a4c28502434a6d6dd5d63f93c39950e5f366775d7a9b114a",
    input={
        "steps": 28,
        "prompt": "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves",
        "control_type": "pose",
        "control_image": "https://replicate.delivery/pbxt/LWGXlTeS6tfDc0J0PqzYhDYePQs7f62kfhc195388gzTikWu/assets_openpose.jpg",
        "guidance_scale": 3.5,
        "control_strength": 0.3
    }
)

# To access the file URL:
print(output.url())
#=> "http://example.com"

# To write the file to disk:
with open("my-image.png", "wb") as file:
    file.write(output.read())

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run lucataco/controlnet-union-pro using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "lucataco/controlnet-union-pro:bd0dacc60e6247a2a4c28502434a6d6dd5d63f93c39950e5f366775d7a9b114a",
    "input": {
      "steps": 28,
      "prompt": "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves",
      "control_type": "pose",
      "control_image": "https://replicate.delivery/pbxt/LWGXlTeS6tfDc0J0PqzYhDYePQs7f62kfhc195388gzTikWu/assets_openpose.jpg",
      "guidance_scale": 3.5,
      "control_strength": 0.3
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-08-27T15:56:57.507381Z",
  "created_at": "2024-08-27T15:56:40.374000Z",
  "data_removed": false,
  "error": null,
  "id": "10aenmdm6srgj0chjj6vff7s8m",
  "input": {
    "steps": 28,
    "prompt": "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves",
    "control_type": "pose",
    "control_image": "https://replicate.delivery/pbxt/LWGXlTeS6tfDc0J0PqzYhDYePQs7f62kfhc195388gzTikWu/assets_openpose.jpg",
    "guidance_scale": 3.5,
    "control_strength": 0.3
  },
  "logs": "Using seed: 53768\n  0%|          | 0/28 [00:00<?, ?it/s]\n  4%|▎         | 1/28 [00:00<00:14,  1.92it/s]\n  7%|▋         | 2/28 [00:00<00:11,  2.18it/s]\n 11%|█         | 3/28 [00:01<00:12,  2.05it/s]\n 14%|█▍        | 4/28 [00:01<00:12,  1.99it/s]\n 18%|█▊        | 5/28 [00:02<00:11,  1.94it/s]\n 21%|██▏       | 6/28 [00:03<00:11,  1.93it/s]\n 25%|██▌       | 7/28 [00:03<00:10,  1.92it/s]\n 29%|██▊       | 8/28 [00:04<00:10,  1.91it/s]\n 32%|███▏      | 9/28 [00:04<00:09,  1.91it/s]\n 36%|███▌      | 10/28 [00:05<00:09,  1.91it/s]\n 39%|███▉      | 11/28 [00:05<00:08,  1.90it/s]\n 43%|████▎     | 12/28 [00:06<00:08,  1.90it/s]\n 46%|████▋     | 13/28 [00:06<00:07,  1.90it/s]\n 50%|█████     | 14/28 [00:07<00:07,  1.90it/s]\n 54%|█████▎    | 15/28 [00:07<00:06,  1.90it/s]\n 57%|█████▋    | 16/28 [00:08<00:06,  1.90it/s]\n 61%|██████    | 17/28 [00:08<00:05,  1.90it/s]\n 64%|██████▍   | 18/28 [00:09<00:05,  1.90it/s]\n 68%|██████▊   | 19/28 [00:09<00:04,  1.90it/s]\n 71%|███████▏  | 20/28 [00:10<00:04,  1.90it/s]\n 75%|███████▌  | 21/28 [00:10<00:03,  1.89it/s]\n 79%|███████▊  | 22/28 [00:11<00:03,  1.90it/s]\n 82%|████████▏ | 23/28 [00:12<00:02,  1.89it/s]\n 86%|████████▌ | 24/28 [00:12<00:02,  1.89it/s]\n 89%|████████▉ | 25/28 [00:13<00:01,  1.89it/s]\n 93%|█████████▎| 26/28 [00:13<00:01,  1.89it/s]\n 96%|█████████▋| 27/28 [00:14<00:00,  1.89it/s]\n100%|██████████| 28/28 [00:14<00:00,  1.89it/s]\n100%|██████████| 28/28 [00:14<00:00,  1.91it/s]",
  "metrics": {
    "predict_time": 17.018506061,
    "total_time": 17.133381
  },
  "output": "https://replicate.delivery/pbxt/5KdaQjcyCZLAMNeFugMZpMu8w74ac3Kokw7eVb6aOyqIl6WTA/output.png",
  "started_at": "2024-08-27T15:56:40.488875Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/10aenmdm6srgj0chjj6vff7s8m",
    "cancel": "https://api.replicate.com/v1/predictions/10aenmdm6srgj0chjj6vff7s8m/cancel"
  },
  "version": "bd0dacc60e6247a2a4c28502434a6d6dd5d63f93c39950e5f366775d7a9b114a"
}

Generated in

17.0 seconds

Tweak it ShareReport View full prediction

Examples

View more examples

Run time and cost

This model costs approximately $0.016 to run on Replicate, or 62 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 17 seconds. The predict time for this model varies significantly based on the inputs.

Readme

FLUX.1-dev-ControlNet-Union-Pro

This repository contains a unified ControlNet for FLUX.1-dev model jointly released by researchers from InstantX Team and Shakker Labs.

Model Cards

This checkpoint is a Pro version of FLUX.1-dev-Controlnet-Union trained with more steps and datasets.
This model supports 7 control modes, including canny (0), tile (1), depth (2), blur (3), pose (4), gray (5), low quality (6).
The recommended controlnet_conditioning_scale is 0.3-0.8.
This model can be jointly used with other ControlNets.

Showcases

Resources

Acknowledgements

This project is trained by InstantX Team and sponsored by Shakker AI. The original idea is inspired by xinsir/controlnet-union-sdxl-1.0. All copyright reserved.