anotherjesse/multi-control – Run with an API on Replicate

anotherjesse / multi-control

All the original Controlnets & QR

Cold

Public
60.5K runs
A100 (80GB)
GitHub

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

whippet, flemish baroque, el yunque rainforest, 35mm film, quadtone color grading, chromakey
whippet, flemish baroque, el yunque rainforest, 35mm film, quadtone color grading, chromakey

Prompt for the model

canny_image

file

Control image for canny controlnet

canny_conditioning_scale

number

Conditioning scale for canny controlnet

Default: 1

depth_image

file

Control image for depth controlnet

depth_conditioning_scale

number

Conditioning scale for depth controlnet

Default: 1

hed_image

file

Control image for hed controlnet

hed_conditioning_scale

number

Conditioning scale for hed controlnet

Default: 1

hough_image

file

Control image for hough controlnet

hough_conditioning_scale

number

Conditioning scale for hough controlnet

Default: 1

normal_image

file

Control image for normal controlnet

normal_conditioning_scale

number

Conditioning scale for normal controlnet

Default: 1

pose_image

file

Control image for pose controlnet

pose_conditioning_scale

number

Conditioning scale for pose controlnet

Default: 1

scribble_image

file

Control image for scribble controlnet

scribble_conditioning_scale

number

Conditioning scale for scribble controlnet

Default: 1

seg_image

file

Control image for seg controlnet

seg_conditioning_scale

number

Conditioning scale for seg controlnet

Default: 1

qr_image

file

Preview

Control image for qr controlnet

qr_conditioning_scale

number

Conditioning scale for qr controlnet

Default: 1

num_outputs

integer

(minimum: 1, maximum: 10)

Number of images to generate

Default: 1

image_resolution

integer

Resolution of image (smallest dimension)

Default: 512

scheduler

string

Choose a scheduler.

Default: "DDIM"

num_inference_steps

integer

Steps to run denoising

Default: 20

guidance_scale

number

(minimum: 0.1, maximum: 30)

Scale for classifier-free guidance

Default: 9

seed

integer

Seed

eta

number

Controls the amount of noise that is added to the input data during the denoising diffusion process. Higher value -> more noise

Default: 0

negative_prompt

string

Shift + Return to add a new line

Negative prompt

Default: "Longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality"

low_threshold

integer

(minimum: 1, maximum: 255)

[canny only] Line detection low threshold

Default: 100

high_threshold

integer

(minimum: 1, maximum: 255)

[canny only] Line detection high threshold

Default: 200

guess_mode

boolean

In this mode, the ControlNet encoder will try best to recognize the content of the input image even if you remove all prompts. The `guidance_scale` between 3.0 and 5.0 is recommended.

Default: false

disable_safety_check

boolean

Disable safety check. Use at your own risk!

Default: false

Run this model in Node.js with one line of code:

npx create-replicate --model=anotherjesse/multi-control

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run anotherjesse/multi-control using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "anotherjesse/multi-control:76d8414a702e66c84fe2e6e9c8cbdc12e53f950f255aae9ffa5caa7873b12de0",
  {
    input: {
      eta: 0,
      prompt: "whippet, flemish baroque, el yunque rainforest, 35mm film, quadtone color grading, chromakey\n",
      qr_image: "https://replicate.delivery/pbxt/J0ZYvssNVIBI906LgbXe5kpjkMrugsb5gDklk6erALej1efO/replicate-qr.png",
      scheduler: "K_EULER",
      guess_mode: false,
      num_outputs: 1,
      low_threshold: 100,
      guidance_scale: 9,
      high_threshold: 200,
      negative_prompt: "",
      image_resolution: 512,
      num_inference_steps: 20,
      disable_safety_check: false,
      qr_conditioning_scale: 1.47,
      hed_conditioning_scale: 1,
      seg_conditioning_scale: 1,
      pose_conditioning_scale: 1,
      canny_conditioning_scale: 1,
      depth_conditioning_scale: 1,
      hough_conditioning_scale: 1,
      normal_conditioning_scale: 1,
      scribble_conditioning_scale: 1
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run anotherjesse/multi-control using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "anotherjesse/multi-control:76d8414a702e66c84fe2e6e9c8cbdc12e53f950f255aae9ffa5caa7873b12de0",
    input={
        "eta": 0,
        "prompt": "whippet, flemish baroque, el yunque rainforest, 35mm film, quadtone color grading, chromakey\n",
        "qr_image": "https://replicate.delivery/pbxt/J0ZYvssNVIBI906LgbXe5kpjkMrugsb5gDklk6erALej1efO/replicate-qr.png",
        "scheduler": "K_EULER",
        "guess_mode": False,
        "num_outputs": 1,
        "low_threshold": 100,
        "guidance_scale": 9,
        "high_threshold": 200,
        "negative_prompt": "",
        "image_resolution": 512,
        "num_inference_steps": 20,
        "disable_safety_check": False,
        "qr_conditioning_scale": 1.47,
        "hed_conditioning_scale": 1,
        "seg_conditioning_scale": 1,
        "pose_conditioning_scale": 1,
        "canny_conditioning_scale": 1,
        "depth_conditioning_scale": 1,
        "hough_conditioning_scale": 1,
        "normal_conditioning_scale": 1,
        "scribble_conditioning_scale": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run anotherjesse/multi-control using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "76d8414a702e66c84fe2e6e9c8cbdc12e53f950f255aae9ffa5caa7873b12de0",
    "input": {
      "eta": 0,
      "prompt": "whippet, flemish baroque, el yunque rainforest, 35mm film, quadtone color grading, chromakey\\n",
      "qr_image": "https://replicate.delivery/pbxt/J0ZYvssNVIBI906LgbXe5kpjkMrugsb5gDklk6erALej1efO/replicate-qr.png",
      "scheduler": "K_EULER",
      "guess_mode": false,
      "num_outputs": 1,
      "low_threshold": 100,
      "guidance_scale": 9,
      "high_threshold": 200,
      "negative_prompt": "",
      "image_resolution": 512,
      "num_inference_steps": 20,
      "disable_safety_check": false,
      "qr_conditioning_scale": 1.47,
      "hed_conditioning_scale": 1,
      "seg_conditioning_scale": 1,
      "pose_conditioning_scale": 1,
      "canny_conditioning_scale": 1,
      "depth_conditioning_scale": 1,
      "hough_conditioning_scale": 1,
      "normal_conditioning_scale": 1,
      "scribble_conditioning_scale": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2023-06-16T00:59:07.884533Z",
  "created_at": "2023-06-16T00:59:00.910729Z",
  "data_removed": false,
  "error": null,
  "id": "t5dbu3rbywbcasp3jgsoo24c3a",
  "input": {
    "steps": 50,
    "prompt": "whippet, flemish baroque, el yunque rainforest, 35mm film, quadtone color grading, chromakey\n",
    "qr_image": "https://replicate.delivery/pbxt/J0ZYvssNVIBI906LgbXe5kpjkMrugsb5gDklk6erALej1efO/replicate-qr.png",
    "scheduler": "K_EULER",
    "num_samples": 2,
    "low_threshold": 100,
    "guidance_scale": 9,
    "high_threshold": 200,
    "negative_prompt": "",
    "image_resolution": 512,
    "qr_conditioning_scale": 1.47,
    "hed_conditioning_scale": 1,
    "seg_conditioning_scale": 1,
    "pose_conditioning_scale": 1,
    "canny_conditioning_scale": 1,
    "depth_conditioning_scale": 1,
    "hough_conditioning_scale": 1,
    "normal_conditioning_scale": 1,
    "scribble_conditioning_scale": 1
  },
  "logs": "You have disabled the safety checker for <class 'diffusers.pipelines.controlnet.pipeline_controlnet.StableDiffusionControlNetPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .\nUsing seed: 291\n  0%|          | 0/50 [00:00<?, ?it/s]\n  2%|▏         | 1/50 [00:00<00:05,  9.30it/s]\n  4%|▍         | 2/50 [00:00<00:05,  9.37it/s]\n  8%|▊         | 4/50 [00:00<00:04, 10.22it/s]\n 12%|█▏        | 6/50 [00:00<00:04, 10.57it/s]\n 16%|█▌        | 8/50 [00:00<00:04,  9.93it/s]\n 18%|█▊        | 9/50 [00:00<00:04,  9.81it/s]\n 20%|██        | 10/50 [00:01<00:04,  9.59it/s]\n 24%|██▍       | 12/50 [00:01<00:03, 10.09it/s]\n 26%|██▌       | 13/50 [00:01<00:03,  9.97it/s]\n 28%|██▊       | 14/50 [00:01<00:03,  9.78it/s]\n 30%|███       | 15/50 [00:01<00:03,  9.52it/s]\n 32%|███▏      | 16/50 [00:01<00:03,  9.60it/s]\n 34%|███▍      | 17/50 [00:01<00:03,  9.48it/s]\n 36%|███▌      | 18/50 [00:01<00:03,  9.40it/s]\n 40%|████      | 20/50 [00:02<00:03,  9.75it/s]\n 42%|████▏     | 21/50 [00:02<00:03,  9.56it/s]\n 44%|████▍     | 22/50 [00:02<00:02,  9.60it/s]\n 48%|████▊     | 24/50 [00:02<00:02, 10.22it/s]\n 52%|█████▏    | 26/50 [00:02<00:02,  9.95it/s]\n 54%|█████▍    | 27/50 [00:02<00:02,  9.79it/s]\n 58%|█████▊    | 29/50 [00:02<00:02, 10.21it/s]\n 62%|██████▏   | 31/50 [00:03<00:01, 10.39it/s]\n 66%|██████▌   | 33/50 [00:03<00:01, 10.16it/s]\n 70%|███████   | 35/50 [00:03<00:01, 10.24it/s]\n 74%|███████▍  | 37/50 [00:03<00:01,  9.94it/s]\n 78%|███████▊  | 39/50 [00:03<00:01, 10.15it/s]\n 82%|████████▏ | 41/50 [00:04<00:00, 10.14it/s]\n 86%|████████▌ | 43/50 [00:04<00:00, 10.44it/s]\n 90%|█████████ | 45/50 [00:04<00:00, 10.25it/s]\n 94%|█████████▍| 47/50 [00:04<00:00, 10.08it/s]\n 98%|█████████▊| 49/50 [00:04<00:00,  9.97it/s]\n100%|██████████| 50/50 [00:05<00:00,  9.73it/s]\n100%|██████████| 50/50 [00:05<00:00,  9.95it/s]",
  "metrics": {
    "predict_time": 7.056236,
    "total_time": 6.973804
  },
  "output": [
    "https://replicate.delivery/pbxt/1efdEUS3f7cB0pjAQm64rxzAmfZLUtnKN6f1UCLh5YmR7yyIC/out-0.png",
    "https://replicate.delivery/pbxt/xEGKbZufLuzySSL8lmZ2d77JA461sf1T4K8XZe34UuI3usMiA/out-1.png"
  ],
  "started_at": "2023-06-16T00:59:00.828297Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/t5dbu3rbywbcasp3jgsoo24c3a",
    "cancel": "https://api.replicate.com/v1/predictions/t5dbu3rbywbcasp3jgsoo24c3a/cancel"
  },
  "version": "e785fdfe4b636f62e95835cad6ddd53505687ef4c10571d10fb6b2d0185d46aa"
}

Generated in

7.1 seconds

Tweak itReport

You have disabled the safety checker for <class 'diffusers.pipelines.controlnet.pipeline_controlnet.StableDiffusionControlNetPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
Using seed: 291
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:00<00:05,  9.30it/s]
  4%|▍         | 2/50 [00:00<00:05,  9.37it/s]
  8%|▊         | 4/50 [00:00<00:04, 10.22it/s]
 12%|█▏        | 6/50 [00:00<00:04, 10.57it/s]
 16%|█▌        | 8/50 [00:00<00:04,  9.93it/s]
 18%|█▊        | 9/50 [00:00<00:04,  9.81it/s]
 20%|██        | 10/50 [00:01<00:04,  9.59it/s]
 24%|██▍       | 12/50 [00:01<00:03, 10.09it/s]
 26%|██▌       | 13/50 [00:01<00:03,  9.97it/s]
 28%|██▊       | 14/50 [00:01<00:03,  9.78it/s]
 30%|███       | 15/50 [00:01<00:03,  9.52it/s]
 32%|███▏      | 16/50 [00:01<00:03,  9.60it/s]
 34%|███▍      | 17/50 [00:01<00:03,  9.48it/s]
 36%|███▌      | 18/50 [00:01<00:03,  9.40it/s]
 40%|████      | 20/50 [00:02<00:03,  9.75it/s]
 42%|████▏     | 21/50 [00:02<00:03,  9.56it/s]
 44%|████▍     | 22/50 [00:02<00:02,  9.60it/s]
 48%|████▊     | 24/50 [00:02<00:02, 10.22it/s]
 52%|█████▏    | 26/50 [00:02<00:02,  9.95it/s]
 54%|█████▍    | 27/50 [00:02<00:02,  9.79it/s]
 58%|█████▊    | 29/50 [00:02<00:02, 10.21it/s]
 62%|██████▏   | 31/50 [00:03<00:01, 10.39it/s]
 66%|██████▌   | 33/50 [00:03<00:01, 10.16it/s]
 70%|███████   | 35/50 [00:03<00:01, 10.24it/s]
 74%|███████▍  | 37/50 [00:03<00:01,  9.94it/s]
 78%|███████▊  | 39/50 [00:03<00:01, 10.15it/s]
 82%|████████▏ | 41/50 [00:04<00:00, 10.14it/s]
 86%|████████▌ | 43/50 [00:04<00:00, 10.44it/s]
 90%|█████████ | 45/50 [00:04<00:00, 10.25it/s]
 94%|█████████▍| 47/50 [00:04<00:00, 10.08it/s]
 98%|█████████▊| 49/50 [00:04<00:00,  9.97it/s]
100%|██████████| 50/50 [00:05<00:00,  9.73it/s]
100%|██████████| 50/50 [00:05<00:00,  9.95it/s]

This example was created by a different version, anotherjesse/multi-control:e785fdfe.

Examples

View more examples

Run time and cost

This model costs approximately $0.0066 to run on Replicate, or 151 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 5 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.