tgohblio/instant-id-multicontrolnet | Run with an API on Replicate

tgohblio / instant-id-multicontrolnet

InstantID. ControlNets. More base SDXL models. And the latest ByteDance's ⚡️SDXL-Lightning !⚡️ (Updated 1 year, 1 month ago)

Cold

Public
296.6K runs
L40S
GitHub
Paper
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

face_image_path

*file

Preview

Image of your face

pose_image_path

file

Reference pose image

prompt

string

Shift + Return to add a new line

woman as elven princess, with blue sheen dress, masterpiecewoman as elven princess, with blue sheen dress, masterpiece

Input prompt

Default: "a person"

negative_prompt

string

Shift + Return to add a new line

Input negative prompt

Default: "(lowres, low quality, worst quality:1.2), (text:1.2), watermark, glitch, deformed, mutated, cross-eyed, ugly, disfigured, blurry, grainy"

model

string

Select SDXL model

Default: "AlbedoBase XL V2"

enable_fast_mode

boolean

Enable SDXL-lightning fast inference. If pose, canny or depth map is used, disable it for better quality images.

Default: true

lightning_steps

string

if enable fast mode, choose number of denoising steps

Default: "4step"

scheduler

string

Scheduler options. If enable fast mode, this is not used.

Default: "DPMSolverMultistepScheduler"

adapter_strength_ratio

number

(minimum: 0, maximum: 1)

Image adapter strength (for detail)

Default: 0.8

identitynet_strength_ratio

number

(minimum: 0, maximum: 1)

IdentityNet strength (for fidelity)

Default: 0.8

pose

boolean

Use pose for skeleton inference

Default: false

pose_strength

number

(minimum: 0, maximum: 1.5)

Default: 1

canny

boolean

Use canny for edge detection

Default: false

canny_strength

number

(minimum: 0, maximum: 1.5)

Default: 0.5

depth_map

boolean

Use depth for depth map estimation

Default: false

depth_strength

number

(minimum: 0, maximum: 1.5)

Default: 0.5

num_steps

integer

(minimum: 1, maximum: 50)

Number of denoising steps. If enable fast mode, this is not used.

Default: 25

guidance_scale

number

(minimum: 0, maximum: 10)

Scale for classifier-free guidance. Optimum is 4-8. If enable fast mode, this is not used.

Default: 7

seed

integer

(minimum: 0, maximum: 2147483647)

Seed number. Set to non-zero to make the image reproducible.

Default: 0

enhance_non_face_region

boolean

Enhance non-face region

Default: true

safety_checker

boolean

Safety checker is enabled by default. Un-tick to expose unfiltered results.

Default: true

Run this model in Node.js with one line of code:

npx create-replicate --model=tgohblio/instant-id-multicontrolnet

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run tgohblio/instant-id-multicontrolnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "tgohblio/instant-id-multicontrolnet:35324a7df2397e6e57dfd8f4f9d2910425f5123109c8c3ed035e769aeff9ff3c",
  {
    input: {
      pose: false,
      seed: 0,
      canny: false,
      model: "AlbedoBase XL V2",
      prompt: "woman as elven princess, with blue sheen dress, masterpiece",
      depth_map: false,
      num_steps: 25,
      scheduler: "DPMSolverMultistepScheduler",
      pose_strength: 0.5,
      canny_strength: 0.5,
      depth_strength: 0.5,
      guidance_scale: 7,
      safety_checker: true,
      face_image_path: "https://replicate.delivery/pbxt/KRsl57SjTUo1WOBw1ir3UVI06jpQ7ybyEtdprpqF2qja40Wn/halle-berry.jpeg",
      lightning_steps: "4step",
      negative_prompt: "ugly, low quality, deformed face, nsfw",
      enable_fast_mode: true,
      adapter_strength_ratio: 0.8,
      enhance_non_face_region: true,
      identitynet_strength_ratio: 0.8
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run tgohblio/instant-id-multicontrolnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "tgohblio/instant-id-multicontrolnet:35324a7df2397e6e57dfd8f4f9d2910425f5123109c8c3ed035e769aeff9ff3c",
    input={
        "pose": False,
        "seed": 0,
        "canny": False,
        "model": "AlbedoBase XL V2",
        "prompt": "woman as elven princess, with blue sheen dress, masterpiece",
        "depth_map": False,
        "num_steps": 25,
        "scheduler": "DPMSolverMultistepScheduler",
        "pose_strength": 0.5,
        "canny_strength": 0.5,
        "depth_strength": 0.5,
        "guidance_scale": 7,
        "safety_checker": True,
        "face_image_path": "https://replicate.delivery/pbxt/KRsl57SjTUo1WOBw1ir3UVI06jpQ7ybyEtdprpqF2qja40Wn/halle-berry.jpeg",
        "lightning_steps": "4step",
        "negative_prompt": "ugly, low quality, deformed face, nsfw",
        "enable_fast_mode": True,
        "adapter_strength_ratio": 0.8,
        "enhance_non_face_region": True,
        "identitynet_strength_ratio": 0.8
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run tgohblio/instant-id-multicontrolnet using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "tgohblio/instant-id-multicontrolnet:35324a7df2397e6e57dfd8f4f9d2910425f5123109c8c3ed035e769aeff9ff3c",
    "input": {
      "pose": false,
      "seed": 0,
      "canny": false,
      "model": "AlbedoBase XL V2",
      "prompt": "woman as elven princess, with blue sheen dress, masterpiece",
      "depth_map": false,
      "num_steps": 25,
      "scheduler": "DPMSolverMultistepScheduler",
      "pose_strength": 0.5,
      "canny_strength": 0.5,
      "depth_strength": 0.5,
      "guidance_scale": 7,
      "safety_checker": true,
      "face_image_path": "https://replicate.delivery/pbxt/KRsl57SjTUo1WOBw1ir3UVI06jpQ7ybyEtdprpqF2qja40Wn/halle-berry.jpeg",
      "lightning_steps": "4step",
      "negative_prompt": "ugly, low quality, deformed face, nsfw",
      "enable_fast_mode": true,
      "adapter_strength_ratio": 0.8,
      "enhance_non_face_region": true,
      "identitynet_strength_ratio": 0.8
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-05-05T10:31:55.025763Z",
  "created_at": "2024-05-05T10:07:27.476000Z",
  "data_removed": false,
  "error": null,
  "id": "q5ypk1g4yhrgg0cf90wskpb96r",
  "input": {
    "pose": false,
    "seed": 0,
    "canny": false,
    "model": "AlbedoBase XL V2",
    "prompt": "woman as elven princess, with blue sheen dress, masterpiece",
    "depth_map": false,
    "num_steps": 25,
    "scheduler": "DPMSolverMultistepScheduler",
    "pose_strength": 0.5,
    "canny_strength": 0.5,
    "depth_strength": 0.5,
    "guidance_scale": 7,
    "safety_checker": true,
    "face_image_path": "https://replicate.delivery/pbxt/KRsl57SjTUo1WOBw1ir3UVI06jpQ7ybyEtdprpqF2qja40Wn/halle-berry.jpeg",
    "lightning_steps": "4step",
    "negative_prompt": "ugly, low quality, deformed face, nsfw",
    "enable_fast_mode": true,
    "adapter_strength_ratio": 0.8,
    "enhance_non_face_region": true,
    "identitynet_strength_ratio": 0.8
  },
  "logs": "/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/insightface/utils/transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.\nTo use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.\nP = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4\n  0%|          | 0/4 [00:00<?, ?it/s]\n 25%|██▌       | 1/4 [00:00<00:02,  1.36it/s]\n 50%|█████     | 2/4 [00:00<00:00,  2.32it/s]\n 75%|███████▌  | 3/4 [00:01<00:00,  2.99it/s]\n100%|██████████| 4/4 [00:01<00:00,  3.45it/s]\n100%|██████████| 4/4 [00:01<00:00,  2.87it/s]",
  "metrics": {
    "predict_time": 9.760985,
    "total_time": 1467.549763
  },
  "output": "https://replicate.delivery/pbxt/ykhrJWQyyEqEDZQsySpruv5fRKgFaDtjT35GgL5JEYDNkoYJA/result.jpg",
  "started_at": "2024-05-05T10:31:45.264778Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/q5ypk1g4yhrgg0cf90wskpb96r",
    "cancel": "https://api.replicate.com/v1/predictions/q5ypk1g4yhrgg0cf90wskpb96r/cancel"
  },
  "version": "35324a7df2397e6e57dfd8f4f9d2910425f5123109c8c3ed035e769aeff9ff3c"
}

Generated in

9.8 seconds

Tweak it Share Report View full prediction

Examples

View more examples

Run time and cost

This model costs approximately $0.023 to run on Replicate, or 43 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 24 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.