fofr/kolors-with-ipadapter – Run with an API on Replicate

fofr / kolors-with-ipadapter

Kolors with style transfer, composition transfer and other IPAdapter techniques

Cold

Public
17.7K runs
L40S
GitHub
Paper
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

an illustration of a cute dog jumping over a sleeping catan illustration of a cute dog jumping over a sleeping cat

Default: ""

negative_prompt

string

Shift + Return to add a new line

Things you do not want to see in your image

Default: ""

image

file

Image to use as a reference for the IPAdapter

number_of_images

integer

(minimum: 1, maximum: 10)

Number of images to generate

Default: 1

width

integer

(minimum: 512, maximum: 2048)

Width of the image

Default: 1024

height

integer

(minimum: 512, maximum: 2048)

Height of the image

Default: 1024

steps

integer

(minimum: 1, maximum: 50)

Number of inference steps

Default: 25

cfg

number

(minimum: 0, maximum: 20)

Guidance scale

Default: 4

sampler

string

Sampler

Default: "dpmpp_2m_sde_gpu"

scheduler

string

Scheduler

Default: "karras"

ip_adapter_weight_type

string

Weight type for the IPAdapter

Default: "style transfer precise"

ip_adapter_weight

number

(minimum: 0, maximum: 1)

Strength of the IPAdapter

Default: 1

output_format

string

Format of the output images

Default: "webp"

output_quality

integer

(minimum: 0, maximum: 100)

Quality of the output images, from 0 to 100. 100 is best quality, 0 is lowest quality.

Default: 80

seed

integer

Set a seed for reproducibility. Random by default.

Run this model in Node.js with one line of code:

npx create-replicate --model=fofr/kolors-with-ipadapter

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/kolors-with-ipadapter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/kolors-with-ipadapter:5a1a92b2c0f81813225d48ed8e411813da41aa84e7582fb705d1af46eea36eed",
  {
    input: {
      cfg: 4,
      image: "https://replicate.delivery/pbxt/LI5VAhU2v3jNTjuE76GMTzikT1XMiUoRSznZdXR0cAnK1XJS/ComfyUI_00362_.png",
      steps: 25,
      width: 1024,
      height: 1024,
      prompt: "an illustration of a cute dog jumping over a sleeping cat",
      sampler: "dpmpp_2m_sde_gpu",
      scheduler: "karras",
      output_format: "webp",
      output_quality: 80,
      negative_prompt: "",
      number_of_images: 1,
      ip_adapter_weight: 1,
      ip_adapter_weight_type: "style transfer precise"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run fofr/kolors-with-ipadapter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/kolors-with-ipadapter:5a1a92b2c0f81813225d48ed8e411813da41aa84e7582fb705d1af46eea36eed",
    input={
        "cfg": 4,
        "image": "https://replicate.delivery/pbxt/LI5VAhU2v3jNTjuE76GMTzikT1XMiUoRSznZdXR0cAnK1XJS/ComfyUI_00362_.png",
        "steps": 25,
        "width": 1024,
        "height": 1024,
        "prompt": "an illustration of a cute dog jumping over a sleeping cat",
        "sampler": "dpmpp_2m_sde_gpu",
        "scheduler": "karras",
        "output_format": "webp",
        "output_quality": 80,
        "negative_prompt": "",
        "number_of_images": 1,
        "ip_adapter_weight": 1,
        "ip_adapter_weight_type": "style transfer precise"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run fofr/kolors-with-ipadapter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/kolors-with-ipadapter:5a1a92b2c0f81813225d48ed8e411813da41aa84e7582fb705d1af46eea36eed",
    "input": {
      "cfg": 4,
      "image": "https://replicate.delivery/pbxt/LI5VAhU2v3jNTjuE76GMTzikT1XMiUoRSznZdXR0cAnK1XJS/ComfyUI_00362_.png",
      "steps": 25,
      "width": 1024,
      "height": 1024,
      "prompt": "an illustration of a cute dog jumping over a sleeping cat",
      "sampler": "dpmpp_2m_sde_gpu",
      "scheduler": "karras",
      "output_format": "webp",
      "output_quality": 80,
      "negative_prompt": "",
      "number_of_images": 1,
      "ip_adapter_weight": 1,
      "ip_adapter_weight_type": "style transfer precise"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-07-18T17:18:37.129145Z",
  "created_at": "2024-07-18T17:18:29.145000Z",
  "data_removed": false,
  "error": null,
  "id": "3eqtsdhv35rgm0cgrvd8yr1y7c",
  "input": {
    "cfg": 4,
    "image": "https://replicate.delivery/pbxt/LI5VAhU2v3jNTjuE76GMTzikT1XMiUoRSznZdXR0cAnK1XJS/ComfyUI_00362_.png",
    "steps": 25,
    "width": 1024,
    "height": 1024,
    "prompt": "an illustration of a cute dog jumping over a sleeping cat",
    "sampler": "dpmpp_2m_sde_gpu",
    "scheduler": "karras",
    "output_format": "webp",
    "output_quality": 80,
    "negative_prompt": "",
    "number_of_images": 1,
    "ip_adapter_weight": 1,
    "ip_adapter_weight_type": "style transfer precise"
  },
  "logs": "Random seed set to: 1134600525\nRunning workflow\ngot prompt\nExecuting node 67, title: MinusZone - ChatGLM3TextEncode, class type: MZ_ChatGLM3\nExecuting node 79, title: KSampler, class type: KSampler\nRequested to load SDXL\nLoading 1 new model\n  0%|          | 0/25 [00:00<?, ?it/s]\n  4%|▍         | 1/25 [00:00<00:04,  4.94it/s]\n  8%|▊         | 2/25 [00:00<00:04,  4.90it/s]\n 12%|█▏        | 3/25 [00:00<00:04,  4.89it/s]\n 16%|█▌        | 4/25 [00:00<00:04,  4.89it/s]\n 20%|██        | 5/25 [00:01<00:04,  4.90it/s]\n 24%|██▍       | 6/25 [00:01<00:03,  4.89it/s]\n 28%|██▊       | 7/25 [00:01<00:03,  4.89it/s]\n 32%|███▏      | 8/25 [00:01<00:03,  4.89it/s]\n 36%|███▌      | 9/25 [00:01<00:03,  4.89it/s]\n 40%|████      | 10/25 [00:02<00:03,  4.86it/s]\n 44%|████▍     | 11/25 [00:02<00:02,  4.86it/s]\n 48%|████▊     | 12/25 [00:02<00:02,  4.87it/s]\n 52%|█████▏    | 13/25 [00:02<00:02,  4.88it/s]\n 56%|█████▌    | 14/25 [00:02<00:02,  4.88it/s]\n 60%|██████    | 15/25 [00:03<00:02,  4.89it/s]\n 64%|██████▍   | 16/25 [00:03<00:01,  4.89it/s]\n 68%|██████▊   | 17/25 [00:03<00:01,  4.90it/s]\n 72%|███████▏  | 18/25 [00:03<00:01,  4.90it/s]\n 76%|███████▌  | 19/25 [00:03<00:01,  4.90it/s]\n 80%|████████  | 20/25 [00:04<00:01,  4.90it/s]\n 84%|████████▍ | 21/25 [00:04<00:00,  4.89it/s]\n 88%|████████▊ | 22/25 [00:04<00:00,  4.90it/s]\n 92%|█████████▏| 23/25 [00:04<00:00,  4.90it/s]\n 96%|█████████▌| 24/25 [00:04<00:00,  4.92it/s]\n100%|██████████| 25/25 [00:05<00:00,  4.94it/s]\n100%|██████████| 25/25 [00:05<00:00,  4.90it/s]\nRequested to load AutoencoderKL\nLoading 1 new model\nExecuting node 80, title: VAE Decode, class type: VAEDecode\nExecuting node 89, title: Save Image, class type: SaveImage\nPrompt executed in 5.91 seconds\noutputs:  {'89': {'images': [{'filename': 'ComfyUI_00001_.png', 'subfolder': '', 'type': 'output'}]}}\n====================================\nComfyUI_00001_.png",
  "metrics": {
    "predict_time": 7.888414131,
    "total_time": 7.984145
  },
  "output": [
    "https://replicate.delivery/pbxt/6afB4UMVrtXOCCUwnEN3pGvVx0OqPTr4A1656w6SSUL2A4kJA/ComfyUI_00001_.webp"
  ],
  "started_at": "2024-07-18T17:18:29.240731Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/3eqtsdhv35rgm0cgrvd8yr1y7c",
    "cancel": "https://api.replicate.com/v1/predictions/3eqtsdhv35rgm0cgrvd8yr1y7c/cancel"
  },
  "version": "5a1a92b2c0f81813225d48ed8e411813da41aa84e7582fb705d1af46eea36eed"
}

Generated in

7.9 seconds

Tweak it Share Report

Random seed set to: 1134600525
Running workflow
got prompt
Executing node 67, title: MinusZone - ChatGLM3TextEncode, class type: MZ_ChatGLM3
Executing node 79, title: KSampler, class type: KSampler
Requested to load SDXL
Loading 1 new model
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:00<00:04,  4.94it/s]
  8%|▊         | 2/25 [00:00<00:04,  4.90it/s]
 12%|█▏        | 3/25 [00:00<00:04,  4.89it/s]
 16%|█▌        | 4/25 [00:00<00:04,  4.89it/s]
 20%|██        | 5/25 [00:01<00:04,  4.90it/s]
 24%|██▍       | 6/25 [00:01<00:03,  4.89it/s]
 28%|██▊       | 7/25 [00:01<00:03,  4.89it/s]
 32%|███▏      | 8/25 [00:01<00:03,  4.89it/s]
 36%|███▌      | 9/25 [00:01<00:03,  4.89it/s]
 40%|████      | 10/25 [00:02<00:03,  4.86it/s]
 44%|████▍     | 11/25 [00:02<00:02,  4.86it/s]
 48%|████▊     | 12/25 [00:02<00:02,  4.87it/s]
 52%|█████▏    | 13/25 [00:02<00:02,  4.88it/s]
 56%|█████▌    | 14/25 [00:02<00:02,  4.88it/s]
 60%|██████    | 15/25 [00:03<00:02,  4.89it/s]
 64%|██████▍   | 16/25 [00:03<00:01,  4.89it/s]
 68%|██████▊   | 17/25 [00:03<00:01,  4.90it/s]
 72%|███████▏  | 18/25 [00:03<00:01,  4.90it/s]
 76%|███████▌  | 19/25 [00:03<00:01,  4.90it/s]
 80%|████████  | 20/25 [00:04<00:01,  4.90it/s]
 84%|████████▍ | 21/25 [00:04<00:00,  4.89it/s]
 88%|████████▊ | 22/25 [00:04<00:00,  4.90it/s]
 92%|█████████▏| 23/25 [00:04<00:00,  4.90it/s]
 96%|█████████▌| 24/25 [00:04<00:00,  4.92it/s]
100%|██████████| 25/25 [00:05<00:00,  4.94it/s]
100%|██████████| 25/25 [00:05<00:00,  4.90it/s]
Requested to load AutoencoderKL
Loading 1 new model
Executing node 80, title: VAE Decode, class type: VAEDecode
Executing node 89, title: Save Image, class type: SaveImage
Prompt executed in 5.91 seconds
outputs:  {'89': {'images': [{'filename': 'ComfyUI_00001_.png', 'subfolder': '', 'type': 'output'}]}}
====================================
ComfyUI_00001_.png

Examples

View more examples

Run time and cost

This model costs approximately $0.0099 to run on Replicate, or 101 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 11 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Kolors

Kolors is a large-scale text-to-image generation model based on latent diffusion, developed by the Kuaishou Kolors team. Trained on billions of text-image pairs, Kolors exhibits significant advantages over both open-source and proprietary models in visual quality, complex semantic accuracy, and text rendering for both Chinese and English characters. Furthermore, Kolors supports both Chinese and English inputs, demonstrating strong performance in understanding and generating Chinese-specific content. For more details, please refer to this technical report.

Paper: https://github.com/Kwai-Kolors/Kolors/blob/master/imgs/Kolors_paper.pdf
Weights: https://huggingface.co/Kwai-Kolors/Kolors
Weights license: https://huggingface.co/Kwai-Kolors/Kolors/blob/main/MODEL_LICENSE

Implementation

Using:

License

Kolors are fully open-sourced for academic research. For commercial use, please fill out this questionnaire and send it to kwai-kolors@kuaishou.com for registration.