zsxkib / ic-light-background

🖼️✨Background images + prompts to auto-magically relights your images (+normal maps🗺️)

Cold

Public
9.9K runs
L40S
GitHub
License

Run with an API

Playground API Examples README Versions

Input

subject_image

*file

Preview

The main foreground image to be relighted

background_image

*file

Preview

The background image that will be used to relight the main foreground image

prompt

*string

Shift + Return to add a new line

Woman, detailed face, sci-fi RGB glowing, cyberpunkWoman, detailed face, sci-fi RGB glowing, cyberpunk

A text description guiding the relighting and generation process

appended_prompt

string

Shift + Return to add a new line

Additional text to be appended to the main prompt, enhancing image quality

Default: "best quality"

negative_prompt

string

Shift + Return to add a new line

lowres, bad anatomy, bad hands, cropped, worst qualitylowres, bad anatomy, bad hands, cropped, worst quality

A text description of attributes to avoid in the generated images

Default: "lowres, bad anatomy, bad hands, cropped, worst quality"

compute_normal

boolean

Whether to compute the normal maps (slower but provides additional output images)

Default: false

width

integer

The width of the generated images in pixels

Default: 512

height

integer

The height of the generated images in pixels

Default: 640

steps

integer

(minimum: 1, maximum: 100)

The number of diffusion steps to perform during generation (more steps generally improves image quality but increases processing time)

Default: 25

cfg

number

(minimum: 1, maximum: 32)

Classifier-Free Guidance scale - higher values encourage adherence to prompt, lower values encourage more creative interpretation

Default: 2

highres_scale

number

(minimum: 1, maximum: 3)

The multiplier for the final output resolution relative to the initial latent resolution

Default: 1.5

highres_denoise

number

(minimum: 0.1, maximum: 1)

Controls the amount of denoising applied when refining the high resolution output (higher = more adherence to the upscaled latent, lower = more creative details added)

Default: 0.5

light_source

string

The type and position of lighting to apply to the initial background latent

Default: "Use Background Image"

seed

integer

A fixed random seed for reproducible results (omit this parameter for a randomized seed)

number_of_images

integer

(minimum: 1, maximum: 12)

The number of unique images to generate from the given input and settings

Default: 1

output_format

string

The image file format of the generated output images

Default: "webp"

output_quality

integer

(minimum: 0, maximum: 100)

The image compression quality (for lossy formats like JPEG and WebP). 100 = best quality, 0 = lowest quality.

Default: 80

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/ic-light-background

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/ic-light-background using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/ic-light-background:60015df78a8a795470da6494822982140d57b150b9ef14354e79302ff89f69e3",
  {
    input: {
      cfg: 2,
      steps: 25,
      width: 512,
      height: 832,
      prompt: "Woman, detailed face, sci-fi RGB glowing, cyberpunk",
      light_source: "Use Background Image",
      highres_scale: 1.5,
      output_format: "webp",
      subject_image: "https://replicate.delivery/pbxt/KtCKrs9sxPF3HciwoWL0TTVM9Nde7ySDWpO9S2flTiyi9Pp3/i3.png",
      compute_normal: false,
      output_quality: 80,
      appended_prompt: "best quality",
      highres_denoise: 0.5,
      negative_prompt: "lowres, bad anatomy, bad hands, cropped, worst quality",
      background_image: "https://replicate.delivery/pbxt/KxPIbJUjSmVBlvn0M3C8PAz6brN5Z0eyZSGcKIVw3XfJ6vNV/7.webp",
      number_of_images: 1
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/ic-light-background using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/ic-light-background:60015df78a8a795470da6494822982140d57b150b9ef14354e79302ff89f69e3",
    input={
        "cfg": 2,
        "steps": 25,
        "width": 512,
        "height": 832,
        "prompt": "Woman, detailed face, sci-fi RGB glowing, cyberpunk",
        "light_source": "Use Background Image",
        "highres_scale": 1.5,
        "output_format": "webp",
        "subject_image": "https://replicate.delivery/pbxt/KtCKrs9sxPF3HciwoWL0TTVM9Nde7ySDWpO9S2flTiyi9Pp3/i3.png",
        "compute_normal": False,
        "output_quality": 80,
        "appended_prompt": "best quality",
        "highres_denoise": 0.5,
        "negative_prompt": "lowres, bad anatomy, bad hands, cropped, worst quality",
        "background_image": "https://replicate.delivery/pbxt/KxPIbJUjSmVBlvn0M3C8PAz6brN5Z0eyZSGcKIVw3XfJ6vNV/7.webp",
        "number_of_images": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/ic-light-background using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "60015df78a8a795470da6494822982140d57b150b9ef14354e79302ff89f69e3",
    "input": {
      "cfg": 2,
      "steps": 25,
      "width": 512,
      "height": 832,
      "prompt": "Woman, detailed face, sci-fi RGB glowing, cyberpunk",
      "light_source": "Use Background Image",
      "highres_scale": 1.5,
      "output_format": "webp",
      "subject_image": "https://replicate.delivery/pbxt/KtCKrs9sxPF3HciwoWL0TTVM9Nde7ySDWpO9S2flTiyi9Pp3/i3.png",
      "compute_normal": false,
      "output_quality": 80,
      "appended_prompt": "best quality",
      "highres_denoise": 0.5,
      "negative_prompt": "lowres, bad anatomy, bad hands, cropped, worst quality",
      "background_image": "https://replicate.delivery/pbxt/KxPIbJUjSmVBlvn0M3C8PAz6brN5Z0eyZSGcKIVw3XfJ6vNV/7.webp",
      "number_of_images": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-05-21T16:00:18.200867Z",
  "created_at": "2024-05-21T15:56:07.892000Z",
  "data_removed": false,
  "error": null,
  "id": "dy04rgfxahrgm0cfkfervg061m",
  "input": {
    "cfg": 2,
    "steps": 25,
    "width": 512,
    "height": 832,
    "prompt": "Woman, detailed face, sci-fi RGB glowing, cyberpunk",
    "light_source": "Use Background Image",
    "highres_scale": 1.5,
    "output_format": "webp",
    "subject_image": "https://replicate.delivery/pbxt/KtCKrs9sxPF3HciwoWL0TTVM9Nde7ySDWpO9S2flTiyi9Pp3/i3.png",
    "compute_normal": false,
    "output_quality": 80,
    "appended_prompt": "best quality",
    "highres_denoise": 0.5,
    "negative_prompt": "lowres, bad anatomy, bad hands, cropped, worst quality",
    "background_image": "https://replicate.delivery/pbxt/KxPIbJUjSmVBlvn0M3C8PAz6brN5Z0eyZSGcKIVw3XfJ6vNV/7.webp",
    "number_of_images": 1
  },
  "logs": "Using seed: 62600\n[!] (<class 'cog.types.Path'>) input_fg=/tmp/tmpcq1zmhq8i3.png\n[!] (<class 'cog.types.Path'>) input_bg=/tmp/tmphw3rq_ql7.webp\n[!] (<class 'str'>) prompt=Woman, detailed face, sci-fi RGB glowing, cyberpunk\n[!] (<class 'int'>) image_width=512\n[!] (<class 'int'>) image_height=832\n[!] (<class 'int'>) num_samples=1\n[!] (<class 'int'>) seed=62600\n[!] (<class 'int'>) steps=25\n[!] (<class 'str'>) a_prompt=best quality\n[!] (<class 'str'>) n_prompt=lowres, bad anatomy, bad hands, cropped, worst quality\n[!] (<class 'float'>) cfg=2.0\n[!] (<class 'float'>) highres_scale=1.5\n[!] (<class 'float'>) highres_denoise=0.5\n[!] (<class 'str'>) bg_source=Use Background Image\n/root/.pyenv/versions/3.10.14/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)\nreturn F.conv2d(input, weight, bias, self.stride,\n  0%|          | 0/25 [00:00<?, ?it/s]\n  4%|▍         | 1/25 [00:00<00:03,  6.67it/s]\n 16%|█▌        | 4/25 [00:00<00:01, 16.07it/s]\n 24%|██▍       | 6/25 [00:00<00:01, 16.74it/s]\n 32%|███▏      | 8/25 [00:00<00:00, 17.09it/s]\n 40%|████      | 10/25 [00:00<00:00, 17.29it/s]\n 48%|████▊     | 12/25 [00:00<00:00, 17.40it/s]\n 56%|█████▌    | 14/25 [00:00<00:00, 17.46it/s]\n 64%|██████▍   | 16/25 [00:00<00:00, 17.52it/s]\n 72%|███████▏  | 18/25 [00:01<00:00, 17.56it/s]\n 80%|████████  | 20/25 [00:01<00:00, 17.55it/s]\n 88%|████████▊ | 22/25 [00:01<00:00, 17.54it/s]\n 96%|█████████▌| 24/25 [00:01<00:00, 17.57it/s]\n100%|██████████| 25/25 [00:01<00:00, 17.04it/s]\n  0%|          | 0/25 [00:00<?, ?it/s]\n  4%|▍         | 1/25 [00:00<00:04,  5.71it/s]\n 12%|█▏        | 3/25 [00:00<00:02,  9.01it/s]\n 16%|█▌        | 4/25 [00:00<00:02,  8.16it/s]\n 20%|██        | 5/25 [00:00<00:02,  7.71it/s]\n 24%|██▍       | 6/25 [00:00<00:02,  7.44it/s]\n 28%|██▊       | 7/25 [00:00<00:02,  7.26it/s]\n 32%|███▏      | 8/25 [00:01<00:02,  7.16it/s]\n 36%|███▌      | 9/25 [00:01<00:02,  7.08it/s]\n 40%|████      | 10/25 [00:01<00:02,  7.03it/s]\n 44%|████▍     | 11/25 [00:01<00:02,  7.00it/s]\n 48%|████▊     | 12/25 [00:01<00:01,  6.98it/s]\n 52%|█████▏    | 13/25 [00:01<00:01,  6.96it/s]\n 56%|█████▌    | 14/25 [00:01<00:01,  6.95it/s]\n 60%|██████    | 15/25 [00:02<00:01,  6.93it/s]\n 64%|██████▍   | 16/25 [00:02<00:01,  6.92it/s]\n 68%|██████▊   | 17/25 [00:02<00:01,  6.92it/s]\n 72%|███████▏  | 18/25 [00:02<00:01,  6.92it/s]\n 76%|███████▌  | 19/25 [00:02<00:00,  6.91it/s]\n 80%|████████  | 20/25 [00:02<00:00,  6.91it/s]\n 84%|████████▍ | 21/25 [00:02<00:00,  6.91it/s]\n 88%|████████▊ | 22/25 [00:03<00:00,  6.91it/s]\n 92%|█████████▏| 23/25 [00:03<00:00,  6.91it/s]\n 96%|█████████▌| 24/25 [00:03<00:00,  6.90it/s]\n100%|██████████| 25/25 [00:03<00:00,  6.90it/s]\n100%|██████████| 25/25 [00:03<00:00,  7.07it/s]\n[~] Saving to output_images/generated_0.webp...\n[~] Output format: WEBP\n[~] Output quality: 80",
  "metrics": {
    "predict_time": 8.91385,
    "total_time": 250.308867
  },
  "output": [
    "https://replicate.delivery/pbxt/0Rvqxkyk7z6uAtrjVIElxQVV5Ke1cdAwIru4h6OU04nIuTbJA/generated_0.webp"
  ],
  "started_at": "2024-05-21T16:00:09.287017Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/dy04rgfxahrgm0cfkfervg061m",
    "cancel": "https://api.replicate.com/v1/predictions/dy04rgfxahrgm0cfkfervg061m/cancel"
  },
  "version": "60015df78a8a795470da6494822982140d57b150b9ef14354e79302ff89f69e3"
}

Generated in

8.9 seconds

Tweak it ShareReport

Using seed: 62600
[!] (<class 'cog.types.Path'>) input_fg=/tmp/tmpcq1zmhq8i3.png
[!] (<class 'cog.types.Path'>) input_bg=/tmp/tmphw3rq_ql7.webp
[!] (<class 'str'>) prompt=Woman, detailed face, sci-fi RGB glowing, cyberpunk
[!] (<class 'int'>) image_width=512
[!] (<class 'int'>) image_height=832
[!] (<class 'int'>) num_samples=1
[!] (<class 'int'>) seed=62600
[!] (<class 'int'>) steps=25
[!] (<class 'str'>) a_prompt=best quality
[!] (<class 'str'>) n_prompt=lowres, bad anatomy, bad hands, cropped, worst quality
[!] (<class 'float'>) cfg=2.0
[!] (<class 'float'>) highres_scale=1.5
[!] (<class 'float'>) highres_denoise=0.5
[!] (<class 'str'>) bg_source=Use Background Image
/root/.pyenv/versions/3.10.14/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)
return F.conv2d(input, weight, bias, self.stride,
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:00<00:03,  6.67it/s]
 16%|█▌        | 4/25 [00:00<00:01, 16.07it/s]
 24%|██▍       | 6/25 [00:00<00:01, 16.74it/s]
 32%|███▏      | 8/25 [00:00<00:00, 17.09it/s]
 40%|████      | 10/25 [00:00<00:00, 17.29it/s]
 48%|████▊     | 12/25 [00:00<00:00, 17.40it/s]
 56%|█████▌    | 14/25 [00:00<00:00, 17.46it/s]
 64%|██████▍   | 16/25 [00:00<00:00, 17.52it/s]
 72%|███████▏  | 18/25 [00:01<00:00, 17.56it/s]
 80%|████████  | 20/25 [00:01<00:00, 17.55it/s]
 88%|████████▊ | 22/25 [00:01<00:00, 17.54it/s]
 96%|█████████▌| 24/25 [00:01<00:00, 17.57it/s]
100%|██████████| 25/25 [00:01<00:00, 17.04it/s]
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:00<00:04,  5.71it/s]
 12%|█▏        | 3/25 [00:00<00:02,  9.01it/s]
 16%|█▌        | 4/25 [00:00<00:02,  8.16it/s]
 20%|██        | 5/25 [00:00<00:02,  7.71it/s]
 24%|██▍       | 6/25 [00:00<00:02,  7.44it/s]
 28%|██▊       | 7/25 [00:00<00:02,  7.26it/s]
 32%|███▏      | 8/25 [00:01<00:02,  7.16it/s]
 36%|███▌      | 9/25 [00:01<00:02,  7.08it/s]
 40%|████      | 10/25 [00:01<00:02,  7.03it/s]
 44%|████▍     | 11/25 [00:01<00:02,  7.00it/s]
 48%|████▊     | 12/25 [00:01<00:01,  6.98it/s]
 52%|█████▏    | 13/25 [00:01<00:01,  6.96it/s]
 56%|█████▌    | 14/25 [00:01<00:01,  6.95it/s]
 60%|██████    | 15/25 [00:02<00:01,  6.93it/s]
 64%|██████▍   | 16/25 [00:02<00:01,  6.92it/s]
 68%|██████▊   | 17/25 [00:02<00:01,  6.92it/s]
 72%|███████▏  | 18/25 [00:02<00:01,  6.92it/s]
 76%|███████▌  | 19/25 [00:02<00:00,  6.91it/s]
 80%|████████  | 20/25 [00:02<00:00,  6.91it/s]
 84%|████████▍ | 21/25 [00:02<00:00,  6.91it/s]
 88%|████████▊ | 22/25 [00:03<00:00,  6.91it/s]
 92%|█████████▏| 23/25 [00:03<00:00,  6.91it/s]
 96%|█████████▌| 24/25 [00:03<00:00,  6.90it/s]
100%|██████████| 25/25 [00:03<00:00,  6.90it/s]
100%|██████████| 25/25 [00:03<00:00,  7.07it/s]
[~] Saving to output_images/generated_0.webp...
[~] Output format: WEBP
[~] Output quality: 80

Examples

View more examples

Run time and cost

This model costs approximately $0.025 to run on Replicate, or 40 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 26 seconds. The predict time for this model varies significantly based on the inputs.

Readme

IC-Light Background: Illuminate Portraits with Stunning Backgrounds 🌄✨

Foreground (Subject) Image + Background Image + Prompt → Subject in Background w/ Correct Lighting

Overview 🖼️

IC-Light Background is a powerful AI-driven tool that takes portrait relighting to the next level by incorporating background images. As a companion to the IC-Light Text model, IC-Light Background allows you to relight your portraits using reference background images, ensuring a cohesive and visually stunning result. 🎨💡

Getting Started 🚀

To begin relighting your portraits with IC-Light Background:

Upload your portrait image
Provide a reference background image with the desired lighting
Watch as IC-Light Background seamlessly applies the background lighting to your portrait! ✨

Background-Conditioned Relighting 📸🌠

IC-Light Background harnesses the power of background images to guide the relighting process. By analyzing the lighting in your reference background, the model can intelligently apply the same illumination to your portrait, creating a harmonious and visually striking result. 🌈

Normal Map Estimation 🗺️

One of the standout features of IC-Light Background is its ability to estimate normal maps from the relit portraits. By relighting the image from four different directions (left, right, top, bottom), the model can compute a detailed normal map that captures the 3D-like shading of your portrait. This adds an extra layer of depth and realism to your relit images. 🌍

Seamless Integration 🧩

IC-Light Background is designed to work hand-in-hand with the IC-Light Text model. While the text-conditioned model focuses on relighting portraits based on textual descriptions, IC-Light Background takes it a step further by incorporating reference background images. Together, these models provide a comprehensive and versatile portrait relighting solution. 🤝

Customization Options 🎚️

Just like its text-conditioned counterpart, IC-Light Background offers a wide range of customization options to fine-tune your relighting results. From adjusting the relighting strength to controlling the output resolution, you have the flexibility to create the perfect look for your portraits. 🔧

Cutting-Edge Technology 🧠

IC-Light Background is powered by state-of-the-art deep learning models, including Stable Diffusion v1.5 and the BriaRMBG model for foreground extraction. By leveraging advanced techniques such as variational autoencoders, U-Nets, and custom loss functions, IC-Light Background achieves unprecedented levels of relighting quality and consistency. 🚀

Elevate Your Portraits 📈

With IC-Light Background, you can take your portrait photography to new heights. By harnessing the power of stunning background lighting, you can create portraits that truly stand out and captivate your audience. Get ready to unleash your creativity and explore a world of breathtaking portrait relighting possibilities! 🌟

Responsible Use 🙏

As with any powerful tool, please use IC-Light Background responsibly. Respect the privacy and rights of the individuals in your photos, and always obtain necessary permissions before sharing or publishing relit images. 📜

Discover the magic of background-conditioned portrait relighting with IC-Light Background! 🎨✨

For text-conditioned portrait relighting, check out the IC-Light Text model. 📝💡