alexgenovese / ominicontrol-lora

WIP | Minimal and Universal Control for Diffusion Transformer

  • Public
  • 31 runs
  • L40S
  • GitHub
  • Weights
  • Paper
  • License
Iterate in playground

Input

Set the REPLICATE_API_TOKEN environment variable:
export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run alexgenovese/ominicontrol-lora using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "alexgenovese/ominicontrol-lora:e5722dc6ff1a32a4f3b47ec4906bb1c5f06ebedab08dfd04d1dbd268d304b34d",
    "input": {
      "lora_scale": 0.8,
      "num_outputs": 1,
      "aspect_ratio": "1:1",
      "output_format": "webp",
      "guidance_scale": 3.5,
      "output_quality": 80,
      "num_inference_steps": 28
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.

Run time and cost

This model runs on Nvidia L40S GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

About

This endpoint is a Cog implementation of OminiControl with Flux Dev 1.

⚠ This version is not working due to a huge model folder of 56GB (flux + omini).

The working copy is on Github; please follow the Github URL below the title

OminiControl is a minimal yet powerful universal control framework for Diffusion Transformer models like FLUX.

  • Universal Control 🌐: A unified control framework that supports both subject-driven control and spatial control (such as edge-guided and in-painting generation).
  • Minimal Design 🚀: Injects control signals while preserving original model structure. Only introduces 0.1% additional parameters to the base model.

Cover Image

Contact me if you need customizations.

Follow me on Twitter/X | Website

Licensing and commercial use

You can use the images commercially if you generate images on Replicate with FLUX.1 models and their fine-tunes.

If you download the weights off Replicate and generate images on your computer, you can’t use the images commercially.

Credits

OminiControl: Minimal and Universal Control for Diffusion Transformer Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang Learning and Vision Lab, National University of Singapore