okaris/omni-zero – Run with an API on Replicate

okaris / omni-zero

Omni-Zero: A diffusion pipeline for zero-shot stylized portrait creation.

Cold

Public
220.5K runs
L40S
GitHub
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

seed

integer

Random seed for the model

Default: 42

model

string

Model to use for the prediction

Default: "omni-zero"

prompt

string

Shift + Return to add a new line

Prompt for the model

Default: "A person"

negative_prompt

string

Shift + Return to add a new line

Negative prompt for the model

Default: "blurry, out of focus"

guidance_scale

number

(minimum: 0, maximum: 14)

Guidance scale for the model

Default: 3

number_of_images

integer

(minimum: 1, maximum: 4)

Number of images to generate

Default: 1

number_of_steps

integer

(minimum: 1, maximum: 50)

Number of steps for the model

Default: 10

image

file

Base image for the model

image_strength

number

(minimum: 0, maximum: 1)

Base image strength for the model

Default: 0.15

composition_image

file

Composition image for the model

composition_strength

number

(minimum: 0, maximum: 1)

Composition image strength for the model

Default: 1

style_image

*file

Style image for the model

style_strength

number

(minimum: 0, maximum: 1)

Style image strength for the model

Default: 1

identity_image

*file

Identity image for the model

identity_strength

number

(minimum: 0, maximum: 1)

Identity image strength for the model

Default: 1

depth_image

file

Depth image for the model

depth_strength

number

(minimum: 0, maximum: 1)

Depth image strength for the model, if not supplied the composition image will be used for depth

Default: 0.5

Run this model in Node.js with one line of code:

npx create-replicate --model=okaris/omni-zero

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run okaris/omni-zero using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "okaris/omni-zero:036947f1e1961875eef47a561293978528bf3a847e79fedb20600c9ad25d0c59",
  {
    input: {
      seed: 42,
      model: "omni-zero",
      prompt: "A person",
      style_image: "https://github.com/okaris/omni-zero/assets/1448702/64dc150b-f683-41b1-be23-b6a52c771584",
      depth_strength: 0.5,
      guidance_scale: 3,
      identity_image: "https://github.com/okaris/omni-zero/assets/1448702/ba193a3a-f90e-4461-848a-560454531c58",
      image_strength: 0.15,
      style_strength: 1,
      negative_prompt: "blurry, out of focus",
      number_of_steps: 10,
      number_of_images: 1,
      composition_image: "https://github.com/okaris/omni-zero/assets/1448702/2ca63443-c7f3-4ba6-95c1-2a341414865f",
      identity_strength: 1,
      composition_strength: 1
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run okaris/omni-zero using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "okaris/omni-zero:036947f1e1961875eef47a561293978528bf3a847e79fedb20600c9ad25d0c59",
    input={
        "seed": 42,
        "model": "omni-zero",
        "prompt": "A person",
        "style_image": "https://github.com/okaris/omni-zero/assets/1448702/64dc150b-f683-41b1-be23-b6a52c771584",
        "depth_strength": 0.5,
        "guidance_scale": 3,
        "identity_image": "https://github.com/okaris/omni-zero/assets/1448702/ba193a3a-f90e-4461-848a-560454531c58",
        "image_strength": 0.15,
        "style_strength": 1,
        "negative_prompt": "blurry, out of focus",
        "number_of_steps": 10,
        "number_of_images": 1,
        "composition_image": "https://github.com/okaris/omni-zero/assets/1448702/2ca63443-c7f3-4ba6-95c1-2a341414865f",
        "identity_strength": 1,
        "composition_strength": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run okaris/omni-zero using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "okaris/omni-zero:036947f1e1961875eef47a561293978528bf3a847e79fedb20600c9ad25d0c59",
    "input": {
      "seed": 42,
      "model": "omni-zero",
      "prompt": "A person",
      "style_image": "https://github.com/okaris/omni-zero/assets/1448702/64dc150b-f683-41b1-be23-b6a52c771584",
      "depth_strength": 0.5,
      "guidance_scale": 3,
      "identity_image": "https://github.com/okaris/omni-zero/assets/1448702/ba193a3a-f90e-4461-848a-560454531c58",
      "image_strength": 0.15,
      "style_strength": 1,
      "negative_prompt": "blurry, out of focus",
      "number_of_steps": 10,
      "number_of_images": 1,
      "composition_image": "https://github.com/okaris/omni-zero/assets/1448702/2ca63443-c7f3-4ba6-95c1-2a341414865f",
      "identity_strength": 1,
      "composition_strength": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-06-17T12:07:52.013887Z",
  "created_at": "2024-06-17T12:06:40.074000Z",
  "data_removed": false,
  "error": null,
  "id": "p28rjd4ws9rgm0cg4rbr23fhdm",
  "input": {
    "seed": 42,
    "prompt": "A person",
    "base_image": "https://github.com/okaris/omni-zero/assets/1448702/2ca63443-c7f3-4ba6-95c1-2a341414865f",
    "style_image": "https://github.com/okaris/omni-zero/assets/1448702/64dc150b-f683-41b1-be23-b6a52c771584",
    "guidance_scale": 3,
    "identity_image": "https://github.com/okaris/omni-zero/assets/1448702/ba193a3a-f90e-4461-848a-560454531c58",
    "negative_prompt": "blurry, out of focus",
    "number_of_steps": 10,
    "number_of_images": 1,
    "composition_image": "https://github.com/okaris/omni-zero/assets/1448702/2ca63443-c7f3-4ba6-95c1-2a341414865f",
    "base_image_strength": 0.15,
    "depth_image_strength": 0.5,
    "style_image_strength": 1,
    "identity_image_strength": 1,
    "composition_image_strength": 1
  },
  "logs": "/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/insightface/utils/transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.\nTo use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.\nP = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4\n  0%|          | 0/10 [00:00<?, ?it/s]/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:608: UserWarning: Should have tb<=t1 but got tb=6.818950160602679 and t1=6.81895.\nwarnings.warn(f\"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.\")\n 10%|█         | 1/10 [00:00<00:02,  3.90it/s]\n 20%|██        | 2/10 [00:00<00:01,  4.58it/s]\n 30%|███       | 3/10 [00:00<00:01,  4.52it/s]\n 40%|████      | 4/10 [00:00<00:01,  4.55it/s]\n 50%|█████     | 5/10 [00:01<00:01,  4.53it/s]\n 60%|██████    | 6/10 [00:01<00:00,  4.50it/s]\n 70%|███████   | 7/10 [00:01<00:00,  4.54it/s]\n 80%|████████  | 8/10 [00:01<00:00,  4.54it/s]/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:599: UserWarning: Should have ta>=t0 but got ta=0.09863785005758784 and t0=0.098638.\nwarnings.warn(f\"Should have ta>=t0 but got ta={ta} and t0={self._start}.\")\n 90%|█████████ | 9/10 [00:01<00:00,  4.53it/s]/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:599: UserWarning: Should have ta>=t0 but got ta=0.0 and t0=0.098638.\nwarnings.warn(f\"Should have ta>=t0 but got ta={ta} and t0={self._start}.\")\n/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:602: UserWarning: Should have tb>=t0 but got tb=0.09863785005758784 and t0=0.098638.\nwarnings.warn(f\"Should have {tb_name}>=t0 but got {tb_name}={tb} and t0={self._start}.\")\n100%|██████████| 10/10 [00:02<00:00,  4.54it/s]\n100%|██████████| 10/10 [00:02<00:00,  4.51it/s]",
  "metrics": {
    "predict_time": 7.063468107,
    "total_time": 71.939887
  },
  "output": [
    "https://replicate.delivery/pbxt/MEDNlRywhva7O9sO5P439PTbhZ5qHJZfd7eqVPCITJlXkdflA/oz_output_0.jpg"
  ],
  "started_at": "2024-06-17T12:07:44.950419Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/p28rjd4ws9rgm0cg4rbr23fhdm",
    "cancel": "https://api.replicate.com/v1/predictions/p28rjd4ws9rgm0cg4rbr23fhdm/cancel"
  },
  "version": "63ae60eb91f6d09acc0ec5bb9892b330e63150cc50ee87d48e93ad73d25291f6"
}

Generated in

7.1 seconds

Tweak it Share Report

/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/insightface/utils/transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
  0%|          | 0/10 [00:00<?, ?it/s]/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:608: UserWarning: Should have tb<=t1 but got tb=6.818950160602679 and t1=6.81895.
warnings.warn(f"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.")
 10%|█         | 1/10 [00:00<00:02,  3.90it/s]
 20%|██        | 2/10 [00:00<00:01,  4.58it/s]
 30%|███       | 3/10 [00:00<00:01,  4.52it/s]
 40%|████      | 4/10 [00:00<00:01,  4.55it/s]
 50%|█████     | 5/10 [00:01<00:01,  4.53it/s]
 60%|██████    | 6/10 [00:01<00:00,  4.50it/s]
 70%|███████   | 7/10 [00:01<00:00,  4.54it/s]
 80%|████████  | 8/10 [00:01<00:00,  4.54it/s]/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:599: UserWarning: Should have ta>=t0 but got ta=0.09863785005758784 and t0=0.098638.
warnings.warn(f"Should have ta>=t0 but got ta={ta} and t0={self._start}.")
 90%|█████████ | 9/10 [00:01<00:00,  4.53it/s]/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:599: UserWarning: Should have ta>=t0 but got ta=0.0 and t0=0.098638.
warnings.warn(f"Should have ta>=t0 but got ta={ta} and t0={self._start}.")
/root/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torchsde/_brownian/brownian_interval.py:602: UserWarning: Should have tb>=t0 but got tb=0.09863785005758784 and t0=0.098638.
warnings.warn(f"Should have {tb_name}>=t0 but got {tb_name}={tb} and t0={self._start}.")
100%|██████████| 10/10 [00:02<00:00,  4.54it/s]
100%|██████████| 10/10 [00:02<00:00,  4.51it/s]

This output was created using a different version of the model, okaris/omni-zero:63ae60eb.

Examples

View more examples

Run time and cost

This model costs approximately $0.021 to run on Replicate, or 47 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 22 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Announcement: Omni-Zero Couples is now live

https://github.com/okaris/omni-zero-couples

Use Omni-Zero in HuggingFace Spaces ZeroGPU https://huggingface.co/spaces/okaris/omni-zero

Omni-Zero

Use Omni-Zero in fal.ai Workflows https://fal.ai/dashboard/workflows/okaris/omni-zero

Omni-Zero

Try our free demo on StyleOf https://styleof.com/s/remix-yourself

Omni-Zero

Single Identity and Style

Omni-Zero

Multiple Identities and Styles (WIP)

Frame 7-3

How to run

git clone --recursive https://github.com/okaris/omni-zero.git
cd omni-zero
pip install -r requirements.txt
python demo.py

Credits

Special thanks to fal.ai for providing compute for the research and hosting
This project wouldn’t be possible without the great work of the InstantX Team
Thanks to @fofrAI for inspiring me with his face-to-many workflow
Thanks to Matteo (@cubiq) for creating the ComfyUI nodes for IP-Adapter