vectorspacelab/omnigen:af66691a – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

vectorspacelab /omnigen:af66691a

Input

prompt

string

Shift + Return to add a new line

The flower <img><|image_1|><\/img> is placed in the vase which is in the middle of <img><|image_2|><\/img> on a wooden table of a living roomThe flower <img><|image_1|><\/img> is placed in the vase which is in the middle of <img><|image_2|><\/img> on a wooden table of a living room

Input prompt. For multi-modal to image generation with one or more input images, the placeholder in the prompt should be in the format of <img><|image_*|></img> (for the first image, the placeholder is <|image_1|>, for the second image, the the placeholder is <|image_2|>). Refer to examples for more details

Default: "a photo of an astronaut riding a horse on mars"

img1

file

Preview

Input image 1. Optional

img2

file

Preview

Input image 2. Optional

img3

file

Input image 3. Optional

width

integer

(minimum: 128, maximum: 2048)

Width of the output image

Default: 1024

height

integer

(minimum: 128, maximum: 2048)

Height of the output image

Default: 1024

inference_steps

integer

(minimum: 1, maximum: 100)

Number of denoising steps

Default: 50

guidance_scale

number

(minimum: 1, maximum: 5)

Classifier-free guidance scale for text prompt

Default: 2.5

img_guidance_scale

number

(minimum: 1, maximum: 2)

Classifier-free guidance scale for images

Default: 1.6

seed

integer

Random seed. Leave blank to randomize the seed

max_input_image_size

integer

(minimum: 128, maximum: 2048)

maximum input image size

Default: 1024

separate_cfg_infer

boolean

Whether to use separate inference process for different guidance. This will reduce the memory cost.

Default: true

offload_model

boolean

Offload model to CPU, which will significantly reduce the memory cost but slow down the generation speed. You can cancel separate_cfg_infer and set offload_model=True. If both separate_cfg_infer and offload_model are True, further reduce the memory, but slowest generation

Default: false

use_input_image_size_as_output

boolean

Automatically adjust the output image size to be same as input image size. For editing and controlnet task, it can make sure the output image has the same size as input image leading to better performance

Default: false

Run this model in Node.js with one line of code:

npx create-replicate --model=vectorspacelab/omnigen

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run vectorspacelab/omnigen using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "vectorspacelab/omnigen:af66691a8952a0ce21b26e840835ad1efe176af159e10169ec5df6916338863b",
  {
    input: {
      img1: "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/1f6249e8bb52cfc61be3595778b68f873ffaa04c26e2c107df9fce503892976d/rose.jpg",
      img2: "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/67cbd1b8e20b17b208c0838eb839d1ddeab037fabdf8839fe7698270d1fc9e0b/vase.jpg",
      width: 1024,
      height: 1024,
      prompt: "The flower <img><|image_1|><\\/img> is placed in the vase which is in the middle of <img><|image_2|><\\/img> on a wooden table of a living room",
      offload_model: false,
      guidance_scale: 2.5,
      inference_steps: 50,
      img_guidance_scale: 1.6,
      separate_cfg_infer: true,
      max_input_image_size: 1024,
      use_input_image_size_as_output: false
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run vectorspacelab/omnigen using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "vectorspacelab/omnigen:af66691a8952a0ce21b26e840835ad1efe176af159e10169ec5df6916338863b",
    input={
        "img1": "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/1f6249e8bb52cfc61be3595778b68f873ffaa04c26e2c107df9fce503892976d/rose.jpg",
        "img2": "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/67cbd1b8e20b17b208c0838eb839d1ddeab037fabdf8839fe7698270d1fc9e0b/vase.jpg",
        "width": 1024,
        "height": 1024,
        "prompt": "The flower <img><|image_1|><\\/img> is placed in the vase which is in the middle of <img><|image_2|><\\/img> on a wooden table of a living room",
        "offload_model": False,
        "guidance_scale": 2.5,
        "inference_steps": 50,
        "img_guidance_scale": 1.6,
        "separate_cfg_infer": True,
        "max_input_image_size": 1024,
        "use_input_image_size_as_output": False
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run vectorspacelab/omnigen using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "vectorspacelab/omnigen:af66691a8952a0ce21b26e840835ad1efe176af159e10169ec5df6916338863b",
    "input": {
      "img1": "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/1f6249e8bb52cfc61be3595778b68f873ffaa04c26e2c107df9fce503892976d/rose.jpg",
      "img2": "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/67cbd1b8e20b17b208c0838eb839d1ddeab037fabdf8839fe7698270d1fc9e0b/vase.jpg",
      "width": 1024,
      "height": 1024,
      "prompt": "The flower <img><|image_1|><\\\\/img> is placed in the vase which is in the middle of <img><|image_2|><\\\\/img> on a wooden table of a living room",
      "offload_model": false,
      "guidance_scale": 2.5,
      "inference_steps": 50,
      "img_guidance_scale": 1.6,
      "separate_cfg_infer": true,
      "max_input_image_size": 1024,
      "use_input_image_size_as_output": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-11-03T22:41:35.146269Z",
  "created_at": "2024-11-03T22:37:37.095000Z",
  "data_removed": false,
  "error": null,
  "id": "sgsb0shqrxrgm0cjygpsebv3mw",
  "input": {
    "img1": "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/1f6249e8bb52cfc61be3595778b68f873ffaa04c26e2c107df9fce503892976d/rose.jpg",
    "img2": "https://shitao-omnigen.hf.space/gradio_api/file=/tmp/gradio/67cbd1b8e20b17b208c0838eb839d1ddeab037fabdf8839fe7698270d1fc9e0b/vase.jpg",
    "width": 1024,
    "height": 1024,
    "prompt": "The flower <img><|image_1|><\\/img> is placed in the vase which is in the middle of <img><|image_2|><\\/img> on a wooden table of a living room",
    "offload_model": false,
    "guidance_scale": 2.5,
    "inference_steps": 50,
    "img_guidance_scale": 1.6,
    "separate_cfg_infer": true,
    "max_input_image_size": 1024,
    "use_input_image_size_as_output": false
  },
  "logs": "Using seed: 65037\n  0%|          | 0/50 [00:00<?, ?it/s]\n  2%|▏         | 1/50 [00:05<04:15,  5.22s/it]\n  4%|▍         | 2/50 [00:07<02:53,  3.61s/it]\n  6%|▌         | 3/50 [00:10<02:23,  3.06s/it]\n  8%|▊         | 4/50 [00:12<02:09,  2.81s/it]\n 10%|█         | 5/50 [00:14<01:59,  2.67s/it]\n 12%|█▏        | 6/50 [00:17<01:53,  2.58s/it]\n 14%|█▍        | 7/50 [00:19<01:48,  2.52s/it]\n 16%|█▌        | 8/50 [00:22<01:44,  2.48s/it]\n 18%|█▊        | 9/50 [00:24<01:40,  2.46s/it]\n 20%|██        | 10/50 [00:26<01:37,  2.44s/it]\n 22%|██▏       | 11/50 [00:29<01:34,  2.43s/it]\n 24%|██▍       | 12/50 [00:31<01:32,  2.43s/it]\n 26%|██▌       | 13/50 [00:34<01:29,  2.42s/it]\n 28%|██▊       | 14/50 [00:36<01:27,  2.42s/it]\n 30%|███       | 15/50 [00:39<01:24,  2.42s/it]\n 32%|███▏      | 16/50 [00:41<01:22,  2.42s/it]\n 34%|███▍      | 17/50 [00:43<01:19,  2.42s/it]\n 36%|███▌      | 18/50 [00:46<01:17,  2.42s/it]\n 38%|███▊      | 19/50 [00:48<01:15,  2.42s/it]\n 40%|████      | 20/50 [00:51<01:12,  2.42s/it]\n 42%|████▏     | 21/50 [00:53<01:10,  2.42s/it]\n 44%|████▍     | 22/50 [00:55<01:07,  2.42s/it]\n 46%|████▌     | 23/50 [00:58<01:05,  2.42s/it]\n 48%|████▊     | 24/50 [01:00<01:03,  2.42s/it]\n 50%|█████     | 25/50 [01:03<01:00,  2.42s/it]\n 52%|█████▏    | 26/50 [01:05<00:58,  2.43s/it]\n 54%|█████▍    | 27/50 [01:08<00:55,  2.43s/it]\n 56%|█████▌    | 28/50 [01:10<00:53,  2.43s/it]\n 58%|█████▊    | 29/50 [01:12<00:51,  2.43s/it]\n 60%|██████    | 30/50 [01:15<00:48,  2.43s/it]\n 62%|██████▏   | 31/50 [01:17<00:46,  2.43s/it]\n 64%|██████▍   | 32/50 [01:20<00:43,  2.43s/it]\n 66%|██████▌   | 33/50 [01:22<00:41,  2.43s/it]\n 68%|██████▊   | 34/50 [01:25<00:38,  2.43s/it]\n 70%|███████   | 35/50 [01:27<00:36,  2.43s/it]\n 72%|███████▏  | 36/50 [01:30<00:34,  2.43s/it]\n 74%|███████▍  | 37/50 [01:32<00:31,  2.43s/it]\n 76%|███████▌  | 38/50 [01:34<00:29,  2.43s/it]\n 78%|███████▊  | 39/50 [01:37<00:26,  2.43s/it]\n 80%|████████  | 40/50 [01:39<00:24,  2.43s/it]\n 82%|████████▏ | 41/50 [01:42<00:21,  2.43s/it]\n 84%|████████▍ | 42/50 [01:44<00:19,  2.43s/it]\n 86%|████████▌ | 43/50 [01:47<00:17,  2.43s/it]\n 88%|████████▊ | 44/50 [01:49<00:14,  2.43s/it]\n 90%|█████████ | 45/50 [01:51<00:12,  2.44s/it]\n 92%|█████████▏| 46/50 [01:54<00:09,  2.44s/it]\n 94%|█████████▍| 47/50 [01:56<00:07,  2.44s/it]\n 96%|█████████▌| 48/50 [01:59<00:04,  2.43s/it]\n 98%|█████████▊| 49/50 [02:01<00:02,  2.43s/it]\n100%|██████████| 50/50 [02:04<00:00,  2.43s/it]\n100%|██████████| 50/50 [02:04<00:00,  2.48s/it]",
  "metrics": {
    "predict_time": 128.587871826,
    "total_time": 238.051269
  },
  "output": "https://replicate.delivery/pbxt/5R7G6AbVoXICNxkUeHCWQRCEXmv8yk06Aej9CNFfhEF8w1anA/out.png",
  "started_at": "2024-11-03T22:39:26.558397Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/sgsb0shqrxrgm0cjygpsebv3mw",
    "cancel": "https://api.replicate.com/v1/predictions/sgsb0shqrxrgm0cjygpsebv3mw/cancel"
  },
  "version": "af66691a8952a0ce21b26e840835ad1efe176af159e10169ec5df6916338863b"
}

Generated in

2 minutes 9 seconds

Tweak it ShareReport

Using seed: 65037
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:05<04:15,  5.22s/it]
  4%|▍         | 2/50 [00:07<02:53,  3.61s/it]
  6%|▌         | 3/50 [00:10<02:23,  3.06s/it]
  8%|▊         | 4/50 [00:12<02:09,  2.81s/it]
 10%|█         | 5/50 [00:14<01:59,  2.67s/it]
 12%|█▏        | 6/50 [00:17<01:53,  2.58s/it]
 14%|█▍        | 7/50 [00:19<01:48,  2.52s/it]
 16%|█▌        | 8/50 [00:22<01:44,  2.48s/it]
 18%|█▊        | 9/50 [00:24<01:40,  2.46s/it]
 20%|██        | 10/50 [00:26<01:37,  2.44s/it]
 22%|██▏       | 11/50 [00:29<01:34,  2.43s/it]
 24%|██▍       | 12/50 [00:31<01:32,  2.43s/it]
 26%|██▌       | 13/50 [00:34<01:29,  2.42s/it]
 28%|██▊       | 14/50 [00:36<01:27,  2.42s/it]
 30%|███       | 15/50 [00:39<01:24,  2.42s/it]
 32%|███▏      | 16/50 [00:41<01:22,  2.42s/it]
 34%|███▍      | 17/50 [00:43<01:19,  2.42s/it]
 36%|███▌      | 18/50 [00:46<01:17,  2.42s/it]
 38%|███▊      | 19/50 [00:48<01:15,  2.42s/it]
 40%|████      | 20/50 [00:51<01:12,  2.42s/it]
 42%|████▏     | 21/50 [00:53<01:10,  2.42s/it]
 44%|████▍     | 22/50 [00:55<01:07,  2.42s/it]
 46%|████▌     | 23/50 [00:58<01:05,  2.42s/it]
 48%|████▊     | 24/50 [01:00<01:03,  2.42s/it]
 50%|█████     | 25/50 [01:03<01:00,  2.42s/it]
 52%|█████▏    | 26/50 [01:05<00:58,  2.43s/it]
 54%|█████▍    | 27/50 [01:08<00:55,  2.43s/it]
 56%|█████▌    | 28/50 [01:10<00:53,  2.43s/it]
 58%|█████▊    | 29/50 [01:12<00:51,  2.43s/it]
 60%|██████    | 30/50 [01:15<00:48,  2.43s/it]
 62%|██████▏   | 31/50 [01:17<00:46,  2.43s/it]
 64%|██████▍   | 32/50 [01:20<00:43,  2.43s/it]
 66%|██████▌   | 33/50 [01:22<00:41,  2.43s/it]
 68%|██████▊   | 34/50 [01:25<00:38,  2.43s/it]
 70%|███████   | 35/50 [01:27<00:36,  2.43s/it]
 72%|███████▏  | 36/50 [01:30<00:34,  2.43s/it]
 74%|███████▍  | 37/50 [01:32<00:31,  2.43s/it]
 76%|███████▌  | 38/50 [01:34<00:29,  2.43s/it]
 78%|███████▊  | 39/50 [01:37<00:26,  2.43s/it]
 80%|████████  | 40/50 [01:39<00:24,  2.43s/it]
 82%|████████▏ | 41/50 [01:42<00:21,  2.43s/it]
 84%|████████▍ | 42/50 [01:44<00:19,  2.43s/it]
 86%|████████▌ | 43/50 [01:47<00:17,  2.43s/it]
 88%|████████▊ | 44/50 [01:49<00:14,  2.43s/it]
 90%|█████████ | 45/50 [01:51<00:12,  2.44s/it]
 92%|█████████▏| 46/50 [01:54<00:09,  2.44s/it]
 94%|█████████▍| 47/50 [01:56<00:07,  2.44s/it]
 96%|█████████▌| 48/50 [01:59<00:04,  2.43s/it]
 98%|█████████▊| 49/50 [02:01<00:02,  2.43s/it]
100%|██████████| 50/50 [02:04<00:00,  2.43s/it]
100%|██████████| 50/50 [02:04<00:00,  2.48s/it]