omerbt / multidiffusion

Fusing Diffusion Paths for Controlled Image Generation

Cold

Public
2.2K runs
A100 (80GB)
GitHub
Paper

Run with an API

Playground API Examples README Versions

Prediction

omerbt/multidiffusion:fd8e12cb

Model

omerbt/multidiffusion:963ffeaa

iwwaat27dnam3jag2kq4ofao6i

Status

Succeeded

Source

Web

Hardware

–

Total duration

5m 18s

Created

about 2 years ago

Input

width: 4096
height: 512
prompt: a photo of the dolomites
scheduler: K_EULER
num_outputs: 1
guidance_scale: 7.5
num_inference_steps: 20

{
  "width": 4096,
  "height": 512,
  "prompt": "a photo of the dolomites",
  "scheduler": "K_EULER",
  "num_outputs": 1,
  "guidance_scale": 7.5,
  "num_inference_steps": 20
}

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run omerbt/multidiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "omerbt/multidiffusion:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387",
  {
    input: {
      width: 4096,
      height: 512,
      prompt: "a photo of the dolomites",
      scheduler: "K_EULER",
      num_outputs: 1,
      guidance_scale: 7.5,
      num_inference_steps: 20
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run omerbt/multidiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "omerbt/multidiffusion:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387",
    input={
        "width": 4096,
        "height": 512,
        "prompt": "a photo of the dolomites",
        "scheduler": "K_EULER",
        "num_outputs": 1,
        "guidance_scale": 7.5,
        "num_inference_steps": 20
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run omerbt/multidiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387",
    "input": {
      "width": 4096,
      "height": 512,
      "prompt": "a photo of the dolomites",
      "scheduler": "K_EULER",
      "num_outputs": 1,
      "guidance_scale": 7.5,
      "num_inference_steps": 20
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/omerbt/multidiffusion@sha256:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387 \
  -i 'width=4096' \
  -i 'height=512' \
  -i 'prompt="a photo of the dolomites"' \
  -i 'scheduler="K_EULER"' \
  -i 'num_outputs=1' \
  -i 'guidance_scale=7.5' \
  -i 'num_inference_steps=20'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/omerbt/multidiffusion@sha256:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "width": 4096,
      "height": 512,
      "prompt": "a photo of the dolomites",
      "scheduler": "K_EULER",
      "num_outputs": 1,
      "guidance_scale": 7.5,
      "num_inference_steps": 20
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2023-02-24T22:54:34.259451Z",
  "created_at": "2023-02-24T22:49:16.619509Z",
  "data_removed": false,
  "error": null,
  "id": "iwwaat27dnam3jag2kq4ofao6i",
  "input": {
    "width": 4096,
    "height": 512,
    "prompt": "a photo of the dolomites",
    "scheduler": "K_EULER",
    "num_outputs": 1,
    "guidance_scale": 7.5,
    "num_inference_steps": 20
  },
  "logs": "Using seed: 11915\n  0%|          | 0/20 [00:00<?, ?it/s]\n  5%|▌         | 1/20 [00:10<03:21, 10.59s/it]\n 10%|█         | 2/20 [00:19<02:51,  9.54s/it]\n 15%|█▌        | 3/20 [00:28<02:37,  9.24s/it]\n 20%|██        | 4/20 [00:37<02:26,  9.13s/it]\n 25%|██▌       | 5/20 [00:46<02:16,  9.09s/it]\n 30%|███       | 6/20 [00:55<02:07,  9.09s/it]\n 35%|███▌      | 7/20 [01:04<01:58,  9.12s/it]\n 40%|████      | 8/20 [01:13<01:49,  9.16s/it]\n 45%|████▌     | 9/20 [01:23<01:41,  9.19s/it]\n 50%|█████     | 10/20 [01:32<01:32,  9.23s/it]\n 55%|█████▌    | 11/20 [01:41<01:23,  9.29s/it]\n 60%|██████    | 12/20 [01:51<01:14,  9.36s/it]\n 65%|██████▌   | 13/20 [02:00<01:05,  9.39s/it]\n 70%|███████   | 14/20 [02:10<00:56,  9.39s/it]\n 75%|███████▌  | 15/20 [02:19<00:46,  9.38s/it]\n 80%|████████  | 16/20 [02:28<00:37,  9.36s/it]\n 85%|████████▌ | 17/20 [02:38<00:28,  9.34s/it]\n 90%|█████████ | 18/20 [02:47<00:18,  9.33s/it]\n 95%|█████████▌| 19/20 [02:56<00:09,  9.34s/it]\n100%|██████████| 20/20 [03:06<00:00,  9.35s/it]\n100%|██████████| 20/20 [03:06<00:00,  9.31s/it]",
  "metrics": {
    "predict_time": 191.736824,
    "total_time": 317.639942
  },
  "output": [
    "https://replicate.delivery/pbxt/5yWjQbNu3paaFZEUdR9eBvJCg3gtWgDheTjSNvM9HYmpIvhQA/out-0.png"
  ],
  "started_at": "2023-02-24T22:51:22.522627Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/iwwaat27dnam3jag2kq4ofao6i",
    "cancel": "https://api.replicate.com/v1/predictions/iwwaat27dnam3jag2kq4ofao6i/cancel"
  },
  "version": "963ffeaa7d992479bbbf6ad5b07f2ebbd51453f5214de98c87b32b97b44f5863"
}

Generated in

3 minutes 12 seconds

Tweak it ShareReport

Prediction

omerbt/multidiffusion:fd8e12cb

Model

omerbt/multidiffusion:24dd8961

qyiwfho7n5aafjr7oni3zgalwi

Status

Succeeded

Source

Web

Hardware

–

Total duration

69.1s

Created

about 2 years ago

Input

width: 4096
height: 512
prompt: a photo of a city skyline at night
scheduler: DDIM
num_outputs: 1
guidance_scale: 7.5
num_inference_steps: 20

{
  "width": 4096,
  "height": 512,
  "prompt": "a photo of a city skyline at night",
  "scheduler": "DDIM",
  "num_outputs": 1,
  "guidance_scale": 7.5,
  "num_inference_steps": 20
}

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run omerbt/multidiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "omerbt/multidiffusion:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387",
  {
    input: {
      width: 4096,
      height: 512,
      prompt: "a photo of a city skyline at night",
      scheduler: "DDIM",
      num_outputs: 1,
      guidance_scale: 7.5,
      num_inference_steps: 20
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run omerbt/multidiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "omerbt/multidiffusion:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387",
    input={
        "width": 4096,
        "height": 512,
        "prompt": "a photo of a city skyline at night",
        "scheduler": "DDIM",
        "num_outputs": 1,
        "guidance_scale": 7.5,
        "num_inference_steps": 20
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run omerbt/multidiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387",
    "input": {
      "width": 4096,
      "height": 512,
      "prompt": "a photo of a city skyline at night",
      "scheduler": "DDIM",
      "num_outputs": 1,
      "guidance_scale": 7.5,
      "num_inference_steps": 20
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/omerbt/multidiffusion@sha256:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387 \
  -i 'width=4096' \
  -i 'height=512' \
  -i 'prompt="a photo of a city skyline at night"' \
  -i 'scheduler="DDIM"' \
  -i 'num_outputs=1' \
  -i 'guidance_scale=7.5' \
  -i 'num_inference_steps=20'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/omerbt/multidiffusion@sha256:fd8e12cbfb7cf355d9d80afef487a34bf58b916bedc5bff5f7a8273fe72c1387
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "width": 4096,
      "height": 512,
      "prompt": "a photo of a city skyline at night",
      "scheduler": "DDIM",
      "num_outputs": 1,
      "guidance_scale": 7.5,
      "num_inference_steps": 20
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2023-02-24T23:06:53.309387Z",
  "created_at": "2023-02-24T23:05:44.172018Z",
  "data_removed": false,
  "error": null,
  "id": "qyiwfho7n5aafjr7oni3zgalwi",
  "input": {
    "width": 4096,
    "height": 512,
    "prompt": "a photo of a city skyline at night",
    "scheduler": "DDIM",
    "num_outputs": 1,
    "guidance_scale": 7.5,
    "num_inference_steps": 20
  },
  "logs": "Using seed: 57162\n  0%|          | 0/20 [00:00<?, ?it/s]\n  5%|▌         | 1/20 [00:03<01:03,  3.33s/it]\n 10%|█         | 2/20 [00:06<00:59,  3.30s/it]\n 15%|█▌        | 3/20 [00:09<00:55,  3.28s/it]\n 20%|██        | 4/20 [00:13<00:53,  3.35s/it]\n 25%|██▌       | 5/20 [00:16<00:49,  3.30s/it]\n 30%|███       | 6/20 [00:19<00:44,  3.21s/it]\n 35%|███▌      | 7/20 [00:22<00:41,  3.17s/it]\n 40%|████      | 8/20 [00:25<00:37,  3.14s/it]\n 45%|████▌     | 9/20 [00:28<00:34,  3.14s/it]\n 50%|█████     | 10/20 [00:32<00:32,  3.27s/it]\n 55%|█████▌    | 11/20 [00:35<00:30,  3.35s/it]\n 60%|██████    | 12/20 [00:39<00:27,  3.41s/it]\n 65%|██████▌   | 13/20 [00:43<00:24,  3.44s/it]\n 70%|███████   | 14/20 [00:46<00:20,  3.47s/it]\n 75%|███████▌  | 15/20 [00:50<00:17,  3.48s/it]\n 80%|████████  | 16/20 [00:53<00:13,  3.43s/it]\n 85%|████████▌ | 17/20 [00:56<00:10,  3.43s/it]\n 90%|█████████ | 18/20 [01:00<00:06,  3.42s/it]\n 95%|█████████▌| 19/20 [01:03<00:03,  3.43s/it]\n100%|██████████| 20/20 [01:07<00:00,  3.41s/it]\n100%|██████████| 20/20 [01:07<00:00,  3.35s/it]",
  "metrics": {
    "predict_time": 69.066816,
    "total_time": 69.137369
  },
  "output": [
    "https://replicate.delivery/pbxt/BmhVHMk1Lx5iDh7Ak0Y3EjmP6OeFotuDtOwbFw8pjf0MUvhQA/out-0.png"
  ],
  "started_at": "2023-02-24T23:05:44.242571Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/qyiwfho7n5aafjr7oni3zgalwi",
    "cancel": "https://api.replicate.com/v1/predictions/qyiwfho7n5aafjr7oni3zgalwi/cancel"
  },
  "version": "24dd8961771359f16dfd1238566483fbfa2a90092ed9eeac0337041f45387aca"
}

Generated in

69.1 seconds

Tweak it ShareReport

Want to make some of these yourself?

Run this model

omerbt / multidiffusion

Prediction

Input

Output

Prediction

Input

Output

Logs (iwwaat27dnam3jag2kq4ofao6i)

Logs (qyiwfho7n5aafjr7oni3zgalwi)