joyanujoy/analog_diffusion – Run with an API on Replicate

joyanujoy / analog_diffusion

Some scrappy experiments 🫣

Cold

Public
719 runs
A100 (80GB)
GitHub
License

Iterate in the playground

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

Input prompt. Use <1>, <2>, <3>, etc., to specify LoRA concepts

Default: "a photo of <1> riding a horse on mars"

negative_prompt

string

Shift + Return to add a new line

Specify things to not see in the output

Default: ""

width

integer

Width of output image. Maximum size is 1024x768 or 768x1024 because of memory limits

Default: 512

height

integer

Height of output image. Maximum size is 1024x768 or 768x1024 because of memory limits

Default: 512

num_outputs

integer

(minimum: 1, maximum: 4)

Number of images to output.

Default: 1

num_inference_steps

integer

(minimum: 1, maximum: 500)

Number of denoising steps

Default: 50

guidance_scale

number

(minimum: 1, maximum: 20)

Scale for classifier-free guidance

Default: 7.5

image

file

(Img2Img) Inital image to generate variations of. If this is not none, Img2Img will be invoked.

remove_background

boolean

Automatically remove background from above Img2Img initial image.

Default: true

clip_interrogator

boolean

Generate prompt from init image. Generated prompt will override manually entered text prompt input

Default: true

prompt_strength

number

(Img2Img) Prompt strength when providing the image. 1.0 corresponds to full destruction of information in init image

Default: 0.8

scheduler

string

Choose a scheduler.

Default: "DPMSolverMultistep"

lora_urls

string

Shift + Return to add a new line

List of urls for safetensors of lora models, seperated with | .

Default: ""

lora_scales

string

Shift + Return to add a new line

List of scales for safetensors of lora models, seperated with |

Default: "0.5"

seed

integer

Random seed. Leave blank to randomize the seed

adapter_condition_image

file

(T2I-adapter) Adapter Condition Image to gain extra control over generation. If this is not none, T2I adapter will be invoked.

adapter_type

string

(T2I-adapter) Choose an adapter type for the additional condition.

Default: "sketch"

upscale

integer

Upscaling factor

verbose_response

boolean

Return a response object with detail info. Prompt, seed used etc.

Default: false

Run this model in Node.js with one line of code:

npx create-replicate --model=joyanujoy/analog_diffusion

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run joyanujoy/analog_diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "joyanujoy/analog_diffusion:eff6035c43836061271808afa1d8ed256e603043489093a9b35bfee9cfe53e48",
  {
    input: {
      width: 512,
      height: 512,
      prompt: "analog style photo of a man",
      upscale: 2,
      lora_urls: "",
      scheduler: "DPMSolverMultistep",
      lora_scales: "0.5",
      num_outputs: 1,
      adapter_type: "sketch",
      guidance_scale: 7.5,
      negative_prompt: "",
      prompt_strength: 0.8,
      verbose_response: false,
      clip_interrogator: true,
      remove_background: false,
      num_inference_steps: 50
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run joyanujoy/analog_diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "joyanujoy/analog_diffusion:eff6035c43836061271808afa1d8ed256e603043489093a9b35bfee9cfe53e48",
    input={
        "width": 512,
        "height": 512,
        "prompt": "analog style photo of a man",
        "upscale": 2,
        "lora_urls": "",
        "scheduler": "DPMSolverMultistep",
        "lora_scales": "0.5",
        "num_outputs": 1,
        "adapter_type": "sketch",
        "guidance_scale": 7.5,
        "negative_prompt": "",
        "prompt_strength": 0.8,
        "verbose_response": False,
        "clip_interrogator": True,
        "remove_background": False,
        "num_inference_steps": 50
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run joyanujoy/analog_diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "joyanujoy/analog_diffusion:eff6035c43836061271808afa1d8ed256e603043489093a9b35bfee9cfe53e48",
    "input": {
      "width": 512,
      "height": 512,
      "prompt": "analog style photo of a man",
      "upscale": 2,
      "lora_urls": "",
      "scheduler": "DPMSolverMultistep",
      "lora_scales": "0.5",
      "num_outputs": 1,
      "adapter_type": "sketch",
      "guidance_scale": 7.5,
      "negative_prompt": "",
      "prompt_strength": 0.8,
      "verbose_response": false,
      "clip_interrogator": true,
      "remove_background": false,
      "num_inference_steps": 50
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2023-03-23T18:30:55.965066Z",
  "created_at": "2023-03-23T18:28:19.794684Z",
  "data_removed": false,
  "error": null,
  "id": "csascvfyijbt3pjmvrfhfyvrja",
  "input": {
    "width": 512,
    "height": 512,
    "prompt": "analog style photo of a man",
    "scheduler": "DPMSolverMultistep",
    "lora_scales": "0.5",
    "num_outputs": 1,
    "adapter_type": "sketch",
    "guidance_scale": 7.5,
    "prompt_strength": 0.8,
    "clip_interrogator": true,
    "remove_background": false,
    "num_inference_steps": 50
  },
  "logs": "Using seed: 55601\nGenerating image of 512 x 512 with prompt: analog style photo of a man\nNo LoRA models provided, using default model...\n  0%|          | 0/50 [00:00<?, ?it/s]\n  2%|▏         | 1/50 [00:01<01:34,  1.93s/it]\n  6%|▌         | 3/50 [00:02<00:26,  1.78it/s]\n 10%|█         | 5/50 [00:02<00:14,  3.19it/s]\n 14%|█▍        | 7/50 [00:02<00:09,  4.69it/s]\n 18%|█▊        | 9/50 [00:02<00:06,  6.07it/s]\n 22%|██▏       | 11/50 [00:02<00:05,  7.35it/s]\n 26%|██▌       | 13/50 [00:02<00:04,  8.57it/s]\n 30%|███       | 15/50 [00:03<00:03,  9.66it/s]\n 34%|███▍      | 17/50 [00:03<00:03, 10.59it/s]\n 38%|███▊      | 19/50 [00:03<00:02, 11.20it/s]\n 42%|████▏     | 21/50 [00:03<00:02, 11.42it/s]\n 46%|████▌     | 23/50 [00:03<00:02, 11.58it/s]\n 50%|█████     | 25/50 [00:03<00:02, 11.88it/s]\n 54%|█████▍    | 27/50 [00:04<00:01, 11.91it/s]\n 58%|█████▊    | 29/50 [00:04<00:01, 12.21it/s]\n 62%|██████▏   | 31/50 [00:04<00:01, 12.08it/s]\n 66%|██████▌   | 33/50 [00:04<00:01, 12.26it/s]\n 70%|███████   | 35/50 [00:04<00:01, 12.02it/s]\n 74%|███████▍  | 37/50 [00:04<00:01, 12.33it/s]\n 78%|███████▊  | 39/50 [00:04<00:00, 12.19it/s]\n 82%|████████▏ | 41/50 [00:05<00:00, 12.07it/s]\n 86%|████████▌ | 43/50 [00:05<00:00, 12.31it/s]\n 90%|█████████ | 45/50 [00:05<00:00, 12.61it/s]\n 94%|█████████▍| 47/50 [00:05<00:00, 12.44it/s]\n 98%|█████████▊| 49/50 [00:05<00:00, 13.00it/s]\n100%|██████████| 50/50 [00:05<00:00,  8.56it/s]",
  "metrics": {
    "predict_time": 7.81628,
    "total_time": 156.170382
  },
  "output": [
    "https://replicate.delivery/pbxt/0iNkiPzqlGbvHpKLkdDga7q6FWd2gZeKNVj71G2ibltvZSVIA/out-0.png"
  ],
  "started_at": "2023-03-23T18:30:48.148786Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/csascvfyijbt3pjmvrfhfyvrja",
    "cancel": "https://api.replicate.com/v1/predictions/csascvfyijbt3pjmvrfhfyvrja/cancel"
  },
  "version": "eff6035c43836061271808afa1d8ed256e603043489093a9b35bfee9cfe53e48"
}

Generated in

7.8 seconds

Tweak itReport

Examples

View more examples

Run time and cost

This model costs approximately $0.012 to run on Replicate, or 83 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 9 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Model description

In your prompt, use the activation token: analog style

Based on wavymulder/Analog-Diffusion. Replicate deployment is adapted from the original lore-inference repo by cloneofsimo. I’m using this for running experiments easily without having to spin up a GPU server for automatic111 webui.

Main modifications are:

Add RealESRGAN for upscaling
Add Clip Interrogator for auto generating initial img2img prompt
Add rembg for background removal from init image.
Automatically resize adaptor condition image size to match output image.
Modify replicate output schema to return additonal info - seed, clip interrogator generated prompt etc.[API works but breaks replicate web UI explorer]
Modify the scripts for download weights, test and deploy to download and cache large model files. Bake the cache files into container image to avoid long wait for model downloads everytime container warms up.

…

Caveats and recommendations

Set verbose_response: false in replicate web ui. Setting this to true this breaks the web ui.
For viewing the output images in replicate web, use the version eff6035c
To receive verbose response with additional fields - clip interrogator generated prompt, seed used etc. use the version 1924c521

…