laion-ai / laionide

Generate images from text quickly. See https://replicate.com/afiaka87/laionide-v2 for a new checkpoint.

Cold

Public
7.3K runs
T4
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

trippy acid lsd psychedelic oil painting of a man with a suittrippy acid lsd psychedelic oil painting of a man with a suit

Text prompt to use.

Default: "The Lovers demon skull werewolf tentacle tarot card | Tarot card on artstation | eldritch card"

batch_size

integer

(minimum: 1, maximum: 6)

Batch size. Number of generations to predict

Default: 3

side_x

integer

Must be multiple of 8. Going above 64 is not recommended. Actual image will be 4x larger.

Default: 64

side_y

integer

Must be multiple of 8. Going above 64 is not recommended. Actual image will be 4x larger.

Default: 64

upsample_stage

boolean

Performs prompt-aware upsampling by 4x base resolution

Default: true

guidance_scale

number

Classifier-free guidance scale. Higher values move further away from unconditional outputs. Lower values move closer to unconditional outputs. Negative values guide towards semantically opposite classes. 4-16 is a reasonable range.

Default: 4

upsample_temp

string

Upsample temperature. Consider lowering to ~0.997 for blurry images with fewer artifacts.

Default: "0.998"

timestep_respacing

string

Number of timesteps to use for base model. Going above 50 has diminishing returns.

Default: "27"

sr_timestep_respacing

string

Number of timesteps to use for base model. Going above 40 has diminishing returns.

Default: "17"

seed

integer

Seed for reproducibility

Default: 0

Run this model in Node.js with one line of code:

npx create-replicate --model=laion-ai/laionide

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run laion-ai/laionide using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "laion-ai/laionide:46a74a85557bac318551c16aef91a1503a029ec1e377a9dd954655bb91197c31",
  {
    input: {
      seed: 0,
      prompt: "trippy acid lsd psychedelic oil painting of a man with a suit",
      side_x: 80,
      side_y: 48,
      batch_size: 3,
      upsample_temp: "0.998",
      guidance_scale: 12,
      upsample_stage: true,
      timestep_respacing: "27",
      sr_timestep_respacing: "17"
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run laion-ai/laionide using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "laion-ai/laionide:46a74a85557bac318551c16aef91a1503a029ec1e377a9dd954655bb91197c31",
    input={
        "seed": 0,
        "prompt": "trippy acid lsd psychedelic oil painting of a man with a suit",
        "side_x": 80,
        "side_y": 48,
        "batch_size": 3,
        "upsample_temp": "0.998",
        "guidance_scale": 12,
        "upsample_stage": True,
        "timestep_respacing": "27",
        "sr_timestep_respacing": "17"
    }
)

# The laion-ai/laionide model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/laion-ai/laionide/api#output-schema
    print(item)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run laion-ai/laionide using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "laion-ai/laionide:46a74a85557bac318551c16aef91a1503a029ec1e377a9dd954655bb91197c31",
    "input": {
      "seed": 0,
      "prompt": "trippy acid lsd psychedelic oil painting of a man with a suit",
      "side_x": 80,
      "side_y": 48,
      "batch_size": 3,
      "upsample_temp": "0.998",
      "guidance_scale": 12,
      "upsample_stage": true,
      "timestep_respacing": "27",
      "sr_timestep_respacing": "17"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2022-02-16T00:19:30.609657Z",
  "created_at": "2022-02-16T00:18:59.571967Z",
  "data_removed": false,
  "error": null,
  "id": "4bcmbmh3lbapxnhjlftzomxis4",
  "input": {
    "seed": 0,
    "prompt": "trippy acid lsd psychedelic oil painting of a man with a suit",
    "side_x": "80",
    "side_y": "48",
    "batch_size": "3",
    "upsample_temp": "0.998",
    "guidance_scale": 12,
    "upsample_stage": true,
    "timestep_respacing": "27",
    "sr_timestep_respacing": "17"
  },
  "logs": "Generating 80x48 samples with 27 timesteps using GLIDE-base-64px...\n\n  0%|          | 0/25 [00:00<?, ?it/s]\n  4%|▍         | 1/25 [00:00<00:18,  1.31it/s]\n  8%|▊         | 2/25 [00:01<00:17,  1.33it/s]\n 12%|█▏        | 3/25 [00:02<00:16,  1.34it/s]\n 16%|█▌        | 4/25 [00:02<00:10,  1.91it/s]\n 20%|██        | 5/25 [00:02<00:08,  2.49it/s]\n 24%|██▍       | 6/25 [00:02<00:06,  3.06it/s]\n 28%|██▊       | 7/25 [00:02<00:05,  3.58it/s]\n 32%|███▏      | 8/25 [00:03<00:04,  4.02it/s]\n 36%|███▌      | 9/25 [00:03<00:03,  4.37it/s]\n 40%|████      | 10/25 [00:03<00:03,  4.64it/s]\n 44%|████▍     | 11/25 [00:03<00:02,  4.83it/s]\n 48%|████▊     | 12/25 [00:03<00:02,  4.99it/s]\n 52%|█████▏    | 13/25 [00:04<00:02,  5.11it/s]\n 56%|█████▌    | 14/25 [00:04<00:02,  5.21it/s]\n 60%|██████    | 15/25 [00:04<00:01,  5.28it/s]\n 64%|██████▍   | 16/25 [00:04<00:01,  5.32it/s]\n 68%|██████▊   | 17/25 [00:04<00:01,  5.35it/s]\n 72%|███████▏  | 18/25 [00:05<00:01,  5.36it/s]\n 76%|███████▌  | 19/25 [00:05<00:01,  5.36it/s]\n 80%|████████  | 20/25 [00:05<00:00,  5.36it/s]\n 84%|████████▍ | 21/25 [00:05<00:00,  5.38it/s]\n 88%|████████▊ | 22/25 [00:05<00:00,  5.38it/s]\n 92%|█████████▏| 23/25 [00:05<00:00,  5.39it/s]\n 96%|█████████▌| 24/25 [00:06<00:00,  5.37it/s]\n100%|██████████| 25/25 [00:06<00:00,  5.35it/s]\n100%|██████████| 25/25 [00:06<00:00,  3.96it/s]\nUpsampling outputs from GLIDE-base 80x48 to 320x192 using 17 timesteps...\n\n  0%|          | 0/15 [00:00<?, ?it/s]\n  7%|▋         | 1/15 [00:01<00:26,  1.89s/it]\n 13%|█▎        | 2/15 [00:03<00:24,  1.89s/it]\n 20%|██        | 3/15 [00:05<00:22,  1.89s/it]\n 27%|██▋       | 4/15 [00:06<00:14,  1.33s/it]\n 33%|███▎      | 5/15 [00:06<00:10,  1.02s/it]\n 40%|████      | 6/15 [00:07<00:07,  1.19it/s]\n 47%|████▋     | 7/15 [00:07<00:05,  1.39it/s]\n 53%|█████▎    | 8/15 [00:08<00:04,  1.55it/s]\n 60%|██████    | 9/15 [00:08<00:03,  1.68it/s]\n 67%|██████▋   | 10/15 [00:09<00:02,  1.78it/s]\n 73%|███████▎  | 11/15 [00:09<00:02,  1.86it/s]\n 80%|████████  | 12/15 [00:10<00:01,  1.91it/s]\n 87%|████████▋ | 13/15 [00:10<00:01,  1.95it/s]\n 93%|█████████▎| 14/15 [00:10<00:00,  1.98it/s]\n100%|██████████| 15/15 [00:11<00:00,  2.00it/s]\n100%|██████████| 15/15 [00:11<00:00,  1.31it/s]",
  "metrics": {
    "predict_time": 30.742278,
    "total_time": 31.03769
  },
  "output": [
    {
      "file": "https://replicate.delivery/mgxm/6e152613-c32a-438c-a7d6-56d8f1d9dc48/upsample_predictions.png"
    }
  ],
  "started_at": "2022-02-16T00:18:59.867379Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/4bcmbmh3lbapxnhjlftzomxis4",
    "cancel": "https://api.replicate.com/v1/predictions/4bcmbmh3lbapxnhjlftzomxis4/cancel"
  },
  "version": "11a727b3c74f842f7e9d7db88df5da4d1b8389c14e186d6ceb1724abeebe4bc9"
}

Generated in

30.8 seconds

Tweak itReport

Generating 80x48 samples with 27 timesteps using GLIDE-base-64px...

  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:00<00:18,  1.31it/s]
  8%|▊         | 2/25 [00:01<00:17,  1.33it/s]
 12%|█▏        | 3/25 [00:02<00:16,  1.34it/s]
 16%|█▌        | 4/25 [00:02<00:10,  1.91it/s]
 20%|██        | 5/25 [00:02<00:08,  2.49it/s]
 24%|██▍       | 6/25 [00:02<00:06,  3.06it/s]
 28%|██▊       | 7/25 [00:02<00:05,  3.58it/s]
 32%|███▏      | 8/25 [00:03<00:04,  4.02it/s]
 36%|███▌      | 9/25 [00:03<00:03,  4.37it/s]
 40%|████      | 10/25 [00:03<00:03,  4.64it/s]
 44%|████▍     | 11/25 [00:03<00:02,  4.83it/s]
 48%|████▊     | 12/25 [00:03<00:02,  4.99it/s]
 52%|█████▏    | 13/25 [00:04<00:02,  5.11it/s]
 56%|█████▌    | 14/25 [00:04<00:02,  5.21it/s]
 60%|██████    | 15/25 [00:04<00:01,  5.28it/s]
 64%|██████▍   | 16/25 [00:04<00:01,  5.32it/s]
 68%|██████▊   | 17/25 [00:04<00:01,  5.35it/s]
 72%|███████▏  | 18/25 [00:05<00:01,  5.36it/s]
 76%|███████▌  | 19/25 [00:05<00:01,  5.36it/s]
 80%|████████  | 20/25 [00:05<00:00,  5.36it/s]
 84%|████████▍ | 21/25 [00:05<00:00,  5.38it/s]
 88%|████████▊ | 22/25 [00:05<00:00,  5.38it/s]
 92%|█████████▏| 23/25 [00:05<00:00,  5.39it/s]
 96%|█████████▌| 24/25 [00:06<00:00,  5.37it/s]
100%|██████████| 25/25 [00:06<00:00,  5.35it/s]
100%|██████████| 25/25 [00:06<00:00,  3.96it/s]
Upsampling outputs from GLIDE-base 80x48 to 320x192 using 17 timesteps...

  0%|          | 0/15 [00:00<?, ?it/s]
  7%|▋         | 1/15 [00:01<00:26,  1.89s/it]
 13%|█▎        | 2/15 [00:03<00:24,  1.89s/it]
 20%|██        | 3/15 [00:05<00:22,  1.89s/it]
 27%|██▋       | 4/15 [00:06<00:14,  1.33s/it]
 33%|███▎      | 5/15 [00:06<00:10,  1.02s/it]
 40%|████      | 6/15 [00:07<00:07,  1.19it/s]
 47%|████▋     | 7/15 [00:07<00:05,  1.39it/s]
 53%|█████▎    | 8/15 [00:08<00:04,  1.55it/s]
 60%|██████    | 9/15 [00:08<00:03,  1.68it/s]
 67%|██████▋   | 10/15 [00:09<00:02,  1.78it/s]
 73%|███████▎  | 11/15 [00:09<00:02,  1.86it/s]
 80%|████████  | 12/15 [00:10<00:01,  1.91it/s]
 87%|████████▋ | 13/15 [00:10<00:01,  1.95it/s]
 93%|█████████▎| 14/15 [00:10<00:00,  1.98it/s]
100%|██████████| 15/15 [00:11<00:00,  2.00it/s]
100%|██████████| 15/15 [00:11<00:00,  1.31it/s]

This output was created using a different version of the model, laion-ai/laionide:11a727b3.

Examples

View more examples

Run time and cost

This model runs on Nvidia T4 GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

LAIONIDE

Generate images from text - quickly.

Update: February 22, 2022

Thanks to stability.ai I was able to continue finetuning on LAION2B for awhile. Check it out at laionide v2

Update: February 15, 2022

Significantly reduced number of steps needed, improved generations.
(thanks to) Added PLMS sampling from Pseudo Numerical Methods for Diffusion. Implementation by Katherine Crowson (https://github.com/crowsonkb/glide-text2im).

Coming soon:

Checkpoints finetuned even more on LAION2B.
ETA: 3-5 days

GLIDE (base filtered) finetuned on 1 million samples from LAION400M for 2 epochs.

The upsampler is also finetuned for about 200K samples.

Checkpoint by Clay Mullis aka afiaka87. Code modified from openai/glide-text2im/notebooks/clip-guided.ipynb

Thanks to nshepperd, neverix, alstroemeria, Jack, Kianne, Thuna, valteralred and others!

Thanks to all of the team and contributors at laion.ai and the dalle-pytorch discord for creating a great dataset and community.