lucataco / style-aligned

GoogleAI: Style Aligned Image Generation via Shared Attention

Cold

Public
1.2K runs
A100 (80GB)
GitHub
Paper
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

Toy train colorful, macro photo.
Toy airplane colorful, macro photo.
Toy car colorful, macro photo.
Toy boat colorful, macro photo.Toy train colorful, macro photo.
Toy airplane colorful, macro photo.
Toy car colorful, macro photo.
Toy boat colorful, macro photo.

Input prompts, separated by newlines

style_prompt

*string

Shift + Return to add a new line

Input style prompt. If using an image for style transfer, this should describe the style of that image

negative_prompt

string

Shift + Return to add a new line

Input negative prompt

Default: "low-resolution"

image

file

Input image for style transfer

image_subject

string

Shift + Return to add a new line

Subject (not style) of image used for style transfer. Ignored if image not set.

Default: "None"

shared_score_shift

number

(minimum: 0.01, maximum: 10)

Shared score shift (Actual value will take log of this). Higher value induces higher fidelity, set 1 for no shift

Default: 2

shared_score_scale

number

(minimum: 0, maximum: 1)

Shared score scale. Higher value induces higher, set 1 for no rescale

Default: 1

guidance_scale

number

(minimum: 1.1, maximum: 20)

Guidance scale

Default: 7

width

integer

(minimum: 256, maximum: 2048)

Output image width. Will be rounded to nearest 64

Default: 768

height

integer

(minimum: 256, maximum: 2048)

Output image height. Will be rounded to nearest 64

Default: 768

num_inference_steps

integer

(minimum: 10, maximum: 150)

Number of inference steps

Default: 50

seed

integer

Random seed. Leave blank to randomize the seed

Run this model in Node.js with one line of code:

npx create-replicate --model=lucataco/style-aligned

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run lucataco/style-aligned using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "lucataco/style-aligned:4ae87bee984bc5346c5ec8d6f8573825d0c602b5f5f82fc15b12460b997f90c8",
  {
    input: {
      seed: 62967,
      width: 768,
      height: 768,
      prompt: "Toy train colorful, macro photo.\nToy airplane colorful, macro photo.\nToy car colorful, macro photo.\nToy boat colorful, macro photo.",
      image_subject: "None",
      guidance_scale: 7,
      negative_prompt: "low-resolution",
      shared_score_scale: 1,
      shared_score_shift: 2,
      num_inference_steps: 50
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run lucataco/style-aligned using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "lucataco/style-aligned:4ae87bee984bc5346c5ec8d6f8573825d0c602b5f5f82fc15b12460b997f90c8",
    input={
        "seed": 62967,
        "width": 768,
        "height": 768,
        "prompt": "Toy train colorful, macro photo.\nToy airplane colorful, macro photo.\nToy car colorful, macro photo.\nToy boat colorful, macro photo.",
        "image_subject": "None",
        "guidance_scale": 7,
        "negative_prompt": "low-resolution",
        "shared_score_scale": 1,
        "shared_score_shift": 2,
        "num_inference_steps": 50
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run lucataco/style-aligned using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "lucataco/style-aligned:4ae87bee984bc5346c5ec8d6f8573825d0c602b5f5f82fc15b12460b997f90c8",
    "input": {
      "seed": 62967,
      "width": 768,
      "height": 768,
      "prompt": "Toy train colorful, macro photo.\\nToy airplane colorful, macro photo.\\nToy car colorful, macro photo.\\nToy boat colorful, macro photo.",
      "image_subject": "None",
      "guidance_scale": 7,
      "negative_prompt": "low-resolution",
      "shared_score_scale": 1,
      "shared_score_shift": 2,
      "num_inference_steps": 50
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/lucataco/style-aligned@sha256:4ae87bee984bc5346c5ec8d6f8573825d0c602b5f5f82fc15b12460b997f90c8 \
  -i 'seed=62967' \
  -i 'width=768' \
  -i 'height=768' \
  -i $'prompt="Toy train colorful, macro photo.\\nToy airplane colorful, macro photo.\\nToy car colorful, macro photo.\\nToy boat colorful, macro photo."' \
  -i 'image_subject="None"' \
  -i 'guidance_scale=7' \
  -i 'negative_prompt="low-resolution"' \
  -i 'shared_score_scale=1' \
  -i 'shared_score_shift=2' \
  -i 'num_inference_steps=50'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/lucataco/style-aligned@sha256:4ae87bee984bc5346c5ec8d6f8573825d0c602b5f5f82fc15b12460b997f90c8
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "seed": 62967,
      "width": 768,
      "height": 768,
      "prompt": "Toy train colorful, macro photo.\\nToy airplane colorful, macro photo.\\nToy car colorful, macro photo.\\nToy boat colorful, macro photo.",
      "image_subject": "None",
      "guidance_scale": 7,
      "negative_prompt": "low-resolution",
      "shared_score_scale": 1,
      "shared_score_shift": 2,
      "num_inference_steps": 50
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2023-12-05T22:18:12.354683Z",
  "created_at": "2023-12-05T22:14:05.156174Z",
  "data_removed": false,
  "error": null,
  "id": "p5xbnxbb5i4u6j7s5p644d6x2u",
  "input": {
    "seed": 62967,
    "width": 768,
    "height": 768,
    "prompt": "Toy train colorful, macro photo.\nToy airplane colorful, macro photo.\nToy car colorful, macro photo.\nToy boat colorful, macro photo.",
    "negative_prompt": "low-resolution",
    "num_inference_steps": 50
  },
  "logs": "Using seed: 62967\n  0%|          | 0/50 [00:00<?, ?it/s]\n  2%|▏         | 1/50 [00:05<04:43,  5.78s/it]\n  4%|▍         | 2/50 [00:06<02:02,  2.56s/it]\n  6%|▌         | 3/50 [00:06<01:11,  1.53s/it]\n  8%|▊         | 4/50 [00:06<00:47,  1.04s/it]\n 10%|█         | 5/50 [00:06<00:34,  1.29it/s]\n 12%|█▏        | 6/50 [00:07<00:27,  1.63it/s]\n 14%|█▍        | 7/50 [00:07<00:22,  1.95it/s]\n 16%|█▌        | 8/50 [00:07<00:18,  2.25it/s]\n 18%|█▊        | 9/50 [00:08<00:16,  2.50it/s]\n 20%|██        | 10/50 [00:08<00:14,  2.71it/s]\n 22%|██▏       | 11/50 [00:08<00:13,  2.87it/s]\n 24%|██▍       | 12/50 [00:09<00:12,  3.00it/s]\n 26%|██▌       | 13/50 [00:09<00:11,  3.09it/s]\n 28%|██▊       | 14/50 [00:09<00:11,  3.16it/s]\n 30%|███       | 15/50 [00:09<00:10,  3.21it/s]\n 32%|███▏      | 16/50 [00:10<00:10,  3.24it/s]\n 34%|███▍      | 17/50 [00:10<00:10,  3.27it/s]\n 36%|███▌      | 18/50 [00:10<00:09,  3.29it/s]\n 38%|███▊      | 19/50 [00:11<00:09,  3.30it/s]\n 40%|████      | 20/50 [00:11<00:09,  3.29it/s]\n 42%|████▏     | 21/50 [00:11<00:08,  3.30it/s]\n 44%|████▍     | 22/50 [00:12<00:08,  3.31it/s]\n 46%|████▌     | 23/50 [00:12<00:08,  3.31it/s]\n 48%|████▊     | 24/50 [00:12<00:07,  3.32it/s]\n 50%|█████     | 25/50 [00:13<00:07,  3.32it/s]\n 52%|█████▏    | 26/50 [00:13<00:07,  3.32it/s]\n 54%|█████▍    | 27/50 [00:13<00:06,  3.32it/s]\n 56%|█████▌    | 28/50 [00:13<00:06,  3.32it/s]\n 58%|█████▊    | 29/50 [00:14<00:06,  3.32it/s]\n 60%|██████    | 30/50 [00:14<00:06,  3.32it/s]\n 62%|██████▏   | 31/50 [00:14<00:05,  3.32it/s]\n 64%|██████▍   | 32/50 [00:15<00:05,  3.32it/s]\n 66%|██████▌   | 33/50 [00:15<00:05,  3.32it/s]\n 68%|██████▊   | 34/50 [00:15<00:04,  3.32it/s]\n 70%|███████   | 35/50 [00:16<00:04,  3.32it/s]\n 72%|███████▏  | 36/50 [00:16<00:04,  3.32it/s]\n 74%|███████▍  | 37/50 [00:16<00:03,  3.31it/s]\n 76%|███████▌  | 38/50 [00:16<00:03,  3.31it/s]\n 78%|███████▊  | 39/50 [00:17<00:03,  3.31it/s]\n 80%|████████  | 40/50 [00:17<00:03,  3.31it/s]\n 82%|████████▏ | 41/50 [00:17<00:02,  3.30it/s]\n 84%|████████▍ | 42/50 [00:18<00:02,  3.31it/s]\n 86%|████████▌ | 43/50 [00:18<00:02,  3.31it/s]\n 88%|████████▊ | 44/50 [00:18<00:01,  3.31it/s]\n 90%|█████████ | 45/50 [00:19<00:01,  3.31it/s]\n 92%|█████████▏| 46/50 [00:19<00:01,  3.31it/s]\n 94%|█████████▍| 47/50 [00:19<00:00,  3.32it/s]\n 96%|█████████▌| 48/50 [00:19<00:00,  3.32it/s]\n 98%|█████████▊| 49/50 [00:20<00:00,  3.32it/s]\n100%|██████████| 50/50 [00:20<00:00,  3.32it/s]\n100%|██████████| 50/50 [00:20<00:00,  2.43it/s]",
  "metrics": {
    "predict_time": 27.773339,
    "total_time": 247.198509
  },
  "output": [
    "https://replicate.delivery/pbxt/BXpfqgS1ErTfRUmI9fxcxBY5inzWYayPXIpKNayZKnqDdqeHB/output-0.png",
    "https://replicate.delivery/pbxt/E7e2jdEllnT7Z6hlXWuCIuj6fi5lrJh99kjeYNnwdw9FdqeHB/output-1.png",
    "https://replicate.delivery/pbxt/oNpi9q3SY26cO1kzjqL7eJqv3G9aGUTMlhPPVOU8if8jOVfjA/output-2.png",
    "https://replicate.delivery/pbxt/pkolzSOMs6JrBpdfaQUrGICCa9gYE2GsumJnFlf54tljOVfjA/output-3.png"
  ],
  "started_at": "2023-12-05T22:17:44.581344Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/p5xbnxbb5i4u6j7s5p644d6x2u",
    "cancel": "https://api.replicate.com/v1/predictions/p5xbnxbb5i4u6j7s5p644d6x2u/cancel"
  },
  "version": "a9086389a701a88fd63e6739e5699d3f1b7ba3433bb2ec60dd3fb6be81ee8a2e"
}

Generated in

27.8 seconds

Tweak itReport View full prediction

Using seed: 62967
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:05<04:43,  5.78s/it]
  4%|▍         | 2/50 [00:06<02:02,  2.56s/it]
  6%|▌         | 3/50 [00:06<01:11,  1.53s/it]
  8%|▊         | 4/50 [00:06<00:47,  1.04s/it]
 10%|█         | 5/50 [00:06<00:34,  1.29it/s]
 12%|█▏        | 6/50 [00:07<00:27,  1.63it/s]
 14%|█▍        | 7/50 [00:07<00:22,  1.95it/s]
 16%|█▌        | 8/50 [00:07<00:18,  2.25it/s]
 18%|█▊        | 9/50 [00:08<00:16,  2.50it/s]
 20%|██        | 10/50 [00:08<00:14,  2.71it/s]
 22%|██▏       | 11/50 [00:08<00:13,  2.87it/s]
 24%|██▍       | 12/50 [00:09<00:12,  3.00it/s]
 26%|██▌       | 13/50 [00:09<00:11,  3.09it/s]
 28%|██▊       | 14/50 [00:09<00:11,  3.16it/s]
 30%|███       | 15/50 [00:09<00:10,  3.21it/s]
 32%|███▏      | 16/50 [00:10<00:10,  3.24it/s]
 34%|███▍      | 17/50 [00:10<00:10,  3.27it/s]
 36%|███▌      | 18/50 [00:10<00:09,  3.29it/s]
 38%|███▊      | 19/50 [00:11<00:09,  3.30it/s]
 40%|████      | 20/50 [00:11<00:09,  3.29it/s]
 42%|████▏     | 21/50 [00:11<00:08,  3.30it/s]
 44%|████▍     | 22/50 [00:12<00:08,  3.31it/s]
 46%|████▌     | 23/50 [00:12<00:08,  3.31it/s]
 48%|████▊     | 24/50 [00:12<00:07,  3.32it/s]
 50%|█████     | 25/50 [00:13<00:07,  3.32it/s]
 52%|█████▏    | 26/50 [00:13<00:07,  3.32it/s]
 54%|█████▍    | 27/50 [00:13<00:06,  3.32it/s]
 56%|█████▌    | 28/50 [00:13<00:06,  3.32it/s]
 58%|█████▊    | 29/50 [00:14<00:06,  3.32it/s]
 60%|██████    | 30/50 [00:14<00:06,  3.32it/s]
 62%|██████▏   | 31/50 [00:14<00:05,  3.32it/s]
 64%|██████▍   | 32/50 [00:15<00:05,  3.32it/s]
 66%|██████▌   | 33/50 [00:15<00:05,  3.32it/s]
 68%|██████▊   | 34/50 [00:15<00:04,  3.32it/s]
 70%|███████   | 35/50 [00:16<00:04,  3.32it/s]
 72%|███████▏  | 36/50 [00:16<00:04,  3.32it/s]
 74%|███████▍  | 37/50 [00:16<00:03,  3.31it/s]
 76%|███████▌  | 38/50 [00:16<00:03,  3.31it/s]
 78%|███████▊  | 39/50 [00:17<00:03,  3.31it/s]
 80%|████████  | 40/50 [00:17<00:03,  3.31it/s]
 82%|████████▏ | 41/50 [00:17<00:02,  3.30it/s]
 84%|████████▍ | 42/50 [00:18<00:02,  3.31it/s]
 86%|████████▌ | 43/50 [00:18<00:02,  3.31it/s]
 88%|████████▊ | 44/50 [00:18<00:01,  3.31it/s]
 90%|█████████ | 45/50 [00:19<00:01,  3.31it/s]
 92%|█████████▏| 46/50 [00:19<00:01,  3.31it/s]
 94%|█████████▍| 47/50 [00:19<00:00,  3.32it/s]
 96%|█████████▌| 48/50 [00:19<00:00,  3.32it/s]
 98%|█████████▊| 49/50 [00:20<00:00,  3.32it/s]
100%|██████████| 50/50 [00:20<00:00,  3.32it/s]
100%|██████████| 50/50 [00:20<00:00,  2.43it/s]

This output was created using a different version of the model, lucataco/style-aligned:a9086389.

Examples

View more examples

Run time and cost

This model costs approximately $0.069 to run on Replicate, or 14 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 50 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Implementation of https://github.com/google/style-aligned/ for SDXL

Changelog

12/11/23 - Added input images for style transfer

Project Github Page

@article{hertz2023StyleAligned,
  title={Style Aligned Image Generation via Shared Attention},
  author={Hertz, Amir and Voynov, Andrey and Fruchter, Shlomi and Cohen-Or, Daniel},
  booktitle={arXiv preprint arxiv:2312.02133},
  year={2023}
}