Run SDXL with an API

SDXL 1.0 is a new text-to-image model by Stability AI. Stable Diffusion XL lets you create better, bigger pictures, with faces that look more real. You can add clear, readable words to your images and make great-looking art with just short prompts.

Like Stable Diffusion 1.5 and 2.1, SDXL is open source. You can modify it, build things with it and use it commercially.

Replicate lets you run generative AI models, like SDXL, from your own code, without having to set up any infrastructure.

What you can do

You can use the SDXL model on Replicate to:

make images from your prompts
make an image from another image (img2img)
inpaint using a mask
use a refiner to add fine-detail to your images

View examples

Use a client library for Replicate

We maintain official Python, Node.js, Swift, and Go clients for Replicate. And the community has made more.

You need to sign up for Replicate, then you can find your API token on your account page.

Then, you can run SDXL with one line of code:

import Replicate from "replicate";
 
const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});
 
const output = await replicate.run(
  "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
  {
    input: {
      prompt: "An astronaut riding a rainbow unicorn"
    }
  }
);

You can use the API for free for a bit, but eventually we’ll ask you to enter your credit card. We only charge you by the second for the amount of time your requests are running, so it works out much cheaper than running your own GPUs.

Refiner

You can use a refiner to add fine detail to images.

The refiner is a new model released with SDXL, it was trained differently and is especially good at adding detail to your images. It functions alongside the base model, correcting discrepancies and enhancing your picture’s overall quality.

You can use the refiner in two ways:

As an ‘ensemble of experts’
As an extra step after the base model has finished

In this example, using the ‘ensemble of experts’ mode, the SDXL base model will handle the first 80% (0.8) of the generation, before handing off to the refiner to add fine detail for the last 20%:

const output = await replicate.run(
  "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
  {
    input: {
      prompt: 'A studio portrait photo of a cat',
      negative_prompt: 'ugly, soft, blurry, out of focus, low quality, garish, distorted, disfigured',
      width: 1024,
      height: 1024,
      num_inference_steps: 50,
      scheduler: 'DDIM',
      guidance_scale: 7.5,
      refine: 'expert_ensemble_refiner',
      high_noise_frac: '0.8'
    }
  }
);

You can alternatively run the refiner model after the base model using the ‘base image refiner’ mode.

In this example the base model will run for 50 steps and produce an output. This will be passed into the refiner which will run for a further 20 steps:

const output = await replicate.run(
  "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
  {
    input: {
      prompt: 'A studio portrait photo of a cat',
      negative_prompt: 'ugly, soft, blurry, out of focus, low quality, garish, distorted, disfigured',
      width: 1024,
      height: 1024,
      num_inference_steps: 50,
      scheduler: 'DDIM',
      guidance_scale: 7.5,
      refine: 'base_image_refiner',
      refine_steps: '20'
    }
  }
);

For comparison:

Left: Base model without a refiner
Middle: Base model plus refiner
Right: Ensemble of experts refiner

A comparison of cat images using the SDXL refiner

Image to image

You can pass an image to the model to create a new image based on it. Change the prompt_strength to alter how much of the original image is kept. 0.6 to 0.8 is a good starting range.

View image to image example

const output = await replicate.run(
  "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
  {
    input: {
      prompt: 'A rainbow colored tiger',
      negative_prompt: 'ugly, soft, blurry, out of focus, low quality, garish, distorted, disfigured',
      image: 'https://replicate.delivery/pbxt/JF3foGR90vm9BXSEXNaYkaeVKHYbJPinmpbMFvRtlDpH4MMk/out-0-1.png',
      prompt_strength: 0.8,
      width: 1024,
      height: 1024,
      num_inference_steps: 50,
      scheduler: 'DDIM',
      guidance_scale: 7.5,
      refine: 'base_image_refiner',
      refine_steps: '20'
    }
  }
);

A rainbow colored tiger

Inpainting

You can include a mask with your prompt and image to control which parts of your input image will be updated. Use a black and white mask, where black pixels are kept and white ones will be updated.

View inpainting example

const output = await replicate.run(
  "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
  {
    input: {
      prompt: 'A rainbow colored bear',
      negative_prompt: 'ugly, soft, blurry, out of focus, low quality, garish, distorted, disfigured',
      image: 'https://replicate.delivery/pbxt/JF3foGR90vm9BXSEXNaYkaeVKHYbJPinmpbMFvRtlDpH4MMk/out-0-1.png',
      mask: 'https://replicate.delivery/pbxt/JFIZFfJsSnWgxbTmEYLqhHIGdZo9o2BX3p47wSdn55HWtMON/mask1.png',
      prompt_strength: 0.95,
      width: 1024,
      height: 1024,
      num_inference_steps: 50,
      scheduler: 'DDIM',
      guidance_scale: 7.5,
      refine: 'base_image_refiner',
      refine_steps: '20'
    }
  }
);

A rainbow colored bear

Keep in touch

Join us in Discord to show us what you’ve made, or if you need any help.

Happy hacking! ✨