bytedance / sdxl-lightning-4step

SDXL-Lightning by ByteDance: a fast text-to-image model that makes high-quality images in 4 steps

  • Public
  • 596.6M runs
  • GitHub
  • Weights
  • Paper
  • License

Run time and cost

This model costs approximately $0.0014 to run on Replicate, or 714 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 2 seconds.

Readme

SDXL-Lightning

Intro Image

SDXL-Lightning is a lightning-fast text-to-image generation model. It can generate high-quality 1024px images in a few steps. For more information, please refer to our research paper: SDXL-Lightning: Progressive Adversarial Diffusion Distillation. We open-source the model as part of the research.

Our models are distilled from stabilityai/stable-diffusion-xl-base-1.0. This repository contains checkpoints for 1-step, 2-step, 4-step, and 8-step distilled models. The generation quality of our 2-step, 4-step, and 8-step model is amazing. Our 1-step model is more experimental.

We provide both full UNet and LoRA checkpoints. The full UNet models have the best quality while the LoRA models can be applied to other base models.

Demos

The community has made some amazing third-party demos based on SDXL-Lightning. Here we feature some of the best:

  • Generate with all configurations, best quality: Demo
  • Real-time generation as you type, lightning-fast: Demo1, Demo2
  • Comparison to other models: Link

Diffusers Usage

Please always use the correct checkpoint for the corresponding inference steps.

2-Step, 4-Step, 8-Step UNet

2-Step, 4-Step, 8-Step LoRA

1-Step UNet

The 1-step model is only experimental and the quality is much less stable. Consider using the 2-step model for much better quality.

The 1-step model uses “sample” prediction instead of “epsilon” prediction! The scheduler needs to be configured correctly.

ComfyUI Usage

Please always use the correct checkpoint for the corresponding inference steps. Please use Euler sampler with sgm_uniform scheduler.

2-Step, 4-Step, 8-Step UNet

  1. Download the full checkpoint (sdxl_lightning_Nstep.safetensors) to /ComfyUI/models/checkpoints.
  2. Download our ComfyUI full workflow.

SDXL-Lightning ComfyUI Full Workflow

2-Step, 4-Step, 8-Step LoRA

  1. Prepare your own base model.
  2. Download the LoRA checkpoint (sdxl_lightning_Nstep_lora.safetensors) to /ComfyUI/models/loras
  3. Download our ComfyUI LoRA workflow.

SDXL-Lightning ComfyUI LoRA Workflow

1-Step UNet

The 1-step model is only experimental and the quality is much less stable. Consider using the 2-step model for much better quality.

  1. Update your ComfyUI to the latest version.
  2. Download the full checkpoint (sdxl_lightning_1step_x0.safetensors) to /ComfyUI/models/checkpoints.
  3. Download our ComfyUI full 1-step workflow.

SDXL-Lightning ComfyUI Full 1-Step Workflow

Cite Our Work

@misc{lin2024sdxllightning,
      title={SDXL-Lightning: Progressive Adversarial Diffusion Distillation}, 
      author={Shanchuan Lin and Anran Wang and Xiao Yang},
      year={2024},
      eprint={2402.13929},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}