paragekbote/flux-fast-lora-hotswap

A blazing-fast inference setup for Flux.1-dev with dynamic LoRA hotswapping.

Public
31 runs
Run with an API
Pricing
Weights
Playground API Examples README Versions

Examples

View more

Run time and cost

This model runs on Nvidia L40S GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

FLUX.1-dev LoRA Hotswap

This setup uses torch.compile, BitsAndBytes and PEFT LoRA to package the Flux.1 [dev] model, enabling fast text-to-image generation with LoRA adapter hotswapping via trigger words, as well as quantization.

It has also been featured in the HF blog from the work it builds upon.

Features

  • Optimized Performance: torch.compile acceleration for maximum speed.
  • Dynamic LoRA Switching: Seamlessly swap between different art styles using trigger words.
  • Memory Efficient: BitsAndBytes quantization reduces VRAM usage.
  • Multi-Style Generation: Two powerful LoRAs ready to use out of the box.

Available Styles

Enhanced Image Preferences

  • Trigger: ["Cinematic", "Photographic", "Anime", "Manga", "Digital art", "Pixel art", "Fantasy art", "Neonpunk", "3D Model", “Painting”, “Animation” “Illustration”]
  • LoRA: data-is-better-together/open-image-preferences-v1-flux-dev-lora
  • Description: Refined image generation based on curated preference data

Ghibsky Illustration

Model Details

  • Base model: black-forest-labs/FLUX.1-dev
  • Optimization: PyTorch 2.0 compilation + BitsAndBytes quantization with LoRA Hot-swapping.
  • Memory Usage: Significantly reduced through quantization.

Performance

  • Speed: Up to 2x faster generation thanks to torch.compile.
  • Memory: ~40% reduction in VRAM usage with quantization.
  • Quality: Maintains full FLUX.1-dev image quality.
  • Flexibility: Instant style switching without model reloading.

Usage Tips

  • Mix styles by combining trigger words for unique hybrid aesthetics.
  • Use detailed, descriptive prompts for best results.
  • Remember to enter the trigger word as well the prompt for best results.
  • Works optimally with standard FLUX.1-dev resolutions (1024x1024).
  • Quantization makes this suitable for consumer GPUs.

Note: GPU usage at the end of every image generation is also seen. Since LoRA hot-swapping is new, some errors could be observed. Feel free to reach out if any errors occur.

Use Cases

  • Creative workflows requiring multiple art styles.
  • Rapid prototyping of visual concepts.
  • Style exploration and comparison.