paragekbote/flux-fast-lora-hotswap

A blazing-fast inference setup for Flux.1-dev with dynamic LoRA hotswapping.

Public
31 runs
Run with an API
Pricing
Weights
Playground API Examples README Versions

FLUX.1-dev LoRA Hotswap

This setup uses torch.compile, BitsAndBytes and PEFT LoRA to package the Flux.1 [dev] model, enabling fast text-to-image generation with LoRA adapter hotswapping via trigger words, as well as quantization.

It has also been featured in the HF blog from the work it builds upon.

Features

  • Optimized Performance: torch.compile acceleration for maximum speed.
  • Dynamic LoRA Switching: Seamlessly swap between different art styles using trigger words.
  • Memory Efficient: BitsAndBytes quantization reduces VRAM usage.
  • Multi-Style Generation: Two powerful LoRAs ready to use out of the box.

Available Styles

Enhanced Image Preferences

  • Trigger: ["Cinematic", "Photographic", "Anime", "Manga", "Digital art", "Pixel art", "Fantasy art", "Neonpunk", "3D Model", “Painting”, “Animation” “Illustration”]
  • LoRA: data-is-better-together/open-image-preferences-v1-flux-dev-lora
  • Description: Refined image generation based on curated preference data

Ghibsky Illustration

Model Details

  • Base model: black-forest-labs/FLUX.1-dev
  • Optimization: PyTorch 2.0 compilation + BitsAndBytes quantization with LoRA Hot-swapping.
  • Memory Usage: Significantly reduced through quantization.

Performance

  • Speed: Up to 2x faster generation thanks to torch.compile.
  • Memory: ~40% reduction in VRAM usage with quantization.
  • Quality: Maintains full FLUX.1-dev image quality.
  • Flexibility: Instant style switching without model reloading.

Usage Tips

  • Mix styles by combining trigger words for unique hybrid aesthetics.
  • Use detailed, descriptive prompts for best results.
  • Remember to enter the trigger word as well the prompt for best results.
  • Works optimally with standard FLUX.1-dev resolutions (1024x1024).
  • Quantization makes this suitable for consumer GPUs.

Note: GPU usage at the end of every image generation is also seen. Since LoRA hot-swapping is new, some errors could be observed. Feel free to reach out if any errors occur.

Use Cases

  • Creative workflows requiring multiple art styles.
  • Rapid prototyping of visual concepts.
  • Style exploration and comparison.