Readme
FLUX.1-dev LoRA Hotswap
This setup uses torch.compile
, BitsAndBytes
and PEFT
LoRA to package the Flux.1 [dev] model, enabling fast text-to-image generation with LoRA adapter hotswapping via trigger words, as well as quantization.
Features
- Optimized Performance:
torch.compile
acceleration for maximum speed. - Dynamic LoRA Switching: Seamlessly swap between different art styles using trigger words.
- Memory Efficient: BitsAndBytes quantization reduces VRAM usage.
- Multi-Style Generation: Two powerful LoRAs ready to use out of the box.
Available Styles
Enhanced Image Preferences
- Trigger:
["Cinematic", "Photographic", "Anime", "Manga", "Digital art", "Pixel art", "Fantasy art", "Neonpunk", "3D Model", “Painting”, “Animation” “Illustration”]
- LoRA: data-is-better-together/open-image-preferences-v1-flux-dev-lora
- Description: Refined image generation based on curated preference data
Ghibsky Illustration
- Trigger:
[GHIBSKY]
- LoRA: aleksa-codes/flux-ghibsky-illustration
- Description: Studio Ghibli-inspired sky and landscape illustrations
Model Details
- Base model: black-forest-labs/FLUX.1-dev
- Optimization: PyTorch 2.0 compilation + BitsAndBytes quantization with LoRA Hot-swapping.
- Memory Usage: Significantly reduced through quantization.
Performance
- Speed: Up to 2x faster generation thanks to
torch.compile
. - Memory: ~40% reduction in VRAM usage with quantization.
- Quality: Maintains full FLUX.1-dev image quality.
- Flexibility: Instant style switching without model reloading.
Usage Tips
- Mix styles by combining trigger words for unique hybrid aesthetics.
- Use detailed, descriptive prompts for best results.
- Remember to enter the trigger word as well the prompt for best results.
- Works optimally with standard FLUX.1-dev resolutions (1024x1024).
- Quantization makes this suitable for consumer GPUs.
Note: GPU usage at the end of every image generation is also seen. Since LoRA hot-swapping is new, some errors could be observed. Feel free to reach out if any errors occur.
Use Cases
- Creative workflows requiring multiple art styles.
- Rapid prototyping of visual concepts.
- Style exploration and comparison.