xlabs-ai / flux-controlnet

XLabs-AI ControlNet Canny for FLUX.1-dev

  • Public
  • 1.8K runs
  • GitHub
  • License

This model has no enabled versions.

Readme

ControlNet Canny for Flux

This repository provides a checkpoint with trained ControlNet Canny model for FLUX.1-dev model by Black Forest Labs

Training details

XLabs AI team is happy to publish fune-tuning Flux scripts, including: - LoRA πŸ”₯ - ControlNet πŸ”₯

See our github for train script and train configs.

Training dataset

Dataset has the following format for the training process:

β”œβ”€β”€ images/
β”‚    β”œβ”€β”€ 1.png
β”‚    β”œβ”€β”€ 1.json
β”‚    β”œβ”€β”€ 2.png
β”‚    β”œβ”€β”€ 2.json
β”‚    β”œβ”€β”€ ...

A .json file contains “caption” field with a text prompt.

Inference

To test our checkpoints, use commands presented below.

python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a handsome viking man with white hair, cinematic, MM full HD"

Example Picture 1

python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a dark evil mysterius house with ghosts, cinematic, MM full HD"

Example Picture 2

License

controlnet.safetensors falls under the FLUX.1 [dev] Non-Commercial License