xlabs-ai / flux-controlnet

XLabs-AI ControlNet Canny for FLUX.1-dev

  • Public
  • 1.4K runs
  • GitHub
  • License
Run with an API
Playground API Examples README Versions

Input

Output

Examples

View more

Run time and cost

This model costs approximately $0.21 to run on Replicate, or 4 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 3 minutes. The predict time for this model varies significantly based on the inputs.

Readme

ControlNet Canny for Flux

This repository provides a checkpoint with trained ControlNet Canny model for FLUX.1-dev model by Black Forest Labs

Training details

XLabs AI team is happy to publish fune-tuning Flux scripts, including: - LoRA 🔥 - ControlNet 🔥

See our github for train script and train configs.

Training dataset

Dataset has the following format for the training process:

├── images/
│    ├── 1.png
│    ├── 1.json
│    ├── 2.png
│    ├── 2.json
│    ├── ...

A .json file contains “caption” field with a text prompt.

Inference

To test our checkpoints, use commands presented below.

python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a handsome viking man with white hair, cinematic, MM full HD"

Example Picture 1

python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a dark evil mysterius house with ghosts, cinematic, MM full HD"

Example Picture 2

License

controlnet.safetensors falls under the FLUX.1 [dev] Non-Commercial License