Train a Qwen Image LoRA with SimpleTuner

Teach Qwen Image a character, subject, or visual style using your own captioned images. This trainer produces a LoRA adapter that you load alongside the matching base model to generate images.

The model selector defaults to Qwen-Image-2.1. The original Qwen-Image is also available. Use a 2.1 adapter with Qwen/Qwen-Image-2.1, and an original-model adapter with Qwen/Qwen-Image.

Start a training run

1. Upload a captioned ZIP or tar file using train_data, or enter a Hugging Face dataset ID in hf_dataset. Provide exactly one source.
2. Keep model set to Qwen-Image-2.1 unless you want the original model.
3. Choose resolution: 512, 1024, or 512+1024.
4. Set validation_prompt to a prompt containing the trigger or subject from your captions. For example: a photograph of YOUR_TRIGGER.
5. Optionally publish your outputs: set hub_model_id and a writable hf_token, or configure s3_bucket and its credentials. You can use both or leave both blank to download the returned ZIP.
6. Start the prediction. The default run trains for 4,000 optimizer updates and saves checkpoints and validation images every 500 updates.

The trainer runs on one GPU with an 80 GB memory budget; Qwen-Image-2.1 has been tested on H100. Keep train_batch_size=1 for the supplied recipe. Selecting the original model changes its memory requirements; the 2.1 test does not establish its memory use.

Prepare your images and captions

Every image in an uploaded archive needs a matching UTF-8 .txt file with the same filename stem:

training-images.zip
├── subject-01.jpg
├── subject-01.txt
├── subject-02.png
└── subject-02.txt

For example, subject-01.txt could contain:

a photograph of YOUR_TRIGGER waving, against a plain white background

Use the same trigger consistently across captions for the subject you want to teach. Describe the content of each image; the trainer reads the text files rather than deriving captions from image filenames. Nested folders are supported. Common supported formats include JPEG, PNG, WebP, BMP, and TIFF. Convert AVIF images to a supported format before uploading.

For a Hugging Face dataset, use a repository with an image column and a train split. The caption column defaults to caption; change hf_caption_column if yours has a different name. Supply hf_token when the dataset requires authentication.

Cropping is disabled by default. Images are grouped by aspect ratio so portrait and landscape images can train together. To enable cropping, set enable_crop=true and choose crop_style (center or random) and crop_aspect (square or preserve). Dataset safety checks exclude samples classified as NSFW during cache preparation.

Choose resolution and training length

These values describe image area, not a requirement that every training image be square. Validation images are square at the selected resolutions.

max_train_steps counts total optimizer updates. For Qwen-Image-2.1, use_synthetic_regularisation=true by default: half the sampling probability goes to your images and half to synthetic regularisation images. In multi-scale mode, your 512px and 1024px datasets each receive approximately one quarter of the updates. A 4,000-update run therefore has roughly 1,000 concept updates at each scale, not 4,000 at each scale. Actual counts vary with probabilistic sampling.

Set use_synthetic_regularisation=false to train only on your images and skip downloading and caching the synthetic dataset. All updates then go to your data, split equally between the selected scales. You can reduce the total step count by roughly half for the same expected number of concept updates: for example, 2,000 updates without regularisation give approximately the same number of concept updates as 4,000 with it. This does not guarantee equivalent results. Disabling regularisation increases concept bleed, where the learned subject or style appears in unrelated prompts. Keep it enabled when preserving the base model’s behaviour matters. This option has no effect on original Qwen-Image, which already trains only on your images.

Training starts fresh for each prediction; the interface does not expose checkpoint resume. Inspect checkpoint samples and compare several prompts, including simple trigger prompts and new contexts. The interface generates one validation prompt at each selected resolution; that one prompt cannot establish whether a concept was learned reliably.

Save to Hugging Face

Set hub_model_id to a model repository such as YOUR_ACCOUNT/my-qwen-image-lora, and provide hf_token with write access to that account or organisation.

The final upload contains the LoRA weights, a model card, training configuration, and the latest validation images. Intermediate checkpoints are not uploaded to the Hub by this recipe. Configure S3 as well if you want checkpoint publishing during the run.

Save checkpoints to S3-compatible storage

Set s3_bucket, s3_access_key, and s3_secret_key. Add s3_region for your provider and s3_endpoint_url when using a service other than the default AWS endpoint.

• s3_base_path sets the prefix inside your bucket. By default, each run gets its own cog/qwen-image/{job_id} prefix.
• s3_public_base_url is optional and is used to print shareable output locations. It does not change bucket access permissions.
• Access keys can be omitted if the runtime already has an authorised IAM role.

Hugging Face and S3 credentials use secret inputs. Enter them in the corresponding fields rather than putting them in captions or dataset archives.

Outputs and current limitations

The prediction returns a downloadable ZIP containing the final LoRA, retained checkpoints, and validation images. VAE and text preparation caches are stored separately and excluded from the archive. Hugging Face and S3 publishing are optional.

Download and save the ZIP promptly. Replicate removes output files and other prediction data after an hour for API predictions; web-interface predictions are retained until deleted.

checkpoints_total_limit retains between one and five local checkpoints, default five. checkpoint_step_interval controls both checkpoint saving and validation frequency, default 500 updates. return_logs=true prints the end of the training debug log. Preparation includes model downloads, dataset caching, and compilation, so the first optimizer update can take longer to begin than later updates take to run.

The current REPA-enabled export can include training-only projector tensors. Diffusers rejects those tensors with an Invalid LoRA checkpoint error. If your export has this issue, keep the original for training state and make an inference copy with only those projector tensors removed:

from safetensors import safe_open
from safetensors.torch import load_file, save_file

source = "pytorch_lora_weights.safetensors"
with safe_open(source, framework="pt", device="cpu") as checkpoint:
    metadata = checkpoint.metadata()

weights = load_file(source)
weights = {
    name: tensor
    for name, tensor in weights.items()
    if not name.startswith("transformer.crepa_projector.")
}
save_file(weights, "inference_lora.safetensors", metadata=metadata)

This preserves the trained LoRA tensors. Load the inference copy with a pipeline or UI that supports your selected Qwen Image version, using its matching base model. Do not add the training assistant adapter for normal inference. It is used during training and disabled for the trainer’s validation images.

Recipe defaults

The Qwen-Image-2.1 recipe uses the frozen assistant v3, REPA feature alignment, resolution-dependent flow-schedule auto shift, and synthetic regularisation from the generated-image dataset. These settings are preconfigured; the interface does not expose learning-rate, LoRA-rank, or assistant controls.

SimpleTuner is pinned to v4.9.3. For training guidance and checkpoint comparisons, see the Qwen Image 2.1 quickstart and LoRA experiments.