LoRA Pivotal Tuning Inversion Training

Model description

There are many methods to fine-tune Stable diffusion models. One can use Low-rank adaption with pivotal-tuning inversion to achieve high-editable, efficient fine-tuning. Output models can be used with Replicate’s LoRA for inference.

This model has preset hyperparameters for faces, objects, and styles. If you full control of all of the hyperparameters yourself, you can use https://replicate.com/cloneofsimo/lora-advanced-training.

Ethical considerations

Do not use this model to produce harmful results. As this method strictly utilizes stable-diffusion, ethical considerations that were addressed in BigScience OpenRAIL-M license should be addressed here as well.

Caveats and recommendations

• Use many, diverse, high quality dataset. Any blur, noises and artifacts will have negative effect to the training process. Having different lighting conditions, shapes, angles, and various sizes will help very much.

• Images will be resized and cropped to 512 x 512 by default, thus it is recommended to prepare datasets with larger than 512 x 512.

• Using face template requires all input images to have human face, and only one per image. For example, it will not work with animal faces, or highly unhuman-like character faces.