nitrosocke / arcane-diffusion

Arcane on Stable Diffusion via Dreambooth

  • Public
  • 36.9K runs

Readme

Model weights: https://huggingface.co/nitrosocke/Arcane-Diffusion

Arcane Diffusion

This is the fine-tuned Stable Diffusion model trained on images from the TV Show Arcane. Use the tokens arcane style in your prompts for the effect.

img

Sample images from v3:

output Samples v3 output Samples v3

Sample images from the model:

output Samples

Sample images used for training:

Training Samples

Version 3 (arcane-diffusion-v3): This version uses the new train-text-encoder setting and improves the quality and edibility of the model immensely. Trained on 95 images from the show in 8000 steps.

Version 2 (arcane-diffusion-v2): This uses the diffusers based dreambooth training and prior-preservation loss is way more effective. The diffusers where then converted with a script to a ckpt file in order to work with automatics repo. Training was done with 5k steps for a direct comparison to v1 and results show that it needs more steps for a more prominent result. Version 3 will be tested with 11k steps.

Version 1 (arcane-diffusion-5k): This model was trained using Unfrozen Model Textual Inversion utilizing the Training with prior-preservation loss methods. There is still a slight shift towards the style, while not using the arcane token.