lucataco / demofusion

DemoFusion: Democratising High-Resolution Image Generation With No 💰

  • Public
  • 8.1K runs
  • GitHub
  • Paper

Readme

This is an implementation of DemoFusion. DemoFusion provides highres txt2img capabilities based on SDXL. See the demo example that showcases a txt to img run that provides an img in 1024x1024, 2048x2048, and 3076x3076 resolution in under 6min

Abstract

High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls. This paper aims to democratise high-resolution GenAI by advancing the frontier of high-resolution generation while remaining accessible to a broad audience. We demonstrate that existing Latent Diffusion Models (LDMs) possess untapped potential for higher-resolution image generation. Our novel DemoFusion framework seamlessly extends open-source GenAI models, employing Progressive Upscaling, Skip Residual, and Dilated Sampling mechanisms to achieve higher-resolution image generation. The progressive nature of DemoFusion requires more passes, but the intermediate results can serve as “previews”, facilitating rapid prompt iteration.

@article{du2023demofusion,
    title={DemoFusion: Democratising High-Resolution Image Generation With No $$$},
    author={Ruoyi Du and Dongliang Chang and Timothy M. Hospedales and Yi-Zhe Song and Zhanyu Ma},
    journal={arXiv},
    year={2023}
}