Use Runway's Stable-diffusion inpainting model to create an infinite loop video. Inspired by

Run time and cost

Predictions run on Nvidia A100 GPU hardware. Predictions typically complete within 27 seconds. The predict time for this model varies significantly based on the inputs.

Stable Diffusion Infinite Zoom

Run it on Replicate:

This repo is based on Stable Diffusion by CompVis group:
and Stable Inpainting by Runway

The idea is based on this tweet by Matt Henderson

Model description

Given a prompt I run txt2img,py with sd-v1-4.ckpt
Then I paste a downscaled version of the image into it's center and inpaint around the center using using this sd-v1-5-inpainting.ckpt from
I repeat the inpainting step twice.

Then zoom in by upscaling the image and cuting it to the original size while pasting the "center" image in its due area.

How to run

Download text-2-image and inpainting weights

hf_hub_download(repo_id="runwayml/stable-diffusion-v1-5", filename="v1-5-pruned-emaonly.ckpt", cache_dir=".", use_auth_token=<HuggingFace token>)
hf_hub_download(repo_id="runwayml/stable-diffusion-inpainting", filename="sd-v1-5-inpainting.ckpt", cache_dir=".", use_auth_token=<HuggingFace token>)

create video

python3 scripts/ <your prompt>


      title={High-Resolution Image Synthesis with Latent Diffusion Models}, 
      author={Robin Rombach and Andreas Blattmann and Dominik Lorenz and Patrick Esser and Björn Ommer},