wcarle / stable-diffusion-videos-openjourney

Generate videos by interpolating the latent space of Stable Diffusion using the Openjourney Model

  • Public
  • 5.6K runs
  • A100 (80GB)
  • GitHub
  • License

Input

string
Shift + Return to add a new line

Input prompts, separate each prompt with '|'.

Default: "a cat | a dog | a horse"

string
Shift + Return to add a new line

Random seed, separated with '|' to use different seeds for each of the prompt provided above. Leave blank to randomize the seed.

string

Choose the scheduler

Default: "klms"

integer
(minimum: 1, maximum: 500)

Number of denoising steps for each image generated from the prompt

Default: 50

number
(minimum: 1, maximum: 20)

Scale for classifier-free guidance

Default: 7.5

integer

Steps for generating the interpolation video. Recommended to set to 3 or 5 for testing, then up it to 60-200 for better results.

Default: 50

integer
(minimum: 5, maximum: 60)

Frame rate for the video.

Default: 15

Output

Generated in

Run time and cost

This model costs approximately $0.41 to run on Replicate, or 2 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 5 minutes. The predict time for this model varies significantly based on the inputs.

Readme

stable-diffusion-videos-openjourney

Based on nateraw’s project: https://replicate.com/nateraw/stable-diffusion-videos https://github.com/nateraw/stable-diffusion-videos

Swapped out the standard stable diffusion model with Openjourney: https://huggingface.co/prompthero/openjourney

Stable-diffusion-videos allows you to generate videos by interpolating the latent space of Stable Diffusion.

You can either dream up different versions of the same prompt, or morph between different text prompts (with seeds set for each for reproducibility).