zsxkib / animatediff-illusions

Monster Labs' Controlnet QR Code Monster v2 For SD-1.5 on top of AnimateDiff Prompt Travel (Motion Module SD 1.5 v2)

  • Public
  • 10.2K runs
  • L40S
  • GitHub
  • Paper
  • License
Run with an API
Playground API Examples README Versions

Input

controlnet_video
file

A short video/gif that will be used as the keyframes for QR Code Monster to use, Please note, all of the frames will be used as keyframes

boolean

Flag to enable QR Code Monster V2 ControlNet

Default: true

boolean

Flag to pre-process keyframes for QR Code Monster V2 ControlNet

Default: true

boolean

Flag to enable guess mode (un-guided) for QR Code Monster V2 ControlNet

Default: false

number

Strength of ControlNet. The outputs of the ControlNet are multiplied by `controlnet_conditioning_scale` before they are added to the residual in the original UNet

Default: 0.18

boolean

Flag to loop the video. Use when you have an 'infinitely' repeating video/gif ControlNet video

Default: true

string
Shift + Return to add a new line

Primary animation prompt. If a prompt map is provided, this will be prefixed at the start of every individual prompt in the map

Default: "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky"

string
Shift + Return to add a new line

Prompt for changes in animation. Provide 'frame number : prompt at this frame', separate different prompts with '|'. Make sure the frame number does not exceed the length of video (frames)

Default: ""

string
Shift + Return to add a new line

Additional prompt that will be appended at the end of the main prompt or individual prompts in the map

Default: ""

string
Shift + Return to add a new line

Default: ""

integer
(minimum: 1, maximum: 1024)

Length of the video in frames (playback is at 8 fps e.g. 16 frames @ 8 fps is 2 seconds)

Default: 128

integer
(minimum: 64, maximum: 2160)

Width of generated video in pixels, must be divisable by 8

Default: 256

integer
(minimum: 64, maximum: 2160)

Height of generated video in pixels, must be divisable by 8

Default: 384

string

Choose the base model for animation generation. If 'CUSTOM' is selected, provide a custom model URL in the next parameter

Default: "majicmixRealistic_v5Preview"

string
Shift + Return to add a new line

Only used when base model is set to 'CUSTOM'. URL of the custom model to download if 'CUSTOM' is selected in the base model. Only downloads from 'https://civitai.com/api/download/models/' are allowed

Default: ""

number
(minimum: 0, maximum: 1)

Defines the ratio of adherence to the fixed part of the prompt versus the dynamic part (from prompt map). Value should be between 0 (only dynamic) to 1 (only fixed).

Default: 0.5

string

Diffusion scheduler

Default: "k_dpmpp_sde"

integer
(minimum: 1, maximum: 100)

Number of inference steps

Default: 25

number
(minimum: 0, maximum: 20)

Guidance Scale. How closely do we want to adhere to the prompt and its contents

Default: 7.5

integer
(minimum: 1, maximum: 6)

Skip the last N-1 layers of the CLIP text encoder (lower values follow prompt more closely)

Default: 2

integer
(minimum: 1, maximum: 32)

Number of frames to condition on (default: max of <length> or 32). max for motion module v1 is 24

Default: 16

string

Output format of the video. Can be 'mp4' or 'gif'

Default: "mp4"

integer
(minimum: 1, maximum: 60)

Default: 8

boolean

Whether to use FILM for between-frame interpolation (film-net.github.io)

Default: true

integer
(minimum: 1, maximum: 50)

Number of steps to interpolate between animation frames

Default: 3

integer

Seed for different images and reproducibility. Leave blank to randomise seed

Add a payment method to run this model.

Each run costs approximately $0.12. Alternatively, try out our featured models for free.

Sign in with GitHub

By signing in, you agree to our
terms of service and privacy policy

Output

Generated in
Tweak itReport

Examples

View more

Run time and cost

This model costs approximately $0.12 to run on Replicate, or 8 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 125 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.