zsxkib/animatediff-illusions:b3ccb010 | Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

zsxkib /animatediff-illusions:b3ccb010

Input

controlnet_video

file

A short video/gif that will be used as the keyframes for QR Code Monster to use, Please note, all of the frames will be used as keyframes

enable_qr_code_monster_v2

boolean

Flag to enable QR Code Monster V2 ControlNet

Default: true

qr_code_monster_v2_preprocessor

boolean

Flag to pre-process keyframes for QR Code Monster V2 ControlNet

Default: true

qr_code_monster_v2_guess_mode

boolean

Flag to enable guess mode (un-guided) for QR Code Monster V2 ControlNet

Default: false

controlnet_conditioning_scale

number

Strength of ControlNet. The outputs of the ControlNet are multiplied by `controlnet_conditioning_scale` before they are added to the residual in the original UNet

Default: 0.18

loop

boolean

Flag to loop the video. Use when you have an 'infinitely' repeating video/gif ControlNet video

Default: true

head_prompt

string

Shift + Return to add a new line

masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy skymasterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky

Primary animation prompt. If a prompt map is provided, this will be prefixed at the start of every individual prompt in the map

Default: "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky"

prompt_map

string

Shift + Return to add a new line

Prompt for changes in animation. Provide 'frame number : prompt at this frame', separate different prompts with '|'. Make sure the frame number does not exceed the length of video (frames)

Default: ""

tail_prompt

string

Shift + Return to add a new line

Additional prompt that will be appended at the end of the main prompt or individual prompts in the map

Default: ""

negative_prompt

string

Shift + Return to add a new line

Default: ""

frames

integer

(minimum: 1, maximum: 1024)

Length of the video in frames (playback is at 8 fps e.g. 16 frames @ 8 fps is 2 seconds)

Default: 128

width

integer

(minimum: 64, maximum: 2160)

Width of generated video in pixels, must be divisable by 8

Default: 256

height

integer

(minimum: 64, maximum: 2160)

Height of generated video in pixels, must be divisable by 8

Default: 384

base_model

string

Choose the base model for animation generation. If 'CUSTOM' is selected, provide a custom model URL in the next parameter

Default: "majicmixRealistic_v5Preview"

custom_base_model_url

string

Shift + Return to add a new line

Only used when base model is set to 'CUSTOM'. URL of the custom model to download if 'CUSTOM' is selected in the base model. Only downloads from 'https://civitai.com/api/download/models/' are allowed

Default: ""

prompt_fixed_ratio

number

(minimum: 0, maximum: 1)

Defines the ratio of adherence to the fixed part of the prompt versus the dynamic part (from prompt map). Value should be between 0 (only dynamic) to 1 (only fixed).

Default: 0.5

scheduler

string

Diffusion scheduler

Default: "k_dpmpp_sde"

steps

integer

(minimum: 1, maximum: 100)

Number of inference steps

Default: 25

guidance_scale

number

(minimum: 0, maximum: 20)

Guidance Scale. How closely do we want to adhere to the prompt and its contents

Default: 7.5

clip_skip

integer

(minimum: 1, maximum: 6)

Skip the last N-1 layers of the CLIP text encoder (lower values follow prompt more closely)

Default: 2

context

integer

(minimum: 1, maximum: 32)

Number of frames to condition on (default: max of <length> or 32). max for motion module v1 is 24

Default: 16

output_format

string

Output format of the video. Can be 'mp4' or 'gif'

Default: "mp4"

playback_frames_per_second

integer

(minimum: 1, maximum: 60)

Default: 8

film_interpolation

boolean

Whether to use FILM for between-frame interpolation (film-net.github.io)

Default: true

num_interpolation_steps

integer

(minimum: 1, maximum: 50)

Number of steps to interpolate between animation frames

Default: 3

seed

integer

Seed for different images and reproducibility. Leave blank to randomise seed

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/animatediff-illusions

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/animatediff-illusions using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/animatediff-illusions:b3ccb0101402aafd04bfea042950be606223e2abedbad93cf848bfffa072bb61",
  {
    input: {
      loop: true,
      steps: 25,
      width: 256,
      frames: 128,
      height: 384,
      context: 16,
      clip_skip: 2,
      scheduler: "k_dpmpp_sde",
      base_model: "majicmixRealistic_v5Preview",
      prompt_map: "",
      head_prompt: "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky",
      tail_prompt: "",
      output_format: "mp4",
      guidance_scale: 7.5,
      negative_prompt: "",
      film_interpolation: true,
      prompt_fixed_ratio: 0.5,
      custom_base_model_url: "",
      num_interpolation_steps: 3,
      enable_qr_code_monster_v2: true,
      playback_frames_per_second: 8,
      controlnet_conditioning_scale: 0.18,
      qr_code_monster_v2_guess_mode: false,
      qr_code_monster_v2_preprocessor: true
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/animatediff-illusions using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/animatediff-illusions:b3ccb0101402aafd04bfea042950be606223e2abedbad93cf848bfffa072bb61",
    input={
        "loop": True,
        "steps": 25,
        "width": 256,
        "frames": 128,
        "height": 384,
        "context": 16,
        "clip_skip": 2,
        "scheduler": "k_dpmpp_sde",
        "base_model": "majicmixRealistic_v5Preview",
        "prompt_map": "",
        "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky",
        "tail_prompt": "",
        "output_format": "mp4",
        "guidance_scale": 7.5,
        "negative_prompt": "",
        "film_interpolation": True,
        "prompt_fixed_ratio": 0.5,
        "custom_base_model_url": "",
        "num_interpolation_steps": 3,
        "enable_qr_code_monster_v2": True,
        "playback_frames_per_second": 8,
        "controlnet_conditioning_scale": 0.18,
        "qr_code_monster_v2_guess_mode": False,
        "qr_code_monster_v2_preprocessor": True
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/animatediff-illusions using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zsxkib/animatediff-illusions:b3ccb0101402aafd04bfea042950be606223e2abedbad93cf848bfffa072bb61",
    "input": {
      "loop": true,
      "steps": 25,
      "width": 256,
      "frames": 128,
      "height": 384,
      "context": 16,
      "clip_skip": 2,
      "scheduler": "k_dpmpp_sde",
      "base_model": "majicmixRealistic_v5Preview",
      "prompt_map": "",
      "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky",
      "tail_prompt": "",
      "output_format": "mp4",
      "guidance_scale": 7.5,
      "negative_prompt": "",
      "film_interpolation": true,
      "prompt_fixed_ratio": 0.5,
      "custom_base_model_url": "",
      "num_interpolation_steps": 3,
      "enable_qr_code_monster_v2": true,
      "playback_frames_per_second": 8,
      "controlnet_conditioning_scale": 0.18,
      "qr_code_monster_v2_guess_mode": false,
      "qr_code_monster_v2_preprocessor": true
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.