zsxkib/animatediff-illusions:17b2ca47 | Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

zsxkib /animatediff-illusions:17b2ca47

Input

controlnet_video

file

A short video that will be used as the keyframes for QR Code Monster to use, Please note, all of the frames will be used as keyframes

enable_qr_code_monster_v2

boolean

Flag to enable QR Code Monster V2 ControlNet

Default: true

qr_code_monster_v2_preprocessor

boolean

Flag to pre-process keyframes for QR Code Monster V2 ControlNet

Default: true

qr_code_monster_v2_guess_mode

boolean

Flag to enable guess mode (un-guided) for QR Code Monster V2 ControlNet

Default: false

controlnet_conditioning_scale

number

TODO

Default: 0.18

head_prompt

string

Shift + Return to add a new line

masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy skymasterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky

Primary animation prompt. If a prompt map is provided, this will be prefixed at the start of every individual prompt in the map

Default: "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky"

prompt_map

string

Shift + Return to add a new line

0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off

Prompt for changes in animation. Provide 'frame number : prompt at this frame', separate different prompts with '|'. Make sure the frame number does not exceed the length of video (frames)

Default: "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off"

tail_prompt

string

Shift + Return to add a new line

dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmospheredark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere

Additional prompt that will be appended at the end of the main prompt or individual prompts in the map

Default: "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere"

negative_prompt

string

Shift + Return to add a new line

(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island

Default: "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island"

frames

integer

(minimum: 1, maximum: 1024)

Length of the video in frames (playback is at 8 fps e.g. 16 frames @ 8 fps is 2 seconds)

Default: 128

width

integer

(minimum: 64, maximum: 2160)

Width of generated video in pixels, must be divisable by 8

Default: 256

height

integer

(minimum: 64, maximum: 2160)

Height of generated video in pixels, must be divisable by 8

Default: 384

base_model

string

Choose the base model for animation generation. If 'CUSTOM' is selected, provide a custom model URL in the next parameter

Default: "majicmixRealistic_v5Preview"

custom_base_model_url

string

Shift + Return to add a new line

Only used when base model is set to 'CUSTOM'. URL of the custom model to download if 'CUSTOM' is selected in the base model. Only downloads from 'https://civitai.com/api/download/models/' are allowed

Default: ""

prompt_fixed_ratio

number

(minimum: 0, maximum: 1)

Defines the ratio of adherence to the fixed part of the prompt versus the dynamic part (from prompt map). Value should be between 0 (only dynamic) to 1 (only fixed).

Default: 0.5

scheduler

string

Diffusion scheduler

Default: "k_dpmpp_sde"

steps

integer

(minimum: 1, maximum: 100)

Number of inference steps

Default: 25

guidance_scale

number

(minimum: 0, maximum: 20)

Guidance Scale. How closely do we want to adhere to the prompt and its contents

Default: 7.5

clip_skip

integer

(minimum: 1, maximum: 6)

Skip the last N-1 layers of the CLIP text encoder (lower values follow prompt more closely)

Default: 2

context

integer

(minimum: 1, maximum: 32)

Number of frames to condition on (default: max of <length> or 32). max for motion module v1 is 24

Default: 16

output_format

string

Output format of the video. Can be 'mp4' or 'gif'

Default: "mp4"

playback_frames_per_second

integer

(minimum: 1, maximum: 60)

Default: 8

seed

integer

Seed for different images and reproducibility. Use -1 to randomise seed

Default: -1

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/animatediff-illusions

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/animatediff-illusions using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/animatediff-illusions:17b2ca472f9881a8336bfc9d6dcf7d3e0924b1592633faa7ce75d5cada8d1af5",
  {
    input: {
      seed: -1,
      steps: 25,
      width: 256,
      frames: 128,
      height: 384,
      context: 16,
      clip_skip: 2,
      scheduler: "k_dpmpp_sde",
      base_model: "majicmixRealistic_v5Preview",
      prompt_map: "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off",
      head_prompt: "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky",
      tail_prompt: "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere",
      output_format: "mp4",
      guidance_scale: 7.5,
      negative_prompt: "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island",
      prompt_fixed_ratio: 0.5,
      custom_base_model_url: "",
      enable_qr_code_monster_v2: true,
      playback_frames_per_second: 8,
      controlnet_conditioning_scale: 0.18,
      qr_code_monster_v2_guess_mode: false,
      qr_code_monster_v2_preprocessor: true
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/animatediff-illusions using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/animatediff-illusions:17b2ca472f9881a8336bfc9d6dcf7d3e0924b1592633faa7ce75d5cada8d1af5",
    input={
        "seed": -1,
        "steps": 25,
        "width": 256,
        "frames": 128,
        "height": 384,
        "context": 16,
        "clip_skip": 2,
        "scheduler": "k_dpmpp_sde",
        "base_model": "majicmixRealistic_v5Preview",
        "prompt_map": "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off",
        "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky",
        "tail_prompt": "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere",
        "output_format": "mp4",
        "guidance_scale": 7.5,
        "negative_prompt": "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island",
        "prompt_fixed_ratio": 0.5,
        "custom_base_model_url": "",
        "enable_qr_code_monster_v2": True,
        "playback_frames_per_second": 8,
        "controlnet_conditioning_scale": 0.18,
        "qr_code_monster_v2_guess_mode": False,
        "qr_code_monster_v2_preprocessor": True
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/animatediff-illusions using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zsxkib/animatediff-illusions:17b2ca472f9881a8336bfc9d6dcf7d3e0924b1592633faa7ce75d5cada8d1af5",
    "input": {
      "seed": -1,
      "steps": 25,
      "width": 256,
      "frames": 128,
      "height": 384,
      "context": 16,
      "clip_skip": 2,
      "scheduler": "k_dpmpp_sde",
      "base_model": "majicmixRealistic_v5Preview",
      "prompt_map": "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off",
      "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky",
      "tail_prompt": "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship\'s lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere",
      "output_format": "mp4",
      "guidance_scale": 7.5,
      "negative_prompt": "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island",
      "prompt_fixed_ratio": 0.5,
      "custom_base_model_url": "",
      "enable_qr_code_monster_v2": true,
      "playback_frames_per_second": 8,
      "controlnet_conditioning_scale": 0.18,
      "qr_code_monster_v2_guess_mode": false,
      "qr_code_monster_v2_preprocessor": true
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.