zsxkib/animate-diff-scene-assembler:d8d16bc7 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

zsxkib /animate-diff-scene-assembler:d8d16bc7

Input

prompt

string

Shift + Return to add a new line

0: a big (chrome:1.1) cyborg robot dance, (white science lab:1.1) | 100 :a (chrome:1.1) cyborg robot dance, (red led blinksp:1.1), (white science lab:1.1)0: a big (chrome:1.1) cyborg robot dance, (white science lab:1.1) | 100 :a (chrome:1.1) cyborg robot dance, (red led blinksp:1.1), (white science lab:1.1)

Prompt for changes in animation. Provide 'frame number : prompt at this frame', separate different prompts with '|'. Make sure the frame number does not exceed the length of video (frames)

Default: "0: a big (chrome:1.1) cyborg robot dance, (white science lab:1.1) | 100 :a (chrome:1.1) cyborg robot dance, (red led blinksp:1.1), (white science lab:1.1)"

negative_prompt

string

Shift + Return to add a new line

embedding:easynegative, (worst quality, low quality: 1.3), zombie, horror, distorted, photo, nfswembedding:easynegative, (worst quality, low quality: 1.3), zombie, horror, distorted, photo, nfsw

Input Negative Prompt (constant)

Default: "embedding:easynegative, (worst quality, low quality: 1.3), zombie, horror, distorted, photo, nfsw"

first_num_inference_steps

integer

(minimum: 1, maximum: 50)

Number of denoising steps

Default: 15

first_guidance_scale

number

(minimum: 1, maximum: 50)

Scale for classifier-free guidance

Default: 7.5

second_num_inference_steps

integer

(minimum: 1, maximum: 50)

Second pass, for video interoplation. Number of denoising steps

Default: 15

second_guidance_scale

number

(minimum: 1, maximum: 50)

Second pass, for video interoplation. Scale for classifier-free guidance

Default: 8.5

pose_video

file

Video of a subject doing something fun or interesting, this video will be used to track any people and their pose estimations for the final video

subject_image

file

An image of what the subject should look like in the final video

subject_controlnet_strength

number

(minimum: 0, maximum: 1)

Strength of the controlnet for the subject/character image

Default: 0.2

background_image

file

An image of what the background should look like in the final video

background_controlnet_strength

number

(minimum: 0, maximum: 1)

Strength of the controlnet for the background image

Default: 0.4

return_temp_files

boolean

Return any temporary files, such as preprocessed controlnet images. Useful for debugging.

Default: false

randomise_seeds

boolean

Automatically randomise seeds (seed, noise_seed, rand_seed)

Default: true

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/animate-diff-scene-assembler

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/animate-diff-scene-assembler using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/animate-diff-scene-assembler:d8d16bc71ca0e327957e513a2fedb035294dd2ec1fa183d4f60394f01a1265bb",
  {
    input: {
      prompt: "0: a big (chrome:1.1) cyborg robot dance, (white science lab:1.1) | 100 :a (chrome:1.1) cyborg robot dance, (red led blinksp:1.1), (white science lab:1.1)",
      negative_prompt: "embedding:easynegative, (worst quality, low quality: 1.3), zombie, horror, distorted, photo, nfsw",
      randomise_seeds: true,
      return_temp_files: false,
      first_guidance_scale: 7.5,
      second_guidance_scale: 8.5,
      first_num_inference_steps: 15,
      second_num_inference_steps: 15,
      subject_controlnet_strength: 0.2,
      background_controlnet_strength: 0.4
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/animate-diff-scene-assembler using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/animate-diff-scene-assembler:d8d16bc71ca0e327957e513a2fedb035294dd2ec1fa183d4f60394f01a1265bb",
    input={
        "prompt": "0: a big (chrome:1.1) cyborg robot dance, (white science lab:1.1) | 100 :a (chrome:1.1) cyborg robot dance, (red led blinksp:1.1), (white science lab:1.1)",
        "negative_prompt": "embedding:easynegative, (worst quality, low quality: 1.3), zombie, horror, distorted, photo, nfsw",
        "randomise_seeds": True,
        "return_temp_files": False,
        "first_guidance_scale": 7.5,
        "second_guidance_scale": 8.5,
        "first_num_inference_steps": 15,
        "second_num_inference_steps": 15,
        "subject_controlnet_strength": 0.2,
        "background_controlnet_strength": 0.4
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/animate-diff-scene-assembler using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zsxkib/animate-diff-scene-assembler:d8d16bc71ca0e327957e513a2fedb035294dd2ec1fa183d4f60394f01a1265bb",
    "input": {
      "prompt": "0: a big (chrome:1.1) cyborg robot dance, (white science lab:1.1) | 100 :a (chrome:1.1) cyborg robot dance, (red led blinksp:1.1), (white science lab:1.1)",
      "negative_prompt": "embedding:easynegative, (worst quality, low quality: 1.3), zombie, horror, distorted, photo, nfsw",
      "randomise_seeds": true,
      "return_temp_files": false,
      "first_guidance_scale": 7.5,
      "second_guidance_scale": 8.5,
      "first_num_inference_steps": 15,
      "second_num_inference_steps": 15,
      "subject_controlnet_strength": 0.2,
      "background_controlnet_strength": 0.4
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.