hvision-nku/storydiffusion:d0faed1e – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

hvision-nku /storydiffusion:d0faed1e

Playground API

Input

sd_model

string

Choose a model

Default: "Unstable"

ref_image

file

Reference image for the character

character_description

string

Shift + Return to add a new line

General description of the character. If ref_image above is provided, making sure to follow the class word you want to customize with the trigger word 'img', such as: 'man img' or 'woman img' or 'girl img'

Default: "a man, wearing black suit"

negative_prompt

string

Shift + Return to add a new line

bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbsbad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs

Describe things you do not want to see in the output

Default: "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs"

comic_description

string

Shift + Return to add a new line

at home, read new paper #at home, The newspaper says there is a treasure house in the forest.
on the road, near the forest
[NC] The car on the road, near the forest #He drives to the forest in search of treasure.
[NC]A tiger appeared in the forest, at night 
very frightened, open mouth, in the forest, at night
running very fast, in the forest, at night
[NC] A house in the forest, at night #Suddenly, he discovers the treasure house!
in the house filled with  treasure, laughing, at night #He is overjoyed inside the house.at home, read new paper #at home, The newspaper says there is a treasure house in the forest.
on the road, near the forest
[NC] The car on the road, near the forest #He drives to the forest in search of treasure.
[NC]A tiger appeared in the forest, at night 
very frightened, open mouth, in the forest, at night
running very fast, in the forest, at night
[NC] A house in the forest, at night #Suddenly, he discovers the treasure house!
in the house filled with  treasure, laughing, at night #He is overjoyed inside the house.

Comic Description. Each frame is divided by a new line. Only the first 10 prompts are valid for demo speed! For comic_description NOT using ref_image: (1) Support Typesetting Style and Captioning. By default, the prompt is used as the caption for each image. If you need to change the caption, add a '#' at the end of each line. Only the part after the '#' will be added as a caption to the image. (2) The [NC] symbol is used as a flag to indicate that no characters should be present in the generated scene images. If you want do that, prepend the '[NC]' at the beginning of the line.

Default: "at home, read new paper #at home, The newspaper says there is a treasure house in the forest.\non the road, near the forest\n[NC] The car on the road, near the forest #He drives to the forest in search of treasure.\n[NC]A tiger appeared in the forest, at night \nvery frightened, open mouth, in the forest, at night\nrunning very fast, in the forest, at night\n[NC] A house in the forest, at night #Suddenly, he discovers the treasure house!\nin the house filled with treasure, laughing, at night #He is overjoyed inside the house."

style_name

string

Style template

Default: "Japanese Anime"

comic_style

string

Select the comic style for the combined comic

Default: "Classic Comic Style"

style_strength_ratio

integer

(minimum: 15, maximum: 50)

Style strength of Ref Image (%), only used if ref_image is provided

Default: 20

image_width

integer

Width of output image

Default: 768

image_height

integer

Height of output image

Default: 768

num_steps

integer

(minimum: 25, maximum: 50)

Number of sample steps

Default: 50

guidance_scale

number

(minimum: 0.1, maximum: 10)

Scale for classifier-free guidance

Default: 5

seed

integer

Random seed. Leave blank to randomize the seed

sa32_setting

number

(minimum: 0, maximum: 1)

The degree of Paired Attention at 32 x 32 self-attention layers

Default: 0.7

sa64_setting

number

(minimum: 0, maximum: 1)

The degree of Paired Attention at 64 x 64 self-attention layers

Default: 0.7

num_ids

integer

Number of id images in total images. This should not exceed total number of line-separated prompts

Default: 3

output_format

string

Format of the output images

Default: "webp"

output_quality

integer

(minimum: 0, maximum: 100)

Quality of the output images, from 0 to 100. 100 is best quality, 0 is lowest quality

Default: 80

Run this model in Node.js with one line of code:

npx create-replicate --model=hvision-nku/storydiffusion

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run hvision-nku/storydiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "hvision-nku/storydiffusion:d0faed1e8b5779d4d060c5bb0afac8e52c704700f7bee9e4341b0684a8feda04",
  {
    input: {
      num_ids: 3,
      sd_model: "Unstable",
      num_steps: 50,
      style_name: "Japanese Anime",
      comic_style: "Classic Comic Style",
      image_width: 768,
      image_height: 768,
      sa32_setting: 0.7,
      sa64_setting: 0.7,
      output_format: "webp",
      guidance_scale: 5,
      output_quality: 80,
      negative_prompt: "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs",
      comic_description: "at home, read new paper #at home, The newspaper says there is a treasure house in the forest.\non the road, near the forest\n[NC] The car on the road, near the forest #He drives to the forest in search of treasure.\n[NC]A tiger appeared in the forest, at night \nvery frightened, open mouth, in the forest, at night\nrunning very fast, in the forest, at night\n[NC] A house in the forest, at night #Suddenly, he discovers the treasure house!\nin the house filled with  treasure, laughing, at night #He is overjoyed inside the house.",
      style_strength_ratio: 20,
      character_description: "a man, wearing black suit"
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run hvision-nku/storydiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "hvision-nku/storydiffusion:d0faed1e8b5779d4d060c5bb0afac8e52c704700f7bee9e4341b0684a8feda04",
    input={
        "num_ids": 3,
        "sd_model": "Unstable",
        "num_steps": 50,
        "style_name": "Japanese Anime",
        "comic_style": "Classic Comic Style",
        "image_width": 768,
        "image_height": 768,
        "sa32_setting": 0.7,
        "sa64_setting": 0.7,
        "output_format": "webp",
        "guidance_scale": 5,
        "output_quality": 80,
        "negative_prompt": "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs",
        "comic_description": "at home, read new paper #at home, The newspaper says there is a treasure house in the forest.\non the road, near the forest\n[NC] The car on the road, near the forest #He drives to the forest in search of treasure.\n[NC]A tiger appeared in the forest, at night \nvery frightened, open mouth, in the forest, at night\nrunning very fast, in the forest, at night\n[NC] A house in the forest, at night #Suddenly, he discovers the treasure house!\nin the house filled with  treasure, laughing, at night #He is overjoyed inside the house.",
        "style_strength_ratio": 20,
        "character_description": "a man, wearing black suit"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run hvision-nku/storydiffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "d0faed1e8b5779d4d060c5bb0afac8e52c704700f7bee9e4341b0684a8feda04",
    "input": {
      "num_ids": 3,
      "sd_model": "Unstable",
      "num_steps": 50,
      "style_name": "Japanese Anime",
      "comic_style": "Classic Comic Style",
      "image_width": 768,
      "image_height": 768,
      "sa32_setting": 0.7,
      "sa64_setting": 0.7,
      "output_format": "webp",
      "guidance_scale": 5,
      "output_quality": 80,
      "negative_prompt": "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs",
      "comic_description": "at home, read new paper #at home, The newspaper says there is a treasure house in the forest.\\non the road, near the forest\\n[NC] The car on the road, near the forest #He drives to the forest in search of treasure.\\n[NC]A tiger appeared in the forest, at night \\nvery frightened, open mouth, in the forest, at night\\nrunning very fast, in the forest, at night\\n[NC] A house in the forest, at night #Suddenly, he discovers the treasure house!\\nin the house filled with  treasure, laughing, at night #He is overjoyed inside the house.",
      "style_strength_ratio": 20,
      "character_description": "a man, wearing black suit"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.