camenduru/story-diffusion:b78fa478 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

camenduru /story-diffusion:b78fa478

Playground API

Input

input_image

*file

style

string

Shift + Return to add a new line

Style template: '(No style)', 'Japanese Anime', 'Cinematic', 'Disney Character', 'Photographic', 'Comic book', 'Line art'

Default: "Japanese Anime"

style_strength_ratio

integer

Style strength of Ref Image (%)

Default: 20

model_type

string

Control type of the Character

Default: "Using Ref Images"

general_prompt

string

Shift + Return to add a new line

a woman img, wearing a white T-shirt, blue loose haira woman img, wearing a white T-shirt, blue loose hair

1️⃣ Textual Description for Character

Default: "a woman img, wearing a white T-shirt, blue loose hair"

negative_prompt

string

Shift + Return to add a new line

bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbsbad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs

2️⃣ Negative Prompt

Default: "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs"

prompt_array

string

Shift + Return to add a new line

wake up in the bed
have breakfast
is on the road, go to company
work in the company
Take a walk next to the company at noon
lying in bed at nightwake up in the bed
have breakfast
is on the road, go to company
work in the company
Take a walk next to the company at noon
lying in bed at night

3️⃣ Comic Description (each line corresponds to a frame).

Default: "wake up in the bed\nhave breakfast\nis on the road, go to company\nwork in the company\nTake a walk next to the company at noon\nlying in bed at night"

Ip_Adapter_Strength

number

Ip Adapter Strength

Default: 0.5

sa32_

number

The degree of Paired Attention at 32 x 32 self-attention layers

Default: 0.5

sa64_

number

The degree of Paired Attention at 64 x 64 self-attention layers

Default: 0.5

id_length_

integer

Number of id images in total images

Default: 3

seed_

integer

Seed

Default: 1

num_steps

integer

Number of sample steps

Default: 50

G_height

integer

Height

Default: 768

G_width

integer

Width

Default: 768

comic_type

string

Typesetting Style

Default: "Classic Comic Style"

guidance_scale

integer

Guidance scale

Default: 5

Run this model in Node.js with one line of code:

npx create-replicate --model=camenduru/story-diffusion

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run camenduru/story-diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "camenduru/story-diffusion:b78fa478b794749964d6e2d3ec28d5ce3fb4a63014efe665d3e51e025f444551",
  {
    input: {
      sa32_: 0.5,
      sa64_: 0.5,
      seed_: 1,
      style: "Japanese Anime",
      G_width: 768,
      G_height: 768,
      num_steps: 50,
      comic_type: "Classic Comic Style",
      id_length_: 3,
      model_type: "Using Ref Images",
      prompt_array: "wake up in the bed\nhave breakfast\nis on the road, go to company\nwork in the company\nTake a walk next to the company at noon\nlying in bed at night",
      general_prompt: "a woman img, wearing a white T-shirt, blue loose hair",
      guidance_scale: 5,
      negative_prompt: "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs",
      Ip_Adapter_Strength: 0.5,
      style_strength_ratio: 20
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run camenduru/story-diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "camenduru/story-diffusion:b78fa478b794749964d6e2d3ec28d5ce3fb4a63014efe665d3e51e025f444551",
    input={
        "sa32_": 0.5,
        "sa64_": 0.5,
        "seed_": 1,
        "style": "Japanese Anime",
        "G_width": 768,
        "G_height": 768,
        "num_steps": 50,
        "comic_type": "Classic Comic Style",
        "id_length_": 3,
        "model_type": "Using Ref Images",
        "prompt_array": "wake up in the bed\nhave breakfast\nis on the road, go to company\nwork in the company\nTake a walk next to the company at noon\nlying in bed at night",
        "general_prompt": "a woman img, wearing a white T-shirt, blue loose hair",
        "guidance_scale": 5,
        "negative_prompt": "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs",
        "Ip_Adapter_Strength": 0.5,
        "style_strength_ratio": 20
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run camenduru/story-diffusion using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "b78fa478b794749964d6e2d3ec28d5ce3fb4a63014efe665d3e51e025f444551",
    "input": {
      "sa32_": 0.5,
      "sa64_": 0.5,
      "seed_": 1,
      "style": "Japanese Anime",
      "G_width": 768,
      "G_height": 768,
      "num_steps": 50,
      "comic_type": "Classic Comic Style",
      "id_length_": 3,
      "model_type": "Using Ref Images",
      "prompt_array": "wake up in the bed\\nhave breakfast\\nis on the road, go to company\\nwork in the company\\nTake a walk next to the company at noon\\nlying in bed at night",
      "general_prompt": "a woman img, wearing a white T-shirt, blue loose hair",
      "guidance_scale": 5,
      "negative_prompt": "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs",
      "Ip_Adapter_Strength": 0.5,
      "style_strength_ratio": 20
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/camenduru/story-diffusion@sha256:b78fa478b794749964d6e2d3ec28d5ce3fb4a63014efe665d3e51e025f444551 \
  -i 'sa32_=0.5' \
  -i 'sa64_=0.5' \
  -i 'seed_=1' \
  -i 'style="Japanese Anime"' \
  -i 'G_width=768' \
  -i 'G_height=768' \
  -i 'num_steps=50' \
  -i 'comic_type="Classic Comic Style"' \
  -i 'id_length_=3' \
  -i 'model_type="Using Ref Images"' \
  -i $'prompt_array="wake up in the bed\\nhave breakfast\\nis on the road, go to company\\nwork in the company\\nTake a walk next to the company at noon\\nlying in bed at night"' \
  -i 'general_prompt="a woman img, wearing a white T-shirt, blue loose hair"' \
  -i 'guidance_scale=5' \
  -i 'negative_prompt="bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs"' \
  -i 'Ip_Adapter_Strength=0.5' \
  -i 'style_strength_ratio=20'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/camenduru/story-diffusion@sha256:b78fa478b794749964d6e2d3ec28d5ce3fb4a63014efe665d3e51e025f444551
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "sa32_": 0.5,
      "sa64_": 0.5,
      "seed_": 1,
      "style": "Japanese Anime",
      "G_width": 768,
      "G_height": 768,
      "num_steps": 50,
      "comic_type": "Classic Comic Style",
      "id_length_": 3,
      "model_type": "Using Ref Images",
      "prompt_array": "wake up in the bed\\nhave breakfast\\nis on the road, go to company\\nwork in the company\\nTake a walk next to the company at noon\\nlying in bed at night",
      "general_prompt": "a woman img, wearing a white T-shirt, blue loose hair",
      "guidance_scale": 5,
      "negative_prompt": "bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn, cartoon, cg, 3d, unreal, animate, amputation, disconnected limbs",
      "Ip_Adapter_Strength": 0.5,
      "style_strength_ratio": 20
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

No output yet! Press "Submit" to start a prediction.