zsxkib/pyramid-flow | Run with an API on Replicate

Text-to-Video + Image-to-Video: Pyramid Flow Autoregressive Video Generation method based on Flow Matching

Cold

Public

9.1K runs

A100 (80GB)

License

GitHub

Weights

Paper

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flaresA gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares

Text prompt for video generation

image

file

Optional input image for image-to-video generation

duration

integer

(minimum: 1, maximum: 10)

Duration of the video in seconds (1-3 for canonical mode, 1-10 for non-canonical mode)

Default: 3

guidance_scale

number

(minimum: 1, maximum: 15)

Guidance Scale for text-to-video generation

Default: 9

video_guidance_scale

number

(minimum: 1, maximum: 15)

Video Guidance Scale

Default: 5

frames_per_second

integer

Frames per second (8 or 24, only applicable in canonical mode)

Default: 8

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/pyramid-flow

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
  {
    input: {
      prompt: "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
      duration: 5,
      guidance_scale: 9,
      frames_per_second: 24,
      video_guidance_scale: 5
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
    input={
        "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
        "duration": 5,
        "guidance_scale": 9,
        "frames_per_second": 24,
        "video_guidance_scale": 5
    }
)

# To access the file URL:
print(output.url())
#=> "http://example.com"

# To write the file to disk:
with open("my-image.png", "wb") as file:
    file.write(output.read())

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
    "input": {
      "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
      "duration": 5,
      "guidance_scale": 9,
      "frames_per_second": 24,
      "video_guidance_scale": 5
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/zsxkib/pyramid-flow@sha256:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb \
  -i 'prompt="A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares"' \
  -i 'duration=5' \
  -i 'guidance_scale=9' \
  -i 'frames_per_second=24' \
  -i 'video_guidance_scale=5'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/zsxkib/pyramid-flow@sha256:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
      "duration": 5,
      "guidance_scale": 9,
      "frames_per_second": 24,
      "video_guidance_scale": 5
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

{
  "completed_at": "2024-10-10T23:18:30.186752Z",
  "created_at": "2024-10-10T23:14:14.061000Z",
  "data_removed": false,
  "error": null,
  "id": "28b0nb8xnnrj60cjf2vb4fmp94",
  "input": {
    "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
    "duration": 5,
    "guidance_scale": 9,
    "frames_per_second": 24,
    "video_guidance_scale": 5
  },
  "logs": "0%|          | 0/16 [00:00<?, ?it/s]\n  6%|▋         | 1/16 [00:08<02:01,  8.13s/it]\n 12%|█▎        | 2/16 [00:17<02:01,  8.69s/it]\n 19%|█▉        | 3/16 [00:27<02:03,  9.46s/it]\n 25%|██▌       | 4/16 [00:38<02:01, 10.13s/it]\n 31%|███▏      | 5/16 [00:50<01:58, 10.73s/it]\n 38%|███▊      | 6/16 [01:02<01:52, 11.27s/it]\n 44%|████▍     | 7/16 [01:15<01:46, 11.81s/it]\n 50%|█████     | 8/16 [01:29<01:38, 12.33s/it]\n 56%|█████▋    | 9/16 [01:43<01:30, 12.94s/it]\n 62%|██████▎   | 10/16 [01:58<01:21, 13.57s/it]\n 69%|██████▉   | 11/16 [02:13<01:10, 14.17s/it]\n 75%|███████▌  | 12/16 [02:30<00:59, 14.82s/it]\n 81%|████████▏ | 13/16 [02:47<00:46, 15.45s/it]\n 88%|████████▊ | 14/16 [03:05<00:32, 16.16s/it]\n 94%|█████████▍| 15/16 [03:23<00:16, 16.83s/it]\n100%|██████████| 16/16 [03:42<00:00, 17.58s/it]\n100%|██████████| 16/16 [03:42<00:00, 13.92s/it]\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n- Avoid using `tokenizers` before the fork if possible\n- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)",
  "metrics": {
    "predict_time": 256.106868797,
    "total_time": 256.125752
  },
  "output": "https://replicate.delivery/yhqm/5fhDu5VhX21zBK3oCphPd2SvQ1px4wvLgF8Ughg5U1EjlwyJA/output_video.mp4",
  "started_at": "2024-10-10T23:14:14.079883Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/28b0nb8xnnrj60cjf2vb4fmp94",
    "cancel": "https://api.replicate.com/v1/predictions/28b0nb8xnnrj60cjf2vb4fmp94/cancel"
  },
  "version": "8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb"
}

Generated in

4 minutes 16 seconds

Tweak it Iterate in playground ShareReport View full prediction

Examples

View more examples

Run time and cost

This model costs approximately $0.44 to run on Replicate, or 2 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 6 minutes. The predict time for this model varies significantly based on the inputs.

Readme

huggingface https://huggingface.co/rain1011/pyramid-flow-sd3 paper https://arxiv.org/abs/2410.05954 (project page https://pyramid-flow.github.io)

kwaivgi/kling-v2.1

Generate short videos from a start image and a text prompt. Produce 5 or 10 second clips at 24 fps in 720p (standard) or 1080p (pro). Optionally supply an end image in pro mode to guide the final frame or interpolate between start and end.

2.5m runs

Official

kwaivgi/kling-v2.5-turbo-pro

Generate 5–10 second videos from text prompts or a single starting image. Accept a required prompt and optional first-frame image, and output short clips with fluid motion, stable frames, and coherent pacing. Preserve color, lighting, and mood across frames with refined conditioning, and follow multi-step, causal instructions for complex camera moves. Suited for marketing assets, creator shorts, film/animation previz, and educational explainers.

740.3k runs

Official

kwaivgi/kling-v1.6-standard

Generate short videos from text prompts. Optionally condition on a start image to create image-to-video clips and include up to 4 reference images as scene elements. Choose 5s or 10s duration with 720p output at 30fps, and set aspect ratio to 16:9, 9:16, or 1:1. Supports negative prompts to steer content and returns a video.

1.4m runs

Official

bytedance/seedance-1-lite

Generate short videos from text prompts or a starting image. Produce 2–12 second clips at 24 fps in up to 1080p resolution across aspect ratios including 16:9, 4:3, 1:1, 3:4, 9:16, 21:9, and 9:21. Guide subjects, style, and multi-character interactions with 1–4 reference images for character, clothing, and environment consistency. Optionally lock the camera, set a random seed for reproducibility, and anchor start/end frames with first- and last-frame images. Outputs a video.

1.6m runs

Official

bytedance/seedance-1-pro

Generate videos from text prompts or a single input image. Produce 2–12 second clips at 24 fps in 480p, 720p, or 1080p and common aspect ratios (16:9, 9:16, 1:1, 4:3, 3:4, 21:9, 9:21). Control motion with a camera lock option and constrain start/end points by supplying both a start image and a last-frame image; set a seed for reproducible results. Support multi-shot generation with narrative coherence, consistent subjects and visual style across shot transitions, and temporal/spatial shifts. Handle subtle to large-scale motion, complex action sequences, and multi-agent interactions with stable physical realism. Interpret stylistic prompts including photorealism, cyberpunk, illustration, and felt texture while maintaining prompt adherence and source-image consistency in image-to-video.

982.4k runs

Official

minimax/video-01

Generate up to 6-second 720p, 25fps videos from a text prompt or by animating a first-frame image. Maintain a consistent character by providing a subject reference image (S2V-01). Use a first-frame image to set aspect ratio and initial composition while the prompt drives motion and cinematic camera movement. Includes an optional prompt optimizer.

609.4k runs

Official

minimax/hailuo-02

Generate 6–10 second videos from text prompts or a single reference image. Select 512p, 768p (up to 10s), or 1080p (6s), with a pro 1080p mode offering improved motion coherence. Optionally pin the first or last frame with input images (aspect ratio follows the first-frame image) to guide motion and endpoints. Emphasizes realistic physics and reliable instruction following for complex actions and scene transitions. Includes an optional prompt optimizer.

195.3k runs

Official

google/veo-2

Generate short videos from a text prompt or an initial image. Produce silent clips with realistic motion and physics, following simple or complex instructions and camera directions (shot styles, angles, movements) across diverse visual styles, up to 4K. Control duration (5–8 seconds) and aspect ratio (16:9 or 9:16), and optionally fix a random seed for repeatable results.

102.1k runs

Official

Input

Output

Examples

Run time and cost

Readme

Related models

kwaivgi/kling-v2.1

kwaivgi/kling-v2.5-turbo-pro

kwaivgi/kling-v1.6-standard

bytedance/seedance-1-lite

bytedance/seedance-1-pro

minimax/video-01

minimax/hailuo-02

google/veo-2