zsxkib / pyramid-flow

Text-to-Video + Image-to-Video: Pyramid Flow Autoregressive Video Generation method based on Flow Matching

Warm

Public
8.3K runs
A100 (80GB)
GitHub
Weights
Paper
License

Run with an API

Playground API Examples README Versions

Prediction

zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb

Model

zsxkib/pyramid-flow:8e221e66

kr169eyk6xrj40cjf34basf1k0

Status

Succeeded

Source

Web

Hardware

A100 (80GB)

Total duration

6m 59s

Created

7 months ago by @zsxkib

Input

image
prompt: Cowgirl in classic American cowgirl outfit waving goodbye at the San Fransisco bridge with alien drones around her, amazing shot, accidental renaissance, goodbye wave shot in later 1990s disposable digital camera
duration: 5
guidance_scale: 9
frames_per_second: 24
video_guidance_scale: 5

{
  "image": "https://replicate.delivery/pbxt/LlzrbiaC6xMTQPbIsCg8l6cEupjdcQhPSTjJIy9VF8iF2Gln/cow.webp",
  "prompt": "Cowgirl in classic American cowgirl outfit waving goodbye at the San Fransisco bridge with alien drones around her, amazing shot, accidental renaissance, goodbye wave shot in later 1990s disposable digital camera",
  "duration": 5,
  "guidance_scale": 9,
  "frames_per_second": 24,
  "video_guidance_scale": 5
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
  {
    input: {
      image: "https://replicate.delivery/pbxt/LlzrbiaC6xMTQPbIsCg8l6cEupjdcQhPSTjJIy9VF8iF2Gln/cow.webp",
      prompt: "Cowgirl in classic American cowgirl outfit waving goodbye at the San Fransisco bridge with alien drones around her, amazing shot, accidental renaissance, goodbye wave shot in later 1990s disposable digital camera",
      duration: 5,
      guidance_scale: 9,
      frames_per_second: 24,
      video_guidance_scale: 5
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
    input={
        "image": "https://replicate.delivery/pbxt/LlzrbiaC6xMTQPbIsCg8l6cEupjdcQhPSTjJIy9VF8iF2Gln/cow.webp",
        "prompt": "Cowgirl in classic American cowgirl outfit waving goodbye at the San Fransisco bridge with alien drones around her, amazing shot, accidental renaissance, goodbye wave shot in later 1990s disposable digital camera",
        "duration": 5,
        "guidance_scale": 9,
        "frames_per_second": 24,
        "video_guidance_scale": 5
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
    "input": {
      "image": "https://replicate.delivery/pbxt/LlzrbiaC6xMTQPbIsCg8l6cEupjdcQhPSTjJIy9VF8iF2Gln/cow.webp",
      "prompt": "Cowgirl in classic American cowgirl outfit waving goodbye at the San Fransisco bridge with alien drones around her, amazing shot, accidental renaissance, goodbye wave shot in later 1990s disposable digital camera",
      "duration": 5,
      "guidance_scale": 9,
      "frames_per_second": 24,
      "video_guidance_scale": 5
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-10-10T23:41:38.714232Z",
  "created_at": "2024-10-10T23:34:40.183000Z",
  "data_removed": false,
  "error": null,
  "id": "kr169eyk6xrj40cjf34basf1k0",
  "input": {
    "image": "https://replicate.delivery/pbxt/LlzrbiaC6xMTQPbIsCg8l6cEupjdcQhPSTjJIy9VF8iF2Gln/cow.webp",
    "prompt": "Cowgirl in classic American cowgirl outfit waving goodbye at the San Fransisco bridge with alien drones around her, amazing shot, accidental renaissance, goodbye wave shot in later 1990s disposable digital camera",
    "duration": 5,
    "guidance_scale": 9,
    "frames_per_second": 24,
    "video_guidance_scale": 5
  },
  "logs": "/src/predict.py:116: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.\nwith torch.no_grad(), torch.cuda.amp.autocast(\n  0%|          | 0/16 [00:00<?, ?it/s]\n  6%|▋         | 1/16 [00:08<02:13,  8.90s/it]\n 12%|█▎        | 2/16 [00:19<02:17,  9.86s/it]\n 19%|█▉        | 3/16 [00:30<02:15, 10.40s/it]\n 25%|██▌       | 4/16 [00:42<02:11, 10.96s/it]\n 31%|███▏      | 5/16 [00:54<02:06, 11.49s/it]\n 38%|███▊      | 6/16 [01:07<01:59, 11.94s/it]\n 44%|████▍     | 7/16 [01:21<01:52, 12.48s/it]\n 50%|█████     | 8/16 [01:35<01:44, 13.01s/it]\n 56%|█████▋    | 9/16 [01:50<01:35, 13.63s/it]\n 62%|██████▎   | 10/16 [02:05<01:25, 14.25s/it]\n 69%|██████▉   | 11/16 [02:22<01:14, 14.86s/it]\n 75%|███████▌  | 12/16 [02:39<01:03, 15.75s/it]\n 81%|████████▏ | 13/16 [02:57<00:49, 16.38s/it]\n 88%|████████▊ | 14/16 [03:16<00:33, 16.99s/it]\n 94%|█████████▍| 15/16 [03:35<00:17, 17.71s/it]\n100%|██████████| 16/16 [03:55<00:00, 18.42s/it]\n100%|██████████| 16/16 [03:55<00:00, 14.72s/it]\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n- Avoid using `tokenizers` before the fork if possible\n- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)",
  "metrics": {
    "predict_time": 272.589486604,
    "total_time": 418.531232
  },
  "output": "https://replicate.delivery/yhqm/E9bafcnwQ7wxDSHdZGvR7i6DzDOE8ygVDEaLc5EZeAsyghlTA/output_video.mp4",
  "started_at": "2024-10-10T23:37:06.124746Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/kr169eyk6xrj40cjf34basf1k0",
    "cancel": "https://api.replicate.com/v1/predictions/kr169eyk6xrj40cjf34basf1k0/cancel"
  },
  "version": "8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb"
}

Generated in

4 minutes 33 seconds

Tweak it ShareReport

Prediction

zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb

Model

zsxkib/pyramid-flow:8e221e66

28b0nb8xnnrj60cjf2vb4fmp94

Status

Succeeded

Source

Web

Hardware

A100 (80GB)

Total duration

4m 16s

Created

7 months ago by @zsxkib

Input

prompt: A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares
duration: 5
guidance_scale: 9
frames_per_second: 24
video_guidance_scale: 5

{
  "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
  "duration": 5,
  "guidance_scale": 9,
  "frames_per_second": 24,
  "video_guidance_scale": 5
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
  {
    input: {
      prompt: "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
      duration: 5,
      guidance_scale: 9,
      frames_per_second: 24,
      video_guidance_scale: 5
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/pyramid-flow:8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
    input={
        "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
        "duration": 5,
        "guidance_scale": 9,
        "frames_per_second": 24,
        "video_guidance_scale": 5
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run zsxkib/pyramid-flow using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb",
    "input": {
      "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
      "duration": 5,
      "guidance_scale": 9,
      "frames_per_second": 24,
      "video_guidance_scale": 5
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-10-10T23:18:30.186752Z",
  "created_at": "2024-10-10T23:14:14.061000Z",
  "data_removed": false,
  "error": null,
  "id": "28b0nb8xnnrj60cjf2vb4fmp94",
  "input": {
    "prompt": "A gripping movie trailer showcasing a young female astronaut in a iridescent spacesuit, sporting a red wool knitted space helmet. She explores a bioluminescent alien forest under twin moons, filmed in vivid IMAX quality. Retro sci-fi aesthetic, lens flares",
    "duration": 5,
    "guidance_scale": 9,
    "frames_per_second": 24,
    "video_guidance_scale": 5
  },
  "logs": "0%|          | 0/16 [00:00<?, ?it/s]\n  6%|▋         | 1/16 [00:08<02:01,  8.13s/it]\n 12%|█▎        | 2/16 [00:17<02:01,  8.69s/it]\n 19%|█▉        | 3/16 [00:27<02:03,  9.46s/it]\n 25%|██▌       | 4/16 [00:38<02:01, 10.13s/it]\n 31%|███▏      | 5/16 [00:50<01:58, 10.73s/it]\n 38%|███▊      | 6/16 [01:02<01:52, 11.27s/it]\n 44%|████▍     | 7/16 [01:15<01:46, 11.81s/it]\n 50%|█████     | 8/16 [01:29<01:38, 12.33s/it]\n 56%|█████▋    | 9/16 [01:43<01:30, 12.94s/it]\n 62%|██████▎   | 10/16 [01:58<01:21, 13.57s/it]\n 69%|██████▉   | 11/16 [02:13<01:10, 14.17s/it]\n 75%|███████▌  | 12/16 [02:30<00:59, 14.82s/it]\n 81%|████████▏ | 13/16 [02:47<00:46, 15.45s/it]\n 88%|████████▊ | 14/16 [03:05<00:32, 16.16s/it]\n 94%|█████████▍| 15/16 [03:23<00:16, 16.83s/it]\n100%|██████████| 16/16 [03:42<00:00, 17.58s/it]\n100%|██████████| 16/16 [03:42<00:00, 13.92s/it]\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n- Avoid using `tokenizers` before the fork if possible\n- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)",
  "metrics": {
    "predict_time": 256.106868797,
    "total_time": 256.125752
  },
  "output": "https://replicate.delivery/yhqm/5fhDu5VhX21zBK3oCphPd2SvQ1px4wvLgF8Ughg5U1EjlwyJA/output_video.mp4",
  "started_at": "2024-10-10T23:14:14.079883Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/28b0nb8xnnrj60cjf2vb4fmp94",
    "cancel": "https://api.replicate.com/v1/predictions/28b0nb8xnnrj60cjf2vb4fmp94/cancel"
  },
  "version": "8e221e66498a52bb3a928a4b49d85379c99ca60fec41511265deec35d547c1fb"
}

Generated in

4 minutes 16 seconds

Tweak it ShareReport

Want to make some of these yourself?

Run this model

zsxkib / pyramid-flow

Prediction

Input

Output

Prediction

Input

Output

Logs (kr169eyk6xrj40cjf34basf1k0)

Logs (28b0nb8xnnrj60cjf2vb4fmp94)