lucataco / seine

Image-to-video - SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction

Cold

Public
12.8K runs
A100 (80GB)
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=lucataco/seine

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run lucataco/seine using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "lucataco/seine:a3a0a85f7f4dd9895ee1dd5bc8d4ffcad365c49cd94976b71603dd7fda408c11",
  {
    input: {
      image: "https://replicate.delivery/pbxt/JuhRUNPhsIsNIWrFdU8raXtP6rb5eSAUALNZKnkG4gWSeWvY/The_picture_shows_the_beauty_of_the_sea_.jpg",
      width: 560,
      height: 240,
      run_time: 13,
      cfg_scale: 8,
      num_frames: 16,
      num_sampling_steps: 250
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run lucataco/seine using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "lucataco/seine:a3a0a85f7f4dd9895ee1dd5bc8d4ffcad365c49cd94976b71603dd7fda408c11",
    input={
        "image": "https://replicate.delivery/pbxt/JuhRUNPhsIsNIWrFdU8raXtP6rb5eSAUALNZKnkG4gWSeWvY/The_picture_shows_the_beauty_of_the_sea_.jpg",
        "width": 560,
        "height": 240,
        "run_time": 13,
        "cfg_scale": 8,
        "num_frames": 16,
        "num_sampling_steps": 250
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run lucataco/seine using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "a3a0a85f7f4dd9895ee1dd5bc8d4ffcad365c49cd94976b71603dd7fda408c11",
    "input": {
      "image": "https://replicate.delivery/pbxt/JuhRUNPhsIsNIWrFdU8raXtP6rb5eSAUALNZKnkG4gWSeWvY/The_picture_shows_the_beauty_of_the_sea_.jpg",
      "width": 560,
      "height": 240,
      "run_time": 13,
      "cfg_scale": 8,
      "num_frames": 16,
      "num_sampling_steps": 250
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2023-11-23T06:05:30.847913Z",
  "created_at": "2023-11-23T06:04:22.611144Z",
  "data_removed": false,
  "error": null,
  "id": "67kha3rbqvymxk4ymmfrawlj24",
  "input": {
    "image": "https://replicate.delivery/pbxt/JuhRUNPhsIsNIWrFdU8raXtP6rb5eSAUALNZKnkG4gWSeWvY/The_picture_shows_the_beauty_of_the_sea_.jpg",
    "width": 560,
    "height": 240,
    "run_time": 13,
    "cfg_scale": 8,
    "num_frames": 16,
    "num_sampling_steps": 250
  },
  "logs": "Using seed: 28954\nloading model\nloading succeed\nWarnning: using half percision for inferencing!\nloading video from /tmp/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.jpg\nloading the input image\n  0%|          | 0/250 [00:00<?, ?it/s]\n  0%|          | 1/250 [00:00<00:56,  4.38it/s]\n  1%|          | 2/250 [00:00<00:48,  5.09it/s]\n  1%|          | 3/250 [00:00<00:46,  5.36it/s]\n  2%|▏         | 4/250 [00:00<00:44,  5.50it/s]\n  2%|▏         | 5/250 [00:00<00:43,  5.58it/s]\n  2%|▏         | 6/250 [00:01<00:43,  5.63it/s]\n  3%|▎         | 7/250 [00:01<00:42,  5.66it/s]\n  3%|▎         | 8/250 [00:01<00:42,  5.68it/s]\n  4%|▎         | 9/250 [00:01<00:42,  5.70it/s]\n  4%|▍         | 10/250 [00:01<00:42,  5.70it/s]\n  4%|▍         | 11/250 [00:01<00:41,  5.71it/s]\n  5%|▍         | 12/250 [00:02<00:41,  5.71it/s]\n  5%|▌         | 13/250 [00:02<00:41,  5.72it/s]\n  6%|▌         | 14/250 [00:02<00:41,  5.72it/s]\n  6%|▌         | 15/250 [00:02<00:41,  5.72it/s]\n  6%|▋         | 16/250 [00:02<00:40,  5.73it/s]\n  7%|▋         | 17/250 [00:03<00:40,  5.73it/s]\n  7%|▋         | 18/250 [00:03<00:40,  5.73it/s]\n  8%|▊         | 19/250 [00:03<00:40,  5.73it/s]\n  8%|▊         | 20/250 [00:03<00:40,  5.73it/s]\n  8%|▊         | 21/250 [00:03<00:40,  5.72it/s]\n  9%|▉         | 22/250 [00:03<00:39,  5.73it/s]\n  9%|▉         | 23/250 [00:04<00:39,  5.73it/s]\n 10%|▉         | 24/250 [00:04<00:39,  5.73it/s]\n 10%|█         | 25/250 [00:04<00:39,  5.73it/s]\n 10%|█         | 26/250 [00:04<00:39,  5.73it/s]\n 11%|█         | 27/250 [00:04<00:38,  5.72it/s]\n 11%|█         | 28/250 [00:04<00:38,  5.73it/s]\n 12%|█▏        | 29/250 [00:05<00:38,  5.73it/s]\n 12%|█▏        | 30/250 [00:05<00:38,  5.73it/s]\n 12%|█▏        | 31/250 [00:05<00:38,  5.73it/s]\n 13%|█▎        | 32/250 [00:05<00:38,  5.73it/s]\n 13%|█▎        | 33/250 [00:05<00:37,  5.73it/s]\n 14%|█▎        | 34/250 [00:05<00:37,  5.73it/s]\n 14%|█▍        | 35/250 [00:06<00:37,  5.73it/s]\n 14%|█▍        | 36/250 [00:06<00:37,  5.73it/s]\n 15%|█▍        | 37/250 [00:06<00:37,  5.73it/s]\n 15%|█▌        | 38/250 [00:06<00:37,  5.73it/s]\n 16%|█▌        | 39/250 [00:06<00:36,  5.73it/s]\n 16%|█▌        | 40/250 [00:07<00:36,  5.73it/s]\n 16%|█▋        | 41/250 [00:07<00:36,  5.73it/s]\n 17%|█▋        | 42/250 [00:07<00:36,  5.73it/s]\n 17%|█▋        | 43/250 [00:07<00:36,  5.73it/s]\n 18%|█▊        | 44/250 [00:07<00:35,  5.73it/s]\n 18%|█▊        | 45/250 [00:07<00:35,  5.73it/s]\n 18%|█▊        | 46/250 [00:08<00:35,  5.68it/s]\n 19%|█▉        | 47/250 [00:08<00:35,  5.70it/s]\n 19%|█▉        | 48/250 [00:08<00:35,  5.71it/s]\n 20%|█▉        | 49/250 [00:08<00:35,  5.72it/s]\n 20%|██        | 50/250 [00:08<00:34,  5.72it/s]\n 20%|██        | 51/250 [00:08<00:34,  5.72it/s]\n 21%|██        | 52/250 [00:09<00:34,  5.72it/s]\n 21%|██        | 53/250 [00:09<00:34,  5.72it/s]\n 22%|██▏       | 54/250 [00:09<00:34,  5.73it/s]\n 22%|██▏       | 55/250 [00:09<00:34,  5.73it/s]\n 22%|██▏       | 56/250 [00:09<00:33,  5.73it/s]\n 23%|██▎       | 57/250 [00:10<00:33,  5.73it/s]\n 23%|██▎       | 58/250 [00:10<00:33,  5.73it/s]\n 24%|██▎       | 59/250 [00:10<00:33,  5.72it/s]\n 24%|██▍       | 60/250 [00:10<00:33,  5.73it/s]\n 24%|██▍       | 61/250 [00:10<00:33,  5.73it/s]\n 25%|██▍       | 62/250 [00:10<00:32,  5.73it/s]\n 25%|██▌       | 63/250 [00:11<00:32,  5.73it/s]\n 26%|██▌       | 64/250 [00:11<00:32,  5.73it/s]\n 26%|██▌       | 65/250 [00:11<00:32,  5.73it/s]\n 26%|██▋       | 66/250 [00:11<00:32,  5.73it/s]\n 27%|██▋       | 67/250 [00:11<00:31,  5.73it/s]\n 27%|██▋       | 68/250 [00:11<00:31,  5.73it/s]\n 28%|██▊       | 69/250 [00:12<00:31,  5.73it/s]\n 28%|██▊       | 70/250 [00:12<00:31,  5.73it/s]\n 28%|██▊       | 71/250 [00:12<00:31,  5.73it/s]\n 29%|██▉       | 72/250 [00:12<00:31,  5.73it/s]\n 29%|██▉       | 73/250 [00:12<00:30,  5.73it/s]\n 30%|██▉       | 74/250 [00:12<00:30,  5.73it/s]\n 30%|███       | 75/250 [00:13<00:30,  5.73it/s]\n 30%|███       | 76/250 [00:13<00:30,  5.73it/s]\n 31%|███       | 77/250 [00:13<00:30,  5.73it/s]\n 31%|███       | 78/250 [00:13<00:30,  5.73it/s]\n 32%|███▏      | 79/250 [00:13<00:29,  5.73it/s]\n 32%|███▏      | 80/250 [00:14<00:29,  5.73it/s]\n 32%|███▏      | 81/250 [00:14<00:29,  5.73it/s]\n 33%|███▎      | 82/250 [00:14<00:29,  5.73it/s]\n 33%|███▎      | 83/250 [00:14<00:29,  5.73it/s]\n 34%|███▎      | 84/250 [00:14<00:28,  5.73it/s]\n 34%|███▍      | 85/250 [00:14<00:28,  5.73it/s]\n 34%|███▍      | 86/250 [00:15<00:28,  5.73it/s]\n 35%|███▍      | 87/250 [00:15<00:29,  5.61it/s]\n 35%|███▌      | 88/250 [00:15<00:28,  5.64it/s]\n 36%|███▌      | 89/250 [00:15<00:28,  5.67it/s]\n 36%|███▌      | 90/250 [00:15<00:28,  5.68it/s]\n 36%|███▋      | 91/250 [00:15<00:27,  5.70it/s]\n 37%|███▋      | 92/250 [00:16<00:27,  5.71it/s]\n 37%|███▋      | 93/250 [00:16<00:27,  5.71it/s]\n 38%|███▊      | 94/250 [00:16<00:27,  5.72it/s]\n 38%|███▊      | 95/250 [00:16<00:27,  5.72it/s]\n 38%|███▊      | 96/250 [00:16<00:26,  5.72it/s]\n 39%|███▉      | 97/250 [00:17<00:26,  5.72it/s]\n 39%|███▉      | 98/250 [00:17<00:26,  5.73it/s]\n 40%|███▉      | 99/250 [00:17<00:26,  5.72it/s]\n 40%|████      | 100/250 [00:17<00:26,  5.73it/s]\n 40%|████      | 101/250 [00:17<00:26,  5.73it/s]\n 41%|████      | 102/250 [00:17<00:25,  5.73it/s]\n 41%|████      | 103/250 [00:18<00:25,  5.73it/s]\n 42%|████▏     | 104/250 [00:18<00:25,  5.73it/s]\n 42%|████▏     | 105/250 [00:18<00:25,  5.73it/s]\n 42%|████▏     | 106/250 [00:18<00:25,  5.73it/s]\n 43%|████▎     | 107/250 [00:18<00:24,  5.73it/s]\n 43%|████▎     | 108/250 [00:18<00:24,  5.73it/s]\n 44%|████▎     | 109/250 [00:19<00:24,  5.73it/s]\n 44%|████▍     | 110/250 [00:19<00:24,  5.73it/s]\n 44%|████▍     | 111/250 [00:19<00:24,  5.73it/s]\n 45%|████▍     | 112/250 [00:19<00:24,  5.73it/s]\n 45%|████▌     | 113/250 [00:19<00:23,  5.73it/s]\n 46%|████▌     | 114/250 [00:19<00:23,  5.73it/s]\n 46%|████▌     | 115/250 [00:20<00:23,  5.73it/s]\n 46%|████▋     | 116/250 [00:20<00:23,  5.72it/s]\n 47%|████▋     | 117/250 [00:20<00:23,  5.72it/s]\n 47%|████▋     | 118/250 [00:20<00:23,  5.73it/s]\n 48%|████▊     | 119/250 [00:20<00:22,  5.73it/s]\n 48%|████▊     | 120/250 [00:21<00:22,  5.72it/s]\n 48%|████▊     | 121/250 [00:21<00:22,  5.73it/s]\n 49%|████▉     | 122/250 [00:21<00:22,  5.73it/s]\n 49%|████▉     | 123/250 [00:21<00:22,  5.72it/s]\n 50%|████▉     | 124/250 [00:21<00:22,  5.73it/s]\n 50%|█████     | 125/250 [00:21<00:21,  5.73it/s]\n 50%|█████     | 126/250 [00:22<00:21,  5.73it/s]\n 51%|█████     | 127/250 [00:22<00:21,  5.73it/s]\n 51%|█████     | 128/250 [00:22<00:21,  5.67it/s]\n 52%|█████▏    | 129/250 [00:22<00:21,  5.69it/s]\n 52%|█████▏    | 130/250 [00:22<00:21,  5.70it/s]\n 52%|█████▏    | 131/250 [00:22<00:20,  5.71it/s]\n 53%|█████▎    | 132/250 [00:23<00:20,  5.72it/s]\n 53%|█████▎    | 133/250 [00:23<00:20,  5.72it/s]\n 54%|█████▎    | 134/250 [00:23<00:20,  5.72it/s]\n 54%|█████▍    | 135/250 [00:23<00:20,  5.72it/s]\n 54%|█████▍    | 136/250 [00:23<00:19,  5.72it/s]\n 55%|█████▍    | 137/250 [00:23<00:19,  5.73it/s]\n 55%|█████▌    | 138/250 [00:24<00:19,  5.73it/s]\n 56%|█████▌    | 139/250 [00:24<00:19,  5.73it/s]\n 56%|█████▌    | 140/250 [00:24<00:19,  5.73it/s]\n 56%|█████▋    | 141/250 [00:24<00:19,  5.73it/s]\n 57%|█████▋    | 142/250 [00:24<00:18,  5.72it/s]\n 57%|█████▋    | 143/250 [00:25<00:18,  5.72it/s]\n 58%|█████▊    | 144/250 [00:25<00:18,  5.72it/s]\n 58%|█████▊    | 145/250 [00:25<00:18,  5.72it/s]\n 58%|█████▊    | 146/250 [00:25<00:18,  5.72it/s]\n 59%|█████▉    | 147/250 [00:25<00:17,  5.72it/s]\n 59%|█████▉    | 148/250 [00:25<00:17,  5.73it/s]\n 60%|█████▉    | 149/250 [00:26<00:17,  5.73it/s]\n 60%|██████    | 150/250 [00:26<00:17,  5.73it/s]\n 60%|██████    | 151/250 [00:26<00:17,  5.72it/s]\n 61%|██████    | 152/250 [00:26<00:17,  5.72it/s]\n 61%|██████    | 153/250 [00:26<00:16,  5.73it/s]\n 62%|██████▏   | 154/250 [00:26<00:16,  5.72it/s]\n 62%|██████▏   | 155/250 [00:27<00:16,  5.72it/s]\n 62%|██████▏   | 156/250 [00:27<00:16,  5.73it/s]\n 63%|██████▎   | 157/250 [00:27<00:16,  5.73it/s]\n 63%|██████▎   | 158/250 [00:27<00:16,  5.73it/s]\n 64%|██████▎   | 159/250 [00:27<00:15,  5.73it/s]\n 64%|██████▍   | 160/250 [00:28<00:15,  5.73it/s]\n 64%|██████▍   | 161/250 [00:28<00:15,  5.73it/s]\n 65%|██████▍   | 162/250 [00:28<00:15,  5.73it/s]\n 65%|██████▌   | 163/250 [00:28<00:15,  5.73it/s]\n 66%|██████▌   | 164/250 [00:28<00:15,  5.73it/s]\n 66%|██████▌   | 165/250 [00:28<00:14,  5.73it/s]\n 66%|██████▋   | 166/250 [00:29<00:14,  5.73it/s]\n 67%|██████▋   | 167/250 [00:29<00:14,  5.72it/s]\n 67%|██████▋   | 168/250 [00:29<00:14,  5.72it/s]\n 68%|██████▊   | 169/250 [00:29<00:14,  5.60it/s]\n 68%|██████▊   | 170/250 [00:29<00:14,  5.63it/s]\n 68%|██████▊   | 171/250 [00:29<00:13,  5.66it/s]\n 69%|██████▉   | 172/250 [00:30<00:13,  5.68it/s]\n 69%|██████▉   | 173/250 [00:30<00:13,  5.69it/s]\n 70%|██████▉   | 174/250 [00:30<00:13,  5.70it/s]\n 70%|███████   | 175/250 [00:30<00:13,  5.70it/s]\n 70%|███████   | 176/250 [00:30<00:12,  5.71it/s]\n 71%|███████   | 177/250 [00:30<00:12,  5.71it/s]\n 71%|███████   | 178/250 [00:31<00:12,  5.71it/s]\n 72%|███████▏  | 179/250 [00:31<00:12,  5.71it/s]\n 72%|███████▏  | 180/250 [00:31<00:12,  5.72it/s]\n 72%|███████▏  | 181/250 [00:31<00:12,  5.72it/s]\n 73%|███████▎  | 182/250 [00:31<00:11,  5.72it/s]\n 73%|███████▎  | 183/250 [00:32<00:11,  5.73it/s]\n 74%|███████▎  | 184/250 [00:32<00:11,  5.73it/s]\n 74%|███████▍  | 185/250 [00:32<00:11,  5.73it/s]\n 74%|███████▍  | 186/250 [00:32<00:11,  5.73it/s]\n 75%|███████▍  | 187/250 [00:32<00:10,  5.73it/s]\n 75%|███████▌  | 188/250 [00:32<00:10,  5.73it/s]\n 76%|███████▌  | 189/250 [00:33<00:10,  5.73it/s]\n 76%|███████▌  | 190/250 [00:33<00:10,  5.73it/s]\n 76%|███████▋  | 191/250 [00:33<00:10,  5.73it/s]\n 77%|███████▋  | 192/250 [00:33<00:10,  5.73it/s]\n 77%|███████▋  | 193/250 [00:33<00:09,  5.73it/s]\n 78%|███████▊  | 194/250 [00:33<00:09,  5.73it/s]\n 78%|███████▊  | 195/250 [00:34<00:09,  5.73it/s]\n 78%|███████▊  | 196/250 [00:34<00:09,  5.73it/s]\n 79%|███████▉  | 197/250 [00:34<00:09,  5.73it/s]\n 79%|███████▉  | 198/250 [00:34<00:09,  5.73it/s]\n 80%|███████▉  | 199/250 [00:34<00:08,  5.72it/s]\n 80%|████████  | 200/250 [00:35<00:08,  5.72it/s]\n 80%|████████  | 201/250 [00:35<00:08,  5.72it/s]\n 81%|████████  | 202/250 [00:35<00:08,  5.72it/s]\n 81%|████████  | 203/250 [00:35<00:08,  5.72it/s]\n 82%|████████▏ | 204/250 [00:35<00:08,  5.73it/s]\n 82%|████████▏ | 205/250 [00:35<00:07,  5.72it/s]\n 82%|████████▏ | 206/250 [00:36<00:07,  5.73it/s]\n 83%|████████▎ | 207/250 [00:36<00:07,  5.73it/s]\n 83%|████████▎ | 208/250 [00:36<00:07,  5.72it/s]\n 84%|████████▎ | 209/250 [00:36<00:07,  5.72it/s]\n 84%|████████▍ | 210/250 [00:36<00:07,  5.61it/s]\n 84%|████████▍ | 211/250 [00:36<00:06,  5.65it/s]\n 85%|████████▍ | 212/250 [00:37<00:06,  5.67it/s]\n 85%|████████▌ | 213/250 [00:37<00:06,  5.69it/s]\n 86%|████████▌ | 214/250 [00:37<00:06,  5.70it/s]\n 86%|████████▌ | 215/250 [00:37<00:06,  5.70it/s]\n 86%|████████▋ | 216/250 [00:37<00:05,  5.70it/s]\n 87%|████████▋ | 217/250 [00:37<00:05,  5.71it/s]\n 87%|████████▋ | 218/250 [00:38<00:05,  5.71it/s]\n 88%|████████▊ | 219/250 [00:38<00:05,  5.71it/s]\n 88%|████████▊ | 220/250 [00:38<00:05,  5.72it/s]\n 88%|████████▊ | 221/250 [00:38<00:05,  5.72it/s]\n 89%|████████▉ | 222/250 [00:38<00:04,  5.72it/s]\n 89%|████████▉ | 223/250 [00:39<00:04,  5.72it/s]\n 90%|████████▉ | 224/250 [00:39<00:04,  5.72it/s]\n 90%|█████████ | 225/250 [00:39<00:04,  5.72it/s]\n 90%|█████████ | 226/250 [00:39<00:04,  5.72it/s]\n 91%|█████████ | 227/250 [00:39<00:04,  5.72it/s]\n 91%|█████████ | 228/250 [00:39<00:03,  5.73it/s]\n 92%|█████████▏| 229/250 [00:40<00:03,  5.72it/s]\n 92%|█████████▏| 230/250 [00:40<00:03,  5.73it/s]\n 92%|█████████▏| 231/250 [00:40<00:03,  5.73it/s]\n 93%|█████████▎| 232/250 [00:40<00:03,  5.73it/s]\n 93%|█████████▎| 233/250 [00:40<00:02,  5.73it/s]\n 94%|█████████▎| 234/250 [00:40<00:02,  5.73it/s]\n 94%|█████████▍| 235/250 [00:41<00:02,  5.73it/s]\n 94%|█████████▍| 236/250 [00:41<00:02,  5.73it/s]\n 95%|█████████▍| 237/250 [00:41<00:02,  5.73it/s]\n 95%|█████████▌| 238/250 [00:41<00:02,  5.72it/s]\n 96%|█████████▌| 239/250 [00:41<00:01,  5.72it/s]\n 96%|█████████▌| 240/250 [00:42<00:01,  5.73it/s]\n 96%|█████████▋| 241/250 [00:42<00:01,  5.72it/s]\n 97%|█████████▋| 242/250 [00:42<00:01,  5.72it/s]\n 97%|█████████▋| 243/250 [00:42<00:01,  5.72it/s]\n 98%|█████████▊| 244/250 [00:42<00:01,  5.72it/s]\n 98%|█████████▊| 245/250 [00:42<00:00,  5.72it/s]\n 98%|█████████▊| 246/250 [00:43<00:00,  5.73it/s]\n 99%|█████████▉| 247/250 [00:43<00:00,  5.72it/s]\n 99%|█████████▉| 248/250 [00:43<00:00,  5.72it/s]\n100%|█████████▉| 249/250 [00:43<00:00,  5.73it/s]\n100%|██████████| 250/250 [00:43<00:00,  5.72it/s]\n100%|██████████| 250/250 [00:43<00:00,  5.71it/s]\nsave in /src/results/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.mp4\n/src/results/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.mp4",
  "metrics": {
    "predict_time": 68.222499,
    "total_time": 68.236769
  },
  "output": "https://replicate.delivery/pbxt/TXNtJYBDGzY8PhpYZemMyUFOO3A6RzPq1nYqmXkPDeeUtT2jA/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.mp4",
  "started_at": "2023-11-23T06:04:22.625414Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/67kha3rbqvymxk4ymmfrawlj24",
    "cancel": "https://api.replicate.com/v1/predictions/67kha3rbqvymxk4ymmfrawlj24/cancel"
  },
  "version": "a3a0a85f7f4dd9895ee1dd5bc8d4ffcad365c49cd94976b71603dd7fda408c11"
}

Generated in

68.2 seconds

Tweak itReport

Using seed: 28954
loading model
loading succeed
Warnning: using half percision for inferencing!
loading video from /tmp/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.jpg
loading the input image
  0%|          | 0/250 [00:00<?, ?it/s]
  0%|          | 1/250 [00:00<00:56,  4.38it/s]
  1%|          | 2/250 [00:00<00:48,  5.09it/s]
  1%|          | 3/250 [00:00<00:46,  5.36it/s]
  2%|▏         | 4/250 [00:00<00:44,  5.50it/s]
  2%|▏         | 5/250 [00:00<00:43,  5.58it/s]
  2%|▏         | 6/250 [00:01<00:43,  5.63it/s]
  3%|▎         | 7/250 [00:01<00:42,  5.66it/s]
  3%|▎         | 8/250 [00:01<00:42,  5.68it/s]
  4%|▎         | 9/250 [00:01<00:42,  5.70it/s]
  4%|▍         | 10/250 [00:01<00:42,  5.70it/s]
  4%|▍         | 11/250 [00:01<00:41,  5.71it/s]
  5%|▍         | 12/250 [00:02<00:41,  5.71it/s]
  5%|▌         | 13/250 [00:02<00:41,  5.72it/s]
  6%|▌         | 14/250 [00:02<00:41,  5.72it/s]
  6%|▌         | 15/250 [00:02<00:41,  5.72it/s]
  6%|▋         | 16/250 [00:02<00:40,  5.73it/s]
  7%|▋         | 17/250 [00:03<00:40,  5.73it/s]
  7%|▋         | 18/250 [00:03<00:40,  5.73it/s]
  8%|▊         | 19/250 [00:03<00:40,  5.73it/s]
  8%|▊         | 20/250 [00:03<00:40,  5.73it/s]
  8%|▊         | 21/250 [00:03<00:40,  5.72it/s]
  9%|▉         | 22/250 [00:03<00:39,  5.73it/s]
  9%|▉         | 23/250 [00:04<00:39,  5.73it/s]
 10%|▉         | 24/250 [00:04<00:39,  5.73it/s]
 10%|█         | 25/250 [00:04<00:39,  5.73it/s]
 10%|█         | 26/250 [00:04<00:39,  5.73it/s]
 11%|█         | 27/250 [00:04<00:38,  5.72it/s]
 11%|█         | 28/250 [00:04<00:38,  5.73it/s]
 12%|█▏        | 29/250 [00:05<00:38,  5.73it/s]
 12%|█▏        | 30/250 [00:05<00:38,  5.73it/s]
 12%|█▏        | 31/250 [00:05<00:38,  5.73it/s]
 13%|█▎        | 32/250 [00:05<00:38,  5.73it/s]
 13%|█▎        | 33/250 [00:05<00:37,  5.73it/s]
 14%|█▎        | 34/250 [00:05<00:37,  5.73it/s]
 14%|█▍        | 35/250 [00:06<00:37,  5.73it/s]
 14%|█▍        | 36/250 [00:06<00:37,  5.73it/s]
 15%|█▍        | 37/250 [00:06<00:37,  5.73it/s]
 15%|█▌        | 38/250 [00:06<00:37,  5.73it/s]
 16%|█▌        | 39/250 [00:06<00:36,  5.73it/s]
 16%|█▌        | 40/250 [00:07<00:36,  5.73it/s]
 16%|█▋        | 41/250 [00:07<00:36,  5.73it/s]
 17%|█▋        | 42/250 [00:07<00:36,  5.73it/s]
 17%|█▋        | 43/250 [00:07<00:36,  5.73it/s]
 18%|█▊        | 44/250 [00:07<00:35,  5.73it/s]
 18%|█▊        | 45/250 [00:07<00:35,  5.73it/s]
 18%|█▊        | 46/250 [00:08<00:35,  5.68it/s]
 19%|█▉        | 47/250 [00:08<00:35,  5.70it/s]
 19%|█▉        | 48/250 [00:08<00:35,  5.71it/s]
 20%|█▉        | 49/250 [00:08<00:35,  5.72it/s]
 20%|██        | 50/250 [00:08<00:34,  5.72it/s]
 20%|██        | 51/250 [00:08<00:34,  5.72it/s]
 21%|██        | 52/250 [00:09<00:34,  5.72it/s]
 21%|██        | 53/250 [00:09<00:34,  5.72it/s]
 22%|██▏       | 54/250 [00:09<00:34,  5.73it/s]
 22%|██▏       | 55/250 [00:09<00:34,  5.73it/s]
 22%|██▏       | 56/250 [00:09<00:33,  5.73it/s]
 23%|██▎       | 57/250 [00:10<00:33,  5.73it/s]
 23%|██▎       | 58/250 [00:10<00:33,  5.73it/s]
 24%|██▎       | 59/250 [00:10<00:33,  5.72it/s]
 24%|██▍       | 60/250 [00:10<00:33,  5.73it/s]
 24%|██▍       | 61/250 [00:10<00:33,  5.73it/s]
 25%|██▍       | 62/250 [00:10<00:32,  5.73it/s]
 25%|██▌       | 63/250 [00:11<00:32,  5.73it/s]
 26%|██▌       | 64/250 [00:11<00:32,  5.73it/s]
 26%|██▌       | 65/250 [00:11<00:32,  5.73it/s]
 26%|██▋       | 66/250 [00:11<00:32,  5.73it/s]
 27%|██▋       | 67/250 [00:11<00:31,  5.73it/s]
 27%|██▋       | 68/250 [00:11<00:31,  5.73it/s]
 28%|██▊       | 69/250 [00:12<00:31,  5.73it/s]
 28%|██▊       | 70/250 [00:12<00:31,  5.73it/s]
 28%|██▊       | 71/250 [00:12<00:31,  5.73it/s]
 29%|██▉       | 72/250 [00:12<00:31,  5.73it/s]
 29%|██▉       | 73/250 [00:12<00:30,  5.73it/s]
 30%|██▉       | 74/250 [00:12<00:30,  5.73it/s]
 30%|███       | 75/250 [00:13<00:30,  5.73it/s]
 30%|███       | 76/250 [00:13<00:30,  5.73it/s]
 31%|███       | 77/250 [00:13<00:30,  5.73it/s]
 31%|███       | 78/250 [00:13<00:30,  5.73it/s]
 32%|███▏      | 79/250 [00:13<00:29,  5.73it/s]
 32%|███▏      | 80/250 [00:14<00:29,  5.73it/s]
 32%|███▏      | 81/250 [00:14<00:29,  5.73it/s]
 33%|███▎      | 82/250 [00:14<00:29,  5.73it/s]
 33%|███▎      | 83/250 [00:14<00:29,  5.73it/s]
 34%|███▎      | 84/250 [00:14<00:28,  5.73it/s]
 34%|███▍      | 85/250 [00:14<00:28,  5.73it/s]
 34%|███▍      | 86/250 [00:15<00:28,  5.73it/s]
 35%|███▍      | 87/250 [00:15<00:29,  5.61it/s]
 35%|███▌      | 88/250 [00:15<00:28,  5.64it/s]
 36%|███▌      | 89/250 [00:15<00:28,  5.67it/s]
 36%|███▌      | 90/250 [00:15<00:28,  5.68it/s]
 36%|███▋      | 91/250 [00:15<00:27,  5.70it/s]
 37%|███▋      | 92/250 [00:16<00:27,  5.71it/s]
 37%|███▋      | 93/250 [00:16<00:27,  5.71it/s]
 38%|███▊      | 94/250 [00:16<00:27,  5.72it/s]
 38%|███▊      | 95/250 [00:16<00:27,  5.72it/s]
 38%|███▊      | 96/250 [00:16<00:26,  5.72it/s]
 39%|███▉      | 97/250 [00:17<00:26,  5.72it/s]
 39%|███▉      | 98/250 [00:17<00:26,  5.73it/s]
 40%|███▉      | 99/250 [00:17<00:26,  5.72it/s]
 40%|████      | 100/250 [00:17<00:26,  5.73it/s]
 40%|████      | 101/250 [00:17<00:26,  5.73it/s]
 41%|████      | 102/250 [00:17<00:25,  5.73it/s]
 41%|████      | 103/250 [00:18<00:25,  5.73it/s]
 42%|████▏     | 104/250 [00:18<00:25,  5.73it/s]
 42%|████▏     | 105/250 [00:18<00:25,  5.73it/s]
 42%|████▏     | 106/250 [00:18<00:25,  5.73it/s]
 43%|████▎     | 107/250 [00:18<00:24,  5.73it/s]
 43%|████▎     | 108/250 [00:18<00:24,  5.73it/s]
 44%|████▎     | 109/250 [00:19<00:24,  5.73it/s]
 44%|████▍     | 110/250 [00:19<00:24,  5.73it/s]
 44%|████▍     | 111/250 [00:19<00:24,  5.73it/s]
 45%|████▍     | 112/250 [00:19<00:24,  5.73it/s]
 45%|████▌     | 113/250 [00:19<00:23,  5.73it/s]
 46%|████▌     | 114/250 [00:19<00:23,  5.73it/s]
 46%|████▌     | 115/250 [00:20<00:23,  5.73it/s]
 46%|████▋     | 116/250 [00:20<00:23,  5.72it/s]
 47%|████▋     | 117/250 [00:20<00:23,  5.72it/s]
 47%|████▋     | 118/250 [00:20<00:23,  5.73it/s]
 48%|████▊     | 119/250 [00:20<00:22,  5.73it/s]
 48%|████▊     | 120/250 [00:21<00:22,  5.72it/s]
 48%|████▊     | 121/250 [00:21<00:22,  5.73it/s]
 49%|████▉     | 122/250 [00:21<00:22,  5.73it/s]
 49%|████▉     | 123/250 [00:21<00:22,  5.72it/s]
 50%|████▉     | 124/250 [00:21<00:22,  5.73it/s]
 50%|█████     | 125/250 [00:21<00:21,  5.73it/s]
 50%|█████     | 126/250 [00:22<00:21,  5.73it/s]
 51%|█████     | 127/250 [00:22<00:21,  5.73it/s]
 51%|█████     | 128/250 [00:22<00:21,  5.67it/s]
 52%|█████▏    | 129/250 [00:22<00:21,  5.69it/s]
 52%|█████▏    | 130/250 [00:22<00:21,  5.70it/s]
 52%|█████▏    | 131/250 [00:22<00:20,  5.71it/s]
 53%|█████▎    | 132/250 [00:23<00:20,  5.72it/s]
 53%|█████▎    | 133/250 [00:23<00:20,  5.72it/s]
 54%|█████▎    | 134/250 [00:23<00:20,  5.72it/s]
 54%|█████▍    | 135/250 [00:23<00:20,  5.72it/s]
 54%|█████▍    | 136/250 [00:23<00:19,  5.72it/s]
 55%|█████▍    | 137/250 [00:23<00:19,  5.73it/s]
 55%|█████▌    | 138/250 [00:24<00:19,  5.73it/s]
 56%|█████▌    | 139/250 [00:24<00:19,  5.73it/s]
 56%|█████▌    | 140/250 [00:24<00:19,  5.73it/s]
 56%|█████▋    | 141/250 [00:24<00:19,  5.73it/s]
 57%|█████▋    | 142/250 [00:24<00:18,  5.72it/s]
 57%|█████▋    | 143/250 [00:25<00:18,  5.72it/s]
 58%|█████▊    | 144/250 [00:25<00:18,  5.72it/s]
 58%|█████▊    | 145/250 [00:25<00:18,  5.72it/s]
 58%|█████▊    | 146/250 [00:25<00:18,  5.72it/s]
 59%|█████▉    | 147/250 [00:25<00:17,  5.72it/s]
 59%|█████▉    | 148/250 [00:25<00:17,  5.73it/s]
 60%|█████▉    | 149/250 [00:26<00:17,  5.73it/s]
 60%|██████    | 150/250 [00:26<00:17,  5.73it/s]
 60%|██████    | 151/250 [00:26<00:17,  5.72it/s]
 61%|██████    | 152/250 [00:26<00:17,  5.72it/s]
 61%|██████    | 153/250 [00:26<00:16,  5.73it/s]
 62%|██████▏   | 154/250 [00:26<00:16,  5.72it/s]
 62%|██████▏   | 155/250 [00:27<00:16,  5.72it/s]
 62%|██████▏   | 156/250 [00:27<00:16,  5.73it/s]
 63%|██████▎   | 157/250 [00:27<00:16,  5.73it/s]
 63%|██████▎   | 158/250 [00:27<00:16,  5.73it/s]
 64%|██████▎   | 159/250 [00:27<00:15,  5.73it/s]
 64%|██████▍   | 160/250 [00:28<00:15,  5.73it/s]
 64%|██████▍   | 161/250 [00:28<00:15,  5.73it/s]
 65%|██████▍   | 162/250 [00:28<00:15,  5.73it/s]
 65%|██████▌   | 163/250 [00:28<00:15,  5.73it/s]
 66%|██████▌   | 164/250 [00:28<00:15,  5.73it/s]
 66%|██████▌   | 165/250 [00:28<00:14,  5.73it/s]
 66%|██████▋   | 166/250 [00:29<00:14,  5.73it/s]
 67%|██████▋   | 167/250 [00:29<00:14,  5.72it/s]
 67%|██████▋   | 168/250 [00:29<00:14,  5.72it/s]
 68%|██████▊   | 169/250 [00:29<00:14,  5.60it/s]
 68%|██████▊   | 170/250 [00:29<00:14,  5.63it/s]
 68%|██████▊   | 171/250 [00:29<00:13,  5.66it/s]
 69%|██████▉   | 172/250 [00:30<00:13,  5.68it/s]
 69%|██████▉   | 173/250 [00:30<00:13,  5.69it/s]
 70%|██████▉   | 174/250 [00:30<00:13,  5.70it/s]
 70%|███████   | 175/250 [00:30<00:13,  5.70it/s]
 70%|███████   | 176/250 [00:30<00:12,  5.71it/s]
 71%|███████   | 177/250 [00:30<00:12,  5.71it/s]
 71%|███████   | 178/250 [00:31<00:12,  5.71it/s]
 72%|███████▏  | 179/250 [00:31<00:12,  5.71it/s]
 72%|███████▏  | 180/250 [00:31<00:12,  5.72it/s]
 72%|███████▏  | 181/250 [00:31<00:12,  5.72it/s]
 73%|███████▎  | 182/250 [00:31<00:11,  5.72it/s]
 73%|███████▎  | 183/250 [00:32<00:11,  5.73it/s]
 74%|███████▎  | 184/250 [00:32<00:11,  5.73it/s]
 74%|███████▍  | 185/250 [00:32<00:11,  5.73it/s]
 74%|███████▍  | 186/250 [00:32<00:11,  5.73it/s]
 75%|███████▍  | 187/250 [00:32<00:10,  5.73it/s]
 75%|███████▌  | 188/250 [00:32<00:10,  5.73it/s]
 76%|███████▌  | 189/250 [00:33<00:10,  5.73it/s]
 76%|███████▌  | 190/250 [00:33<00:10,  5.73it/s]
 76%|███████▋  | 191/250 [00:33<00:10,  5.73it/s]
 77%|███████▋  | 192/250 [00:33<00:10,  5.73it/s]
 77%|███████▋  | 193/250 [00:33<00:09,  5.73it/s]
 78%|███████▊  | 194/250 [00:33<00:09,  5.73it/s]
 78%|███████▊  | 195/250 [00:34<00:09,  5.73it/s]
 78%|███████▊  | 196/250 [00:34<00:09,  5.73it/s]
 79%|███████▉  | 197/250 [00:34<00:09,  5.73it/s]
 79%|███████▉  | 198/250 [00:34<00:09,  5.73it/s]
 80%|███████▉  | 199/250 [00:34<00:08,  5.72it/s]
 80%|████████  | 200/250 [00:35<00:08,  5.72it/s]
 80%|████████  | 201/250 [00:35<00:08,  5.72it/s]
 81%|████████  | 202/250 [00:35<00:08,  5.72it/s]
 81%|████████  | 203/250 [00:35<00:08,  5.72it/s]
 82%|████████▏ | 204/250 [00:35<00:08,  5.73it/s]
 82%|████████▏ | 205/250 [00:35<00:07,  5.72it/s]
 82%|████████▏ | 206/250 [00:36<00:07,  5.73it/s]
 83%|████████▎ | 207/250 [00:36<00:07,  5.73it/s]
 83%|████████▎ | 208/250 [00:36<00:07,  5.72it/s]
 84%|████████▎ | 209/250 [00:36<00:07,  5.72it/s]
 84%|████████▍ | 210/250 [00:36<00:07,  5.61it/s]
 84%|████████▍ | 211/250 [00:36<00:06,  5.65it/s]
 85%|████████▍ | 212/250 [00:37<00:06,  5.67it/s]
 85%|████████▌ | 213/250 [00:37<00:06,  5.69it/s]
 86%|████████▌ | 214/250 [00:37<00:06,  5.70it/s]
 86%|████████▌ | 215/250 [00:37<00:06,  5.70it/s]
 86%|████████▋ | 216/250 [00:37<00:05,  5.70it/s]
 87%|████████▋ | 217/250 [00:37<00:05,  5.71it/s]
 87%|████████▋ | 218/250 [00:38<00:05,  5.71it/s]
 88%|████████▊ | 219/250 [00:38<00:05,  5.71it/s]
 88%|████████▊ | 220/250 [00:38<00:05,  5.72it/s]
 88%|████████▊ | 221/250 [00:38<00:05,  5.72it/s]
 89%|████████▉ | 222/250 [00:38<00:04,  5.72it/s]
 89%|████████▉ | 223/250 [00:39<00:04,  5.72it/s]
 90%|████████▉ | 224/250 [00:39<00:04,  5.72it/s]
 90%|█████████ | 225/250 [00:39<00:04,  5.72it/s]
 90%|█████████ | 226/250 [00:39<00:04,  5.72it/s]
 91%|█████████ | 227/250 [00:39<00:04,  5.72it/s]
 91%|█████████ | 228/250 [00:39<00:03,  5.73it/s]
 92%|█████████▏| 229/250 [00:40<00:03,  5.72it/s]
 92%|█████████▏| 230/250 [00:40<00:03,  5.73it/s]
 92%|█████████▏| 231/250 [00:40<00:03,  5.73it/s]
 93%|█████████▎| 232/250 [00:40<00:03,  5.73it/s]
 93%|█████████▎| 233/250 [00:40<00:02,  5.73it/s]
 94%|█████████▎| 234/250 [00:40<00:02,  5.73it/s]
 94%|█████████▍| 235/250 [00:41<00:02,  5.73it/s]
 94%|█████████▍| 236/250 [00:41<00:02,  5.73it/s]
 95%|█████████▍| 237/250 [00:41<00:02,  5.73it/s]
 95%|█████████▌| 238/250 [00:41<00:02,  5.72it/s]
 96%|█████████▌| 239/250 [00:41<00:01,  5.72it/s]
 96%|█████████▌| 240/250 [00:42<00:01,  5.73it/s]
 96%|█████████▋| 241/250 [00:42<00:01,  5.72it/s]
 97%|█████████▋| 242/250 [00:42<00:01,  5.72it/s]
 97%|█████████▋| 243/250 [00:42<00:01,  5.72it/s]
 98%|█████████▊| 244/250 [00:42<00:01,  5.72it/s]
 98%|█████████▊| 245/250 [00:42<00:00,  5.72it/s]
 98%|█████████▊| 246/250 [00:43<00:00,  5.73it/s]
 99%|█████████▉| 247/250 [00:43<00:00,  5.72it/s]
 99%|█████████▉| 248/250 [00:43<00:00,  5.72it/s]
100%|█████████▉| 249/250 [00:43<00:00,  5.73it/s]
100%|██████████| 250/250 [00:43<00:00,  5.72it/s]
100%|██████████| 250/250 [00:43<00:00,  5.71it/s]
save in /src/results/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.mp4
/src/results/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.mp4

Examples

View more examples

Run time and cost

This model costs approximately $0.37 to run on Replicate, or 2 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 5 minutes.

Readme

About

Implementation of Vchitect/SEINE Image-To-Video model

BibTeX

@article{chen2023seine,
title={SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction},
author={Chen, Xinyuan and Wang, Yaohui and Zhang, Lingjun and Zhuang, Shaobin and Ma, Xin and Yu, Jiashuo and Wang, Yali and Lin, Dahua and Qiao, Yu and Liu, Ziwei},
journal={arXiv preprint arXiv:2310.20700},
year={2023}
}

Disclaimer

We disclaim responsibility for user-generated content. The model was not trained to realistically represent people or events, so using it to generate such content is beyond the model’s capabilities. It is prohibited for pornographic, violent and bloody content generation, and to generate content that is demeaning or harmful to people or their environment, culture, religion, etc. Users are solely liable for their actions. The project contributors are not legally affiliated with, nor accountable for users’ behaviors. Use the generative model responsibly, adhering to ethical and legal standards.

Contact Us

Xinyuan Chen: chenxinyuan@pjlab.org.cn Yaohui Wang: wangyaohui@pjlab.org.cn

Acknowledgements

The code is built upon diffusers and Stable Diffusion, we thank all the contributors for open-sourcing.

License

The code is licensed under Apache-2.0, model weights are fully open for academic research and also allow free commercial usage. To apply for a commercial license, please contact vchitect@pjlab.org.cn.