chenxwh / ltx-video

DiT-based video generation model for generating high-quality videos in real-time (Updated 6 months, 3 weeks ago)

Cold

Public
3.2K runs
L40S
GitHub
Weights
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

The waves crash against the jagged rocks of the shoreline. The waves crash against the jagged rocks of the shoreline, sending spray high into the air.The rocks are a dark gray color, with sharp edges and deep crevices. The water is a clear blue-green, with white foam where the waves break against the rocks. The sky is a light gray, with a few white clouds dotting the horizon.The waves crash against the jagged rocks of the shoreline. The waves crash against the jagged rocks of the shoreline, sending spray high into the air.The rocks are a dark gray color, with sharp edges and deep crevices. The water is a clear blue-green, with white foam where the waves break against the rocks. The sky is a light gray, with a few white clouds dotting the horizon.

Input prompt

Default: "The waves crash against the jagged rocks of the shoreline. The waves crash against the jagged rocks of the shoreline, sending spray high into the air.The rocks are a dark gray color, with sharp edges and deep crevices. The water is a clear blue-green, with white foam where the waves break against the rocks. The sky is a light gray, with a few white clouds dotting the horizon."

negative_prompt

string

Shift + Return to add a new line

worst quality, inconsistent motion, blurry, jittery, distortedworst quality, inconsistent motion, blurry, jittery, distorted

Negative prompt for undesired features

Default: "worst quality, inconsistent motion, blurry, jittery, distorted"

image

file

Optional, input image for image-to-video generation

width

integer

(maximum: 1280)

Width of the output video frames. Optional if an input image provided

Default: 704

height

integer

(maximum: 720)

Height of the output video frames. Optional if an input image provided

Default: 480

num_frames

integer

(maximum: 257)

Number of frames to generate in the output video

Default: 121

frame_rate

integer

Frame rate for the output video

Default: 25

num_inference_steps

integer

(minimum: 1)

Number of denoising steps

Default: 40

guidance_scale

number

(minimum: 1, maximum: 20)

Scale for classifier-free guidance

Default: 3

seed

integer

Random seed. Leave blank to randomize the seed

Run this model in Node.js with one line of code:

npx create-replicate --model=chenxwh/ltx-video

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";
import fs from "node:fs";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/ltx-video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/ltx-video:69599cebad125acfd3d5c682c187702ea7a84d537603e46b468a44fe94c5fd13",
  {
    input: {
      width: 704,
      height: 480,
      prompt: "The waves crash against the jagged rocks of the shoreline. The waves crash against the jagged rocks of the shoreline, sending spray high into the air.The rocks are a dark gray color, with sharp edges and deep crevices. The water is a clear blue-green, with white foam where the waves break against the rocks. The sky is a light gray, with a few white clouds dotting the horizon.",
      frame_rate: 25,
      num_frames: 121,
      guidance_scale: 3,
      negative_prompt: "worst quality, inconsistent motion, blurry, jittery, distorted",
      num_inference_steps: 40
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run chenxwh/ltx-video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/ltx-video:69599cebad125acfd3d5c682c187702ea7a84d537603e46b468a44fe94c5fd13",
    input={
        "width": 704,
        "height": 480,
        "prompt": "The waves crash against the jagged rocks of the shoreline. The waves crash against the jagged rocks of the shoreline, sending spray high into the air.The rocks are a dark gray color, with sharp edges and deep crevices. The water is a clear blue-green, with white foam where the waves break against the rocks. The sky is a light gray, with a few white clouds dotting the horizon.",
        "frame_rate": 25,
        "num_frames": 121,
        "guidance_scale": 3,
        "negative_prompt": "worst quality, inconsistent motion, blurry, jittery, distorted",
        "num_inference_steps": 40
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run chenxwh/ltx-video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/ltx-video:69599cebad125acfd3d5c682c187702ea7a84d537603e46b468a44fe94c5fd13",
    "input": {
      "width": 704,
      "height": 480,
      "prompt": "The waves crash against the jagged rocks of the shoreline. The waves crash against the jagged rocks of the shoreline, sending spray high into the air.The rocks are a dark gray color, with sharp edges and deep crevices. The water is a clear blue-green, with white foam where the waves break against the rocks. The sky is a light gray, with a few white clouds dotting the horizon.",
      "frame_rate": 25,
      "num_frames": 121,
      "guidance_scale": 3,
      "negative_prompt": "worst quality, inconsistent motion, blurry, jittery, distorted",
      "num_inference_steps": 40
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-11-24T15:01:47.107343Z",
  "created_at": "2024-11-24T14:53:10.803000Z",
  "data_removed": false,
  "error": null,
  "id": "4dksgm4f2drgm0ckbtmvay1qer",
  "input": {
    "width": 704,
    "height": 480,
    "prompt": "The waves crash against the jagged rocks of the shoreline. The waves crash against the jagged rocks of the shoreline, sending spray high into the air.The rocks are a dark gray color, with sharp edges and deep crevices. The water is a clear blue-green, with white foam where the waves break against the rocks. The sky is a light gray, with a few white clouds dotting the horizon.",
    "frame_rate": 25,
    "num_frames": 121,
    "guidance_scale": 3,
    "negative_prompt": "worst quality, inconsistent motion, blurry, jittery, distorted",
    "num_inference_steps": 40
  },
  "logs": "Using seed: 27317\nSetting `clean_caption=True` requires the Beautiful Soup library but it was not found in your environment. You can install it with pip:\n`pip install beautifulsoup4`. Please note that you may need to restart your runtime after installation.\nSetting `clean_caption` to False...\nSetting `clean_caption=True` requires the Beautiful Soup library but it was not found in your environment. You can install it with pip:\n`pip install beautifulsoup4`. Please note that you may need to restart your runtime after installation.\nSetting `clean_caption` to False...\n  0%|          | 0/40 [00:00<?, ?it/s]/root/.pyenv/versions/3.11.10/lib/python3.11/site-packages/torch/functional.py:534: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3595.)\nreturn _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]\n  2%|▎         | 1/40 [00:00<00:29,  1.32it/s]\n  5%|▌         | 2/40 [00:01<00:20,  1.83it/s]\n  8%|▊         | 3/40 [00:01<00:22,  1.63it/s]\n 10%|█         | 4/40 [00:02<00:23,  1.55it/s]\n 12%|█▎        | 5/40 [00:03<00:23,  1.51it/s]\n 15%|█▌        | 6/40 [00:03<00:22,  1.49it/s]\n 18%|█▊        | 7/40 [00:04<00:22,  1.48it/s]\n 20%|██        | 8/40 [00:05<00:21,  1.47it/s]\n 22%|██▎       | 9/40 [00:05<00:21,  1.46it/s]\n 25%|██▌       | 10/40 [00:06<00:20,  1.46it/s]\n 28%|██▊       | 11/40 [00:07<00:19,  1.46it/s]\n 30%|███       | 12/40 [00:08<00:19,  1.46it/s]\n 32%|███▎      | 13/40 [00:08<00:18,  1.45it/s]\n 35%|███▌      | 14/40 [00:09<00:17,  1.45it/s]\n 38%|███▊      | 15/40 [00:10<00:17,  1.45it/s]\n 40%|████      | 16/40 [00:10<00:16,  1.45it/s]\n 42%|████▎     | 17/40 [00:11<00:15,  1.45it/s]\n 45%|████▌     | 18/40 [00:12<00:15,  1.45it/s]\n 48%|████▊     | 19/40 [00:12<00:14,  1.45it/s]\n 50%|█████     | 20/40 [00:13<00:13,  1.45it/s]\n 52%|█████▎    | 21/40 [00:14<00:13,  1.45it/s]\n 55%|█████▌    | 22/40 [00:14<00:12,  1.44it/s]\n 57%|█████▊    | 23/40 [00:15<00:11,  1.44it/s]\n 60%|██████    | 24/40 [00:16<00:11,  1.44it/s]\n 62%|██████▎   | 25/40 [00:17<00:10,  1.44it/s]\n 65%|██████▌   | 26/40 [00:17<00:09,  1.44it/s]\n 68%|██████▊   | 27/40 [00:18<00:09,  1.44it/s]\n 70%|███████   | 28/40 [00:19<00:08,  1.44it/s]\n 72%|███████▎  | 29/40 [00:19<00:07,  1.44it/s]\n 75%|███████▌  | 30/40 [00:20<00:06,  1.44it/s]\n 78%|███████▊  | 31/40 [00:21<00:06,  1.44it/s]\n 80%|████████  | 32/40 [00:21<00:05,  1.44it/s]\n 82%|████████▎ | 33/40 [00:22<00:04,  1.44it/s]\n 85%|████████▌ | 34/40 [00:23<00:04,  1.44it/s]\n 88%|████████▊ | 35/40 [00:23<00:03,  1.44it/s]\n 90%|█████████ | 36/40 [00:24<00:02,  1.44it/s]\n 92%|█████████▎| 37/40 [00:25<00:02,  1.44it/s]\n 95%|█████████▌| 38/40 [00:26<00:01,  1.44it/s]\n 98%|█████████▊| 39/40 [00:26<00:00,  1.44it/s]\n100%|██████████| 40/40 [00:27<00:00,  1.44it/s]\n100%|██████████| 40/40 [00:27<00:00,  1.46it/s]\n/src/ltx_video/pipelines/pipeline_ltx_video.py:1073: FutureWarning: Accessing config attribute `in_channels` directly via 'Transformer3DModel' object attribute is deprecated. Please access 'in_channels' over 'Transformer3DModel's config object instead, e.g. 'unet.config.in_channels'.\nout_channels=self.transformer.in_channels",
  "metrics": {
    "predict_time": 31.15204648,
    "total_time": 516.304343
  },
  "output": "https://replicate.delivery/pbxt/FyGCkk6CAeRGDaBhXu1ePcbHDEZulZ7P3Oy99EoxfpnzOeQPB/out.mp4",
  "started_at": "2024-11-24T15:01:15.955296Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/4dksgm4f2drgm0ckbtmvay1qer",
    "cancel": "https://api.replicate.com/v1/predictions/4dksgm4f2drgm0ckbtmvay1qer/cancel"
  },
  "version": "69599cebad125acfd3d5c682c187702ea7a84d537603e46b468a44fe94c5fd13"
}

Generated in

31.2 seconds

Tweak it ShareReport View full prediction

Using seed: 27317
Setting `clean_caption=True` requires the Beautiful Soup library but it was not found in your environment. You can install it with pip:
`pip install beautifulsoup4`. Please note that you may need to restart your runtime after installation.
Setting `clean_caption` to False...
Setting `clean_caption=True` requires the Beautiful Soup library but it was not found in your environment. You can install it with pip:
`pip install beautifulsoup4`. Please note that you may need to restart your runtime after installation.
Setting `clean_caption` to False...
  0%|          | 0/40 [00:00<?, ?it/s]/root/.pyenv/versions/3.11.10/lib/python3.11/site-packages/torch/functional.py:534: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3595.)
return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
  2%|▎         | 1/40 [00:00<00:29,  1.32it/s]
  5%|▌         | 2/40 [00:01<00:20,  1.83it/s]
  8%|▊         | 3/40 [00:01<00:22,  1.63it/s]
 10%|█         | 4/40 [00:02<00:23,  1.55it/s]
 12%|█▎        | 5/40 [00:03<00:23,  1.51it/s]
 15%|█▌        | 6/40 [00:03<00:22,  1.49it/s]
 18%|█▊        | 7/40 [00:04<00:22,  1.48it/s]
 20%|██        | 8/40 [00:05<00:21,  1.47it/s]
 22%|██▎       | 9/40 [00:05<00:21,  1.46it/s]
 25%|██▌       | 10/40 [00:06<00:20,  1.46it/s]
 28%|██▊       | 11/40 [00:07<00:19,  1.46it/s]
 30%|███       | 12/40 [00:08<00:19,  1.46it/s]
 32%|███▎      | 13/40 [00:08<00:18,  1.45it/s]
 35%|███▌      | 14/40 [00:09<00:17,  1.45it/s]
 38%|███▊      | 15/40 [00:10<00:17,  1.45it/s]
 40%|████      | 16/40 [00:10<00:16,  1.45it/s]
 42%|████▎     | 17/40 [00:11<00:15,  1.45it/s]
 45%|████▌     | 18/40 [00:12<00:15,  1.45it/s]
 48%|████▊     | 19/40 [00:12<00:14,  1.45it/s]
 50%|█████     | 20/40 [00:13<00:13,  1.45it/s]
 52%|█████▎    | 21/40 [00:14<00:13,  1.45it/s]
 55%|█████▌    | 22/40 [00:14<00:12,  1.44it/s]
 57%|█████▊    | 23/40 [00:15<00:11,  1.44it/s]
 60%|██████    | 24/40 [00:16<00:11,  1.44it/s]
 62%|██████▎   | 25/40 [00:17<00:10,  1.44it/s]
 65%|██████▌   | 26/40 [00:17<00:09,  1.44it/s]
 68%|██████▊   | 27/40 [00:18<00:09,  1.44it/s]
 70%|███████   | 28/40 [00:19<00:08,  1.44it/s]
 72%|███████▎  | 29/40 [00:19<00:07,  1.44it/s]
 75%|███████▌  | 30/40 [00:20<00:06,  1.44it/s]
 78%|███████▊  | 31/40 [00:21<00:06,  1.44it/s]
 80%|████████  | 32/40 [00:21<00:05,  1.44it/s]
 82%|████████▎ | 33/40 [00:22<00:04,  1.44it/s]
 85%|████████▌ | 34/40 [00:23<00:04,  1.44it/s]
 88%|████████▊ | 35/40 [00:23<00:03,  1.44it/s]
 90%|█████████ | 36/40 [00:24<00:02,  1.44it/s]
 92%|█████████▎| 37/40 [00:25<00:02,  1.44it/s]
 95%|█████████▌| 38/40 [00:26<00:01,  1.44it/s]
 98%|█████████▊| 39/40 [00:26<00:00,  1.44it/s]
100%|██████████| 40/40 [00:27<00:00,  1.44it/s]
100%|██████████| 40/40 [00:27<00:00,  1.46it/s]
/src/ltx_video/pipelines/pipeline_ltx_video.py:1073: FutureWarning: Accessing config attribute `in_channels` directly via 'Transformer3DModel' object attribute is deprecated. Please access 'in_channels' over 'Transformer3DModel's config object instead, e.g. 'unet.config.in_channels'.
out_channels=self.transformer.in_channels

Examples

View more examples

Run time and cost

This model costs approximately $0.012 to run on Replicate, or 83 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 13 seconds. The predict time for this model varies significantly based on the inputs.

Readme

LTX-Video

Introduction

LTX-Video is the first DiT-based video generation model that can generate high-quality videos in real-time. It can generate 24 FPS videos at 768x512 resolution, faster than it takes to watch them. The model is trained on a large-scale dataset of diverse videos and can generate high-resolution videos with realistic and diverse content.## More to come…

Acknowledgement

We are grateful for the following awesome projects when implementing LTX-Video: * DiT and PixArt-alpha: vision transformers for image generation.