chenxwh/depthcrafter | Run with an API on Replicate

Input

video

*file

Input video

num_denoising_steps

integer

(minimum: 1, maximum: 25)

Number of denoising steps

Default: 10

guidance_scale

number

(minimum: 1, maximum: 1.2)

Scale for classifier-free guidance

Default: 1.2

max_res

integer

Maximum resolution

Default: 1024

process_length

integer

(minimum: 1, maximum: 280)

Number of frames to process

Default: 60

target_fps

integer

fps of the output video

Default: 15

window_size

integer

Window size

Default: 110

overlap

integer

Overlap size

Default: 15

save_npz

boolean

Save npz file

Default: false

datast

string

Assigned resolution for specific dataset evaluation

Default: "open"

seed

integer

Random seed. Leave blank to randomize the seed

Run this model in Node.js with one line of code:

npx create-replicate --model=chenxwh/depthcrafter

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/depthcrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/depthcrafter:1c9bfc62fd0750ddadeccf1af4cdef4d74a8a064debd8152ade87f3c9dc30d2b",
  {
    input: {
      video: "https://replicate.delivery/pbxt/LiSKH5N17Dpsep8CEGdvmzx8uxQjvtArSYJXpB6USd5ScQlC/example_01.mp4",
      datast: "open",
      max_res: 1024,
      overlap: 15,
      save_npz: false,
      target_fps: 15,
      window_size: 110,
      guidance_scale: 1.2,
      process_length: 60,
      num_denoising_steps: 10
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run chenxwh/depthcrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/depthcrafter:1c9bfc62fd0750ddadeccf1af4cdef4d74a8a064debd8152ade87f3c9dc30d2b",
    input={
        "video": "https://replicate.delivery/pbxt/LiSKH5N17Dpsep8CEGdvmzx8uxQjvtArSYJXpB6USd5ScQlC/example_01.mp4",
        "datast": "open",
        "max_res": 1024,
        "overlap": 15,
        "save_npz": False,
        "target_fps": 15,
        "window_size": 110,
        "guidance_scale": 1.2,
        "process_length": 60,
        "num_denoising_steps": 10
    }
)

print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run chenxwh/depthcrafter using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/depthcrafter:1c9bfc62fd0750ddadeccf1af4cdef4d74a8a064debd8152ade87f3c9dc30d2b",
    "input": {
      "video": "https://replicate.delivery/pbxt/LiSKH5N17Dpsep8CEGdvmzx8uxQjvtArSYJXpB6USd5ScQlC/example_01.mp4",
      "datast": "open",
      "max_res": 1024,
      "overlap": 15,
      "save_npz": false,
      "target_fps": 15,
      "window_size": 110,
      "guidance_scale": 1.2,
      "process_length": 60,
      "num_denoising_steps": 10
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-10-01T00:18:38.636578Z",
  "created_at": "2024-10-01T00:15:38.406000Z",
  "data_removed": false,
  "error": null,
  "id": "w88pztdxmsrgj0cj8nq9yz034g",
  "input": {
    "video": "https://replicate.delivery/pbxt/LiSKH5N17Dpsep8CEGdvmzx8uxQjvtArSYJXpB6USd5ScQlC/example_01.mp4",
    "datast": "open",
    "max_res": 1024,
    "overlap": 15,
    "save_npz": false,
    "target_fps": 15,
    "window_size": 110,
    "guidance_scale": 1.2,
    "process_length": 60,
    "num_denoising_steps": 10
  },
  "logs": "Using seed: 7396\n==> frames shape: (60, 576, 1024, 3)\nElapsed time for encoding video: 9502.76953125 ms\n  0%|          | 0/10 [00:00<?, ?it/s]\n 10%|█         | 1/10 [00:05<00:49,  5.55s/it]\n 20%|██        | 2/10 [00:12<00:50,  6.26s/it]\n 30%|███       | 3/10 [00:19<00:45,  6.49s/it]\n 40%|████      | 4/10 [00:25<00:39,  6.59s/it]\n 50%|█████     | 5/10 [00:32<00:33,  6.66s/it]\n 60%|██████    | 6/10 [00:39<00:26,  6.70s/it]\n 70%|███████   | 7/10 [00:46<00:20,  6.73s/it]\n 80%|████████  | 8/10 [00:52<00:13,  6.75s/it]\n 90%|█████████ | 9/10 [00:59<00:06,  6.77s/it]\n100%|██████████| 10/10 [01:06<00:00,  6.78s/it]\n100%|██████████| 10/10 [01:06<00:00,  6.66s/it]\nElapsed time for denoising video: 68187.703125 ms\nElapsed time for decoding video: 17718.001953125 ms",
  "metrics": {
    "predict_time": 97.857017597,
    "total_time": 180.230578
  },
  "output": {
    "npz": null,
    "depth_video": "https://replicate.delivery/pbxt/GMO4aeo2ykxcVanVyeWMne4EiYiJvok8s1dbZEBuS1E8OeIOB/out.mp4"
  },
  "started_at": "2024-10-01T00:17:00.779561Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/w88pztdxmsrgj0cj8nq9yz034g",
    "cancel": "https://api.replicate.com/v1/predictions/w88pztdxmsrgj0cj8nq9yz034g/cancel"
  },
  "version": "1c9bfc62fd0750ddadeccf1af4cdef4d74a8a064debd8152ade87f3c9dc30d2b"
}

Generated in

1 minute 38 seconds

Tweak it Share Report View full prediction

Examples

View more examples

Run time and cost

This model runs on Nvidia L40S GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos

🤗 DepthCrafter can generate temporally consistent long-depth sequences with fine-grained details for open-world videos, without requiring additional information such as camera poses or optical flow.

📜 Citation

If you find this work helpful, please consider citing:

@article{hu2024-DepthCrafter,
            author      = {Hu, Wenbo and Gao, Xiangjun and Li, Xiaoyu and Zhao, Sijie and Cun, Xiaodong and Zhang, Yong and Quan, Long and Shan, Ying},
            title       = {DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos},
            journal     = {arXiv preprint arXiv:2409.02095},
            year        = {2024}
    }