Versions
Failed to load versions. Head to the versions page to see all versions for this model.
You're looking at a specific version of this model. Jump to the model overview.
camenduru /streaming-t2v:ce9e0977
Input
Run this model in Node.js with one line of code:
npx create-replicate --model=camenduru/streaming-t2v
or set up a project from scratch
Install Replicate’s Node.js client library:
npm install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import and set up the client:
import Replicate from "replicate";
const replicate = new Replicate({
auth: process.env.REPLICATE_API_TOKEN,
});Run camenduru/streaming-t2v using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run(
"camenduru/streaming-t2v:ce9e09771a164b46913a7006bfa6c602735d82b748e6e433acc27103fc973b96",
{
input: {
seed: 33,
prompt: "Dive into the depths of the ocean: explore vibrant coral reefs, mysterious underwater caves, and the mesmerizing creatures that call the sea home.",
enhance: true,
num_steps: 50,
num_frames: 56,
image_guidance: 9,
negative_prompt: ""
}
}
);
// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"
// To write the file to disk:
fs.writeFile("my-image.png", output[0]);To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:
pip install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import the client:
import replicateRun camenduru/streaming-t2v using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run(
"camenduru/streaming-t2v:ce9e09771a164b46913a7006bfa6c602735d82b748e6e433acc27103fc973b96",
input={
"seed": 33,
"prompt": "Dive into the depths of the ocean: explore vibrant coral reefs, mysterious underwater caves, and the mesmerizing creatures that call the sea home.",
"enhance": True,
"num_steps": 50,
"num_frames": 56,
"image_guidance": 9,
"negative_prompt": ""
}
)
print(output)To learn more, take a look at the guide on getting started with Python.
Set the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Run camenduru/streaming-t2v using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
-H "Content-Type: application/json" \
-H "Prefer: wait" \
-d $'{
"version": "camenduru/streaming-t2v:ce9e09771a164b46913a7006bfa6c602735d82b748e6e433acc27103fc973b96",
"input": {
"seed": 33,
"prompt": "Dive into the depths of the ocean: explore vibrant coral reefs, mysterious underwater caves, and the mesmerizing creatures that call the sea home.",
"enhance": true,
"num_steps": 50,
"num_frames": 56,
"image_guidance": 9,
"negative_prompt": ""
}
}' \
https://api.replicate.com/v1/predictionsTo learn more, take a look at Replicate’s HTTP API reference docs.
Add a payment method to run this model.
Sign in with GitHub
By signing in, you agree to our
terms of service and privacy policy
Output
{
"completed_at": "2024-04-10T04:37:34.217029Z",
"created_at": "2024-04-10T04:30:01.848000Z",
"data_removed": false,
"error": null,
"id": "gkfgk6rwf1rgp0cers2sk3s09r",
"input": {
"seed": 33,
"prompt": "Dive into the depths of the ocean: explore vibrant coral reefs, mysterious underwater caves, and the mesmerizing creatures that call the sea home.",
"enhance": true,
"num_steps": 50,
"num_frames": 56,
"image_guidance": 9,
"negative_prompt": ""
},
"logs": "You are using a CUDA device ('NVIDIA A100-SXM4-40GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision\nLOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n/usr/local/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:442: PossibleUserWarning: The dataloader, predict_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 128 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\nrank_zero_warn(\nINFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}\nINFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}\nINFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}\nINFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}\nINFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}\n/usr/local/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py:234: UserWarning: predict returned None if it was on purpose, ignore this warning...\nself._warning_cache.warn(\"predict returned None if it was on purpose, ignore this warning...\")\nPredicting ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 0:02:23 • 0:00:00 0.00it/s\n2024-04-10 04:35:05,319 - modelscope - WARNING - task video-to-video input definition is missing\n/usr/local/lib/python3.10/site-packages/torchsde/_brownian/brownian_interval.py:608: UserWarning: Should have tb<=t1 but got tb=4.164773464202881 and t1=4.164773.\nwarnings.warn(f\"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.\")\n2024-04-10 04:37:31,267 - modelscope - WARNING - task video-to-video output keys are missing",
"metrics": {
"predict_time": 309.690938,
"total_time": 452.369029
},
"output": [
"https://replicate.delivery/pbxt/Ine4fu7JgsrLwEYeec5Xb6UqcyjAZgBJ7IEzYRq3uz40YyjKB/output.mp4",
"https://replicate.delivery/pbxt/wnuzP6LIHP4QBlw95wlrsqG7xsfzQBDPmLcnycUcN96GTeoSA/output_enhanced.mp4"
],
"started_at": "2024-04-10T04:32:24.526091Z",
"status": "succeeded",
"urls": {
"get": "https://api.replicate.com/v1/predictions/gkfgk6rwf1rgp0cers2sk3s09r",
"cancel": "https://api.replicate.com/v1/predictions/gkfgk6rwf1rgp0cers2sk3s09r/cancel"
},
"version": "ce9e09771a164b46913a7006bfa6c602735d82b748e6e433acc27103fc973b96"
}Generated in
You are using a CUDA device ('NVIDIA A100-SXM4-40GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/usr/local/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:442: PossibleUserWarning: The dataloader, predict_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 128 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
rank_zero_warn(
INFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}
INFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}
INFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}
INFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}
INFERENCE PARAMS = {'concat_video': True, 'conditioning_from_all_past': False, 'conditioning_type': 'fixed', 'eta': 1.0, 'eval_loss_metrics': False, 'frame_rate': 8, 'guidance_scale': 7.5, 'height': 256, 'mode': 'long_video', 'n_autoregressive_generations': 4, 'negative_prompt': '', 'num_inference_steps': 50, 'result_formats': ['eval_mp4'], 'scheduler_cls': '', 'seed': 33, 'start_from_real_input': False, 'use_dec_scaling': True, 'validation_samples': 80, 'video_length': 16, 'width': 256}
/usr/local/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py:234: UserWarning: predict returned None if it was on purpose, ignore this warning...
self._warning_cache.warn("predict returned None if it was on purpose, ignore this warning...")
Predicting ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 0:02:23 • 0:00:00 0.00it/s
2024-04-10 04:35:05,319 - modelscope - WARNING - task video-to-video input definition is missing
/usr/local/lib/python3.10/site-packages/torchsde/_brownian/brownian_interval.py:608: UserWarning: Should have tb<=t1 but got tb=4.164773464202881 and t1=4.164773.
warnings.warn(f"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.")
2024-04-10 04:37:31,267 - modelscope - WARNING - task video-to-video output keys are missing