camenduru/minigpt4-video:56793424 | Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

camenduru /minigpt4-video:56793424

Playground API

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=camenduru/minigpt4-video

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run camenduru/minigpt4-video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "camenduru/minigpt4-video:5679342473d4fd99cf75e140a403e6463f8d5cdc324525783e0d7e35cf27f68b",
  {
    input: {
      question: "What's this video talking about?",
      video_path: "https://replicate.delivery/pbxt/Ki1Xy9IzGUX16CXvlMU1f9VYq89OpJk7hihhBR0CjScxp6so/Great%20white%20shark%20swims%20into%20cage.mp4",
      add_subtitles: false
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run camenduru/minigpt4-video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "camenduru/minigpt4-video:5679342473d4fd99cf75e140a403e6463f8d5cdc324525783e0d7e35cf27f68b",
    input={
        "question": "What's this video talking about?",
        "video_path": "https://replicate.delivery/pbxt/Ki1Xy9IzGUX16CXvlMU1f9VYq89OpJk7hihhBR0CjScxp6so/Great%20white%20shark%20swims%20into%20cage.mp4",
        "add_subtitles": False
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run camenduru/minigpt4-video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "camenduru/minigpt4-video:5679342473d4fd99cf75e140a403e6463f8d5cdc324525783e0d7e35cf27f68b",
    "input": {
      "question": "What\'s this video talking about?",
      "video_path": "https://replicate.delivery/pbxt/Ki1Xy9IzGUX16CXvlMU1f9VYq89OpJk7hihhBR0CjScxp6so/Great%20white%20shark%20swims%20into%20cage.mp4",
      "add_subtitles": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

This video is about a man and sharks in the ocean. The main focus of this footage captures close-up images of one large tiger shark that swims up to observe him from beneath, as well as other scuba divers interacting with these animals underwater while being observed by both themselves and others on land or sea level through telescopes

{
  "completed_at": "2024-04-08T02:59:09.100880Z",
  "created_at": "2024-04-08T02:53:36.016000Z",
  "data_removed": false,
  "error": null,
  "id": "rj1a0cgvj1rgg0ceqegavhrm84",
  "input": {
    "question": "What's this video talking about?",
    "video_path": "https://replicate.delivery/pbxt/Ki1Xy9IzGUX16CXvlMU1f9VYq89OpJk7hihhBR0CjScxp6so/Great%20white%20shark%20swims%20into%20cage.mp4",
    "add_subtitles": false
  },
  "logs": "WARNING:py.warnings:/usr/local/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\nwarnings.warn(\nWARNING:py.warnings:/usr/local/lib/python3.10/site-packages/torch/utils/checkpoint.py:61: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\nwarnings.warn(\nimg_embeds shape torch.Size([45, 64, 4096])\ninputs_embeds shape torch.Size([1, 3034, 4096])\nattention_mask shape torch.Size([1, 3034])",
  "metrics": {
    "predict_time": 14.426456,
    "total_time": 333.08488
  },
  "output": "This video is about a man and sharks in the ocean. The main focus of this footage captures close-up images of one large tiger shark that swims up to observe him from beneath, as well as other scuba divers interacting with these animals underwater while being observed by both themselves and others on land or sea level through telescopes",
  "started_at": "2024-04-08T02:58:54.674424Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/rj1a0cgvj1rgg0ceqegavhrm84",
    "cancel": "https://api.replicate.com/v1/predictions/rj1a0cgvj1rgg0ceqegavhrm84/cancel"
  },
  "version": "5679342473d4fd99cf75e140a403e6463f8d5cdc324525783e0d7e35cf27f68b"
}

Generated in

14.4 seconds

Tweak it Share Report View full prediction