cjwbw/unival – Run with an API on Replicate

cjwbw / unival

Unified Model for Image, Video, Audio and Language Tasks

Cold

Public
990 runs
L40S
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=cjwbw/unival

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run cjwbw/unival using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "cjwbw/unival:00a9af2b0889db2a73f5002b3a3000ba4eec8c5fed0cb4fd842a3fc75ab0e98f",
  {
    input: {
      task_type: "Visual Grounding",
      input_image: "https://replicate.delivery/pbxt/JKlg8Ru5XgAnr03vzxqrP5V9ztpyfYuaNpUm0hVXs0XEKmu1/1.png",
      instruction: "detached banana"
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run cjwbw/unival using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "cjwbw/unival:00a9af2b0889db2a73f5002b3a3000ba4eec8c5fed0cb4fd842a3fc75ab0e98f",
    input={
        "task_type": "Visual Grounding",
        "input_image": "https://replicate.delivery/pbxt/JKlg8Ru5XgAnr03vzxqrP5V9ztpyfYuaNpUm0hVXs0XEKmu1/1.png",
        "instruction": "detached banana"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run cjwbw/unival using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "cjwbw/unival:00a9af2b0889db2a73f5002b3a3000ba4eec8c5fed0cb4fd842a3fc75ab0e98f",
    "input": {
      "task_type": "Visual Grounding",
      "input_image": "https://replicate.delivery/pbxt/JKlg8Ru5XgAnr03vzxqrP5V9ztpyfYuaNpUm0hVXs0XEKmu1/1.png",
      "instruction": "detached banana"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Generated in

2.3 seconds

Tweak itReport

This output was created using a different version of the model, cjwbw/unival:b2c88629.

Examples

View more examples

Run time and cost

This model costs approximately $0.0034 to run on Replicate, or 294 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 4 seconds. The predict time for this model varies significantly based on the inputs.

Readme

UniVAL: Unified Model for Image, Video, Audio and Language Tasks

UnIVAL is a 0.25B-parameter unified model that is multitask pretrained on image and video-text data and target image, video and audio-text downstream tasks.

Citation

If you find the work helpful, you can cite it using the following citation:

@article{shukor2023unified,
  title={Unified Model for Image, Video, Audio and Language Tasks},
  author={Shukor, Mustafa and Dancette, Corentin and Rame, Alexandre and Cord, Matthieu},
  journal={arXiv preprint arXiv:2307.16184},
  year={2023}
}

Aknowledgment

This code is based mainly on the following repos:

We thank the authors for releasing their code.