pku-yuangroup/llava-cot:40c17578 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

pku-yuangroup /llava-cot:40c17578

Playground API

Input

prompt

string

Shift + Return to add a new line

Subtract all tiny shiny balls. Subtract all red objects. How many objects are left?Subtract all tiny shiny balls. Subtract all red objects. How many objects are left?

Text prompt

Default: "If I had to write a haiku for this one, it would be: "

image

*file

Grayscale input image

max_new_tokens

integer

Max number of generated tokens

Default: 1024

temperature

number

(minimum: 0, maximum: 5)

Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.

Default: 0.9

top_p

number

(minimum: 0, maximum: 1)

When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens, used when temperature > 0

Default: 0.95

Run this model in Node.js with one line of code:

npx create-replicate --model=pku-yuangroup/llava-cot

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run pku-yuangroup/llava-cot using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "pku-yuangroup/llava-cot:40c17578a6b98679fc08e1a92fcef1b8000f857bf300a381091a13a67c28754d",
  {
    input: {
      image: "https://replicate.delivery/pbxt/M4VHTaDOClDN7YpKvnlHEqJdyahSXz2losoZE2V5qZnbrx77/teaser_crop.jpg",
      top_p: 0.95,
      prompt: "Subtract all tiny shiny balls. Subtract all red objects. How many objects are left?",
      temperature: 0,
      max_new_tokens: 1024
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run pku-yuangroup/llava-cot using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "pku-yuangroup/llava-cot:40c17578a6b98679fc08e1a92fcef1b8000f857bf300a381091a13a67c28754d",
    input={
        "image": "https://replicate.delivery/pbxt/M4VHTaDOClDN7YpKvnlHEqJdyahSXz2losoZE2V5qZnbrx77/teaser_crop.jpg",
        "top_p": 0.95,
        "prompt": "Subtract all tiny shiny balls. Subtract all red objects. How many objects are left?",
        "temperature": 0,
        "max_new_tokens": 1024
    }
)

# The pku-yuangroup/llava-cot model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/pku-yuangroup/llava-cot/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run pku-yuangroup/llava-cot using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "pku-yuangroup/llava-cot:40c17578a6b98679fc08e1a92fcef1b8000f857bf300a381091a13a67c28754d",
    "input": {
      "image": "https://replicate.delivery/pbxt/M4VHTaDOClDN7YpKvnlHEqJdyahSXz2losoZE2V5qZnbrx77/teaser_crop.jpg",
      "top_p": 0.95,
      "prompt": "Subtract all tiny shiny balls. Subtract all red objects. How many objects are left?",
      "temperature": 0,
      "max_new_tokens": 1024
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

<SUMMARY> I will identify and count all the objects in the image, then subtract the tiny shiny balls and red objects to determine how many remain. </SUMMARY> <CAPTION> The image contains various 3D shapes, including spheres, cubes, and cylinders. The objects are of different colors: red, gold, blue, green, and teal. There are multiple shiny spheres and a red cylinder. </CAPTION> <REASONING> First, I will count all the objects in the image. There are several spheres, cubes, and cylinders. Next, I will identify and subtract the tiny shiny balls. There are two shiny gold spheres that fit this description. Then, I will subtract all red objects. There is one red sphere and one red cylinder. After subtracting these objects, I will count the remaining objects. </REASONING> <CONCLUSION> 5 </CONCLUSION>

{
  "completed_at": "2024-12-02T02:53:35.456557Z",
  "created_at": "2024-12-02T02:53:26.672000Z",
  "data_removed": false,
  "error": null,
  "id": "g6gx0ykz21rma0ckgn4rxzxsdw",
  "input": {
    "image": "https://replicate.delivery/pbxt/M4VHTaDOClDN7YpKvnlHEqJdyahSXz2losoZE2V5qZnbrx77/teaser_crop.jpg",
    "top_p": 0.95,
    "prompt": "Subtract all tiny shiny balls. Subtract all red objects. How many objects are left?",
    "temperature": 0,
    "max_new_tokens": 1024
  },
  "logs": "/root/.pyenv/versions/3.11.10/lib/python3.11/site-packages/transformers/generation/configuration_utils.py:590: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\nwarnings.warn(\n/root/.pyenv/versions/3.11.10/lib/python3.11/site-packages/transformers/generation/configuration_utils.py:595: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.95` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.\nwarnings.warn(",
  "metrics": {
    "predict_time": 8.77747611,
    "total_time": 8.784557
  },
  "output": [
    "",
    "",
    "",
    "",
    "<SUMMARY> ",
    "I ",
    "will ",
    "identify ",
    "and ",
    "count ",
    "all ",
    "the ",
    "objects ",
    "in ",
    "the ",
    "",
    "image, ",
    "then ",
    "subtract ",
    "the ",
    "tiny ",
    "shiny ",
    "balls ",
    "and ",
    "red ",
    "objects ",
    "to ",
    "determine ",
    "how ",
    "many ",
    "",
    "remain. ",
    "",
    "",
    "</SUMMARY>\n\n",
    "",
    "",
    "",
    "",
    "<CAPTION> ",
    "The ",
    "image ",
    "contains ",
    "various ",
    "",
    "",
    "3D ",
    "",
    "shapes, ",
    "including ",
    "",
    "spheres, ",
    "",
    "cubes, ",
    "and ",
    "",
    "cylinders. ",
    "The ",
    "objects ",
    "are ",
    "of ",
    "different ",
    "",
    "colors: ",
    "",
    "red, ",
    "",
    "gold, ",
    "",
    "blue, ",
    "",
    "green, ",
    "and ",
    "",
    "teal. ",
    "There ",
    "are ",
    "multiple ",
    "shiny ",
    "spheres ",
    "and ",
    "a ",
    "red ",
    "",
    "cylinder. ",
    "",
    "",
    "</CAPTION>\n\n",
    "",
    "",
    "",
    "",
    "",
    "<REASONING> ",
    "",
    "First, ",
    "I ",
    "will ",
    "count ",
    "all ",
    "the ",
    "objects ",
    "in ",
    "the ",
    "",
    "image. ",
    "There ",
    "are ",
    "several ",
    "",
    "spheres, ",
    "",
    "cubes, ",
    "and ",
    "",
    "cylinders. ",
    "",
    "Next, ",
    "I ",
    "will ",
    "identify ",
    "and ",
    "subtract ",
    "the ",
    "tiny ",
    "shiny ",
    "",
    "balls. ",
    "There ",
    "are ",
    "two ",
    "shiny ",
    "gold ",
    "spheres ",
    "that ",
    "fit ",
    "this ",
    "",
    "description. ",
    "",
    "Then, ",
    "I ",
    "will ",
    "subtract ",
    "all ",
    "red ",
    "",
    "objects. ",
    "There ",
    "is ",
    "one ",
    "red ",
    "sphere ",
    "and ",
    "one ",
    "red ",
    "",
    "cylinder. ",
    "After ",
    "",
    "subtracting ",
    "these ",
    "",
    "objects, ",
    "I ",
    "will ",
    "count ",
    "the ",
    "remaining ",
    "",
    "objects. ",
    "",
    "",
    "",
    "</REASONING>\n\n",
    "",
    "",
    "",
    "",
    "<CONCLUSION> ",
    "",
    "5 ",
    "",
    "",
    "",
    "",
    "</CONCLUSION>"
  ],
  "started_at": "2024-12-02T02:53:26.679081Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://stream-b.svc.ric1.c.replicate.net/v1/streams/u7pjit6mf47fhpxotna5edny4hlsraspyc24s6kqymkwv2s52bda",
    "get": "https://api.replicate.com/v1/predictions/g6gx0ykz21rma0ckgn4rxzxsdw",
    "cancel": "https://api.replicate.com/v1/predictions/g6gx0ykz21rma0ckgn4rxzxsdw/cancel"
  },
  "version": "40c17578a6b98679fc08e1a92fcef1b8000f857bf300a381091a13a67c28754d"
}

Generated in

8.8 seconds

Tweak it Share Report