mattt/orca-2-13b – Run with an API on Replicate

mattt / orca-2-13b

Cold

Public
1.5K runs
L40S

Run with an API

Playground API Examples README Versions

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=mattt/orca-2-13b

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run mattt/orca-2-13b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "mattt/orca-2-13b:dac511fd566228a0dd189e95318523fccb6ba175f9f0ecd29214b579b00b64c7",
  {
    input: {
      top_k: 50,
      top_p: 1,
      prompt: "Tell me some fun facts about orcas",
      temperature: 1,
      max_new_tokens: 256
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run mattt/orca-2-13b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "mattt/orca-2-13b:dac511fd566228a0dd189e95318523fccb6ba175f9f0ecd29214b579b00b64c7",
    input={
        "top_k": 50,
        "top_p": 1,
        "prompt": "Tell me some fun facts about orcas",
        "temperature": 1,
        "max_new_tokens": 256
    }
)

# The mattt/orca-2-13b model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/mattt/orca-2-13b/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run mattt/orca-2-13b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "mattt/orca-2-13b:dac511fd566228a0dd189e95318523fccb6ba175f9f0ecd29214b579b00b64c7",
    "input": {
      "top_k": 50,
      "top_p": 1,
      "prompt": "Tell me some fun facts about orcas",
      "temperature": 1,
      "max_new_tokens": 256
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

! 1. Orcas, also known as killer whales, are actually dolphins, not whales. 2. They are the largest members of the dolphin family, with males reaching up to 32 feet in length and weighing up 6 tons. 3. Orcas can live in a variety of environments, including oceans, seas, and even some freshwater rivers. 4. They are highly intelligent and social animals, often living in groups called pods. 5. There are different types of orcas, each with its own unique appearance and behavior. Some types include the resident, transient, and offshore orcas. 6. Orcas are known for their impressive hunting skills, often working together to take down large prey such as seals, sea lions, and even whales. 7. They are also considered apex predators, meaning they have no natural predators in their environment. 8. Orcas have powerful vocalizations, including clicks, whistles, and calls that they use for communication and echolocation. 9. Their lifespans can range from 50 to 90 years, with

{
  "completed_at": "2023-11-27T19:17:46.773736Z",
  "created_at": "2023-11-27T19:14:27.730879Z",
  "data_removed": false,
  "error": null,
  "id": "xmqy4u3b5vw4ex33okoyugi4j4",
  "input": {
    "top_k": 50,
    "top_p": 1,
    "prompt": "Tell me some fun facts about orcas",
    "temperature": 1,
    "max_new_tokens": 256
  },
  "logs": null,
  "metrics": {
    "predict_time": 21.131516,
    "total_time": 199.042857
  },
  "output": [
    "",
    "!\n",
    "\n",
    "",
    "",
    "1. ",
    "",
    "",
    "Orcas, ",
    "also ",
    "known ",
    "as ",
    "",
    "killer ",
    "",
    "",
    "whales, ",
    "are ",
    "actually ",
    "",
    "",
    "",
    "dolphins, ",
    "not ",
    "",
    "",
    "whales.\n",
    "",
    "",
    "2. ",
    "They ",
    "are ",
    "the ",
    "largest ",
    "members ",
    "of ",
    "the ",
    "",
    "",
    "dolphin ",
    "",
    "family, ",
    "with ",
    "males ",
    "reaching ",
    "up ",
    "to ",
    "",
    "",
    "32 ",
    "feet ",
    "in ",
    "length ",
    "and ",
    "",
    "",
    "weighing ",
    "up ",
    "",
    "6 ",
    "",
    "tons.\n",
    "",
    "",
    "3. ",
    "",
    "Orcas ",
    "can ",
    "live ",
    "in ",
    "a ",
    "variety ",
    "of ",
    "",
    "environments, ",
    "including ",
    "",
    "",
    "",
    "oceans, ",
    "",
    "",
    "seas, ",
    "and ",
    "even ",
    "some ",
    "",
    "freshwater ",
    "",
    "rivers.\n",
    "",
    "",
    "4. ",
    "They ",
    "are ",
    "highly ",
    "",
    "intelligent ",
    "and ",
    "social ",
    "",
    "animals, ",
    "often ",
    "living ",
    "in ",
    "groups ",
    "called ",
    "",
    "",
    "pods.\n",
    "",
    "",
    "5. ",
    "There ",
    "are ",
    "different ",
    "types ",
    "of ",
    "",
    "",
    "orcas, ",
    "each ",
    "with ",
    "its ",
    "own ",
    "unique ",
    "appearance ",
    "and ",
    "",
    "behavior. ",
    "Some ",
    "types ",
    "include ",
    "the ",
    "",
    "resident, ",
    "",
    "",
    "transient, ",
    "and ",
    "",
    "",
    "offshore ",
    "",
    "",
    "orcas.\n",
    "",
    "",
    "6. ",
    "",
    "Orcas ",
    "are ",
    "known ",
    "for ",
    "their ",
    "",
    "impressive ",
    "hunting ",
    "",
    "skills, ",
    "often ",
    "working ",
    "together ",
    "to ",
    "take ",
    "down ",
    "large ",
    "",
    "prey ",
    "such ",
    "as ",
    "",
    "",
    "seals, ",
    "sea ",
    "",
    "",
    "lions, ",
    "and ",
    "even ",
    "",
    "",
    "whales.\n",
    "",
    "",
    "7. ",
    "They ",
    "are ",
    "also ",
    "considered ",
    "",
    "",
    "apex ",
    "",
    "",
    "predators, ",
    "meaning ",
    "they ",
    "have ",
    "no ",
    "natural ",
    "",
    "predators ",
    "in ",
    "their ",
    "",
    "environment.\n",
    "",
    "",
    "8. ",
    "",
    "Orcas ",
    "have ",
    "powerful ",
    "",
    "",
    "vocalizations, ",
    "including ",
    "",
    "clicks, ",
    "",
    "",
    "",
    "whistles, ",
    "and ",
    "calls ",
    "that ",
    "they ",
    "use ",
    "for ",
    "communication ",
    "and ",
    "",
    "",
    "",
    "",
    "echolocation.\n",
    "",
    "",
    "9. ",
    "Their ",
    "",
    "",
    "lifespans ",
    "can ",
    "range ",
    "from ",
    "",
    "",
    "50 ",
    "to ",
    "",
    "",
    "90 ",
    "",
    "years, ",
    "with"
  ],
  "started_at": "2023-11-27T19:17:25.642220Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.us.c.replicate.net/v1/predictions/xmqy4u3b5vw4ex33okoyugi4j4",
    "get": "https://api.replicate.com/v1/predictions/xmqy4u3b5vw4ex33okoyugi4j4",
    "cancel": "https://api.replicate.com/v1/predictions/xmqy4u3b5vw4ex33okoyugi4j4/cancel"
  },
  "version": "dac511fd566228a0dd189e95318523fccb6ba175f9f0ecd29214b579b00b64c7"
}

Generated in

21.1 seconds

Tweak it Report

Run time and cost

This model costs approximately $0.0011 to run on Replicate, or 909 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 2 seconds.

Readme

This model doesn't have a readme.