nateraw/mistral-7b-openorca – Run with an API on Replicate

nateraw / mistral-7b-openorca

Mistral-7B-v0.1 fine tuned for chat with the OpenOrca dataset.

Cold

Public
65.9K runs
L40S
GitHub

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

max_new_tokens

integer

The maximum number of tokens the model should generate as output.

Default: 512

temperature

number

The value used to modulate the next token probabilities.

Default: 0.8

top_p

number

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).

Default: 0.95

top_k

integer

The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).

Default: 50

presence_penalty

number

Presence penalty

Default: 0

frequency_penalty

number

Frequency penalty

Default: 0

prompt_template

string

Shift + Return to add a new line

<|im_start|>system
You are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!
<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
<|im_start|>system
You are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!
<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

The template used to format the prompt. The input prompt is inserted into the template using the `{prompt}` placeholder.

Default: "<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!\n<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n"

Run this model in Node.js with one line of code:

npx create-replicate --model=nateraw/mistral-7b-openorca

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run nateraw/mistral-7b-openorca using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "nateraw/mistral-7b-openorca:7afe21847d582f7811327c903433e29334c31fe861a7cf23c62882b181bacb88",
  {
    input: {
      top_k: 50,
      top_p: 0.95,
      prompt: "Explain what's so cool about Python",
      temperature: 0.2,
      max_new_tokens: 512,
      prompt_template: "<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!\n<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n",
      presence_penalty: 0,
      frequency_penalty: 0
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run nateraw/mistral-7b-openorca using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "nateraw/mistral-7b-openorca:7afe21847d582f7811327c903433e29334c31fe861a7cf23c62882b181bacb88",
    input={
        "top_k": 50,
        "top_p": 0.95,
        "prompt": "Explain what's so cool about Python",
        "temperature": 0.2,
        "max_new_tokens": 512,
        "prompt_template": "<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!\n<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n",
        "presence_penalty": 0,
        "frequency_penalty": 0
    }
)

# The nateraw/mistral-7b-openorca model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/nateraw/mistral-7b-openorca/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run nateraw/mistral-7b-openorca using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "nateraw/mistral-7b-openorca:7afe21847d582f7811327c903433e29334c31fe861a7cf23c62882b181bacb88",
    "input": {
      "top_k": 50,
      "top_p": 0.95,
      "prompt": "Explain what\'s so cool about Python",
      "temperature": 0.2,
      "max_new_tokens": 512,
      "prompt_template": "<|im_start|>system\\nYou are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!\\n<|im_end|>\\n<|im_start|>user\\n{prompt}<|im_end|>\\n<|im_start|>assistant\\n",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

To explain what's so cool about Python, let's break it down into several key aspects: 1. Versatility: Python is a versatile programming language that can be used for a wide range of tasks, including web development, data analysis, machine learning, and more. This makes it a popular choice for developers and data scientists alike. 2. Ease of use: Python has a simple and easy-to-read syntax, which makes it beginner-friendly. This means that even if you're new to programming, you can quickly pick up the language and start writing code. 3. Open-source libraries: Python has a vast ecosystem of open-source libraries and frameworks, such as NumPy, Pandas, and TensorFlow, which can be used to accelerate development and make complex tasks easier. 4. Community support: The Python community is large and active, which means there's plenty of help available if you run into any issues or have questions. Additionally, the community contributes to the development and improvement of the language. 5. Portability: Python code can be run on various platforms, including Windows, macOS, and Linux, making it a great choice for cross-platform development. 6. Scalability: Python's code can be easily scaled up or down depending on the requirements of the project, making it suitable for both small and large-scale applications. 7. Interpreted language: Python is an interpreted language, which means that the code is executed directly by the Python interpreter, without the need for compiling. This can save time during development and deployment. In summary, Python is a cool programming language because it is versatile, easy to learn, has a large community and library support, is portable, and is scalable. These factors make it a popular choice for a wide range of applications and developers.

{
  "completed_at": "2023-10-14T00:36:45.087485Z",
  "created_at": "2023-10-14T00:36:40.099389Z",
  "data_removed": false,
  "error": null,
  "id": "73glludbc6knkvihbaeunzovym",
  "input": {
    "top_k": 50,
    "top_p": 0.95,
    "prompt": "Explain what's so cool about Python",
    "temperature": 0.2,
    "max_new_tokens": 512,
    "prompt_template": "<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!\n<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n",
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "logs": "Generated 398 tokens in 4.958377122879028 seconds.",
  "metrics": {
    "predict_time": 4.966608,
    "total_time": 4.988096
  },
  "output": [
    " To",
    " explain",
    " what",
    "'",
    "s",
    " so",
    " cool",
    " about",
    " Python",
    ",",
    " let",
    "'",
    "s",
    " break",
    " it",
    " down",
    " into",
    " several",
    " key",
    " aspects",
    ":",
    "\n",
    "\n",
    "1",
    ".",
    " Vers",
    "at",
    "ility",
    ":",
    " Python",
    " is",
    " a",
    " vers",
    "atile",
    " programming",
    " language",
    " that",
    " can",
    " be",
    " used",
    " for",
    " a",
    " wide",
    " range",
    " of",
    " tasks",
    ",",
    " including",
    " web",
    " development",
    ",",
    " data",
    " analysis",
    ",",
    " machine",
    " learning",
    ",",
    " and",
    " more",
    ".",
    " This",
    " makes",
    " it",
    " a",
    " popular",
    " choice",
    " for",
    " developers",
    " and",
    " data",
    " scientists",
    " alike",
    ".",
    "\n",
    "\n",
    "2",
    ".",
    " E",
    "ase",
    " of",
    " use",
    ":",
    " Python",
    " has",
    " a",
    " simple",
    " and",
    " easy",
    "-",
    "to",
    "-",
    "read",
    " syntax",
    ",",
    " which",
    " makes",
    " it",
    " begin",
    "ner",
    "-",
    "friendly",
    ".",
    " This",
    " means",
    " that",
    " even",
    " if",
    " you",
    "'",
    "re",
    " new",
    " to",
    " programming",
    ",",
    " you",
    " can",
    " quickly",
    " pick",
    " up",
    " the",
    " language",
    " and",
    " start",
    " writing",
    " code",
    ".",
    "\n",
    "\n",
    "3",
    ".",
    " Open",
    "-",
    "source",
    " libraries",
    ":",
    " Python",
    " has",
    " a",
    " vast",
    " ecosystem",
    " of",
    " open",
    "-",
    "source",
    " libraries",
    " and",
    " framework",
    "s",
    ",",
    " such",
    " as",
    " Num",
    "Py",
    ",",
    " Pand",
    "as",
    ",",
    " and",
    " Tensor",
    "Flow",
    ",",
    " which",
    " can",
    " be",
    " used",
    " to",
    " acceler",
    "ate",
    " development",
    " and",
    " make",
    " complex",
    " tasks",
    " easier",
    ".",
    "\n",
    "\n",
    "4",
    ".",
    " Community",
    " support",
    ":",
    " The",
    " Python",
    " community",
    " is",
    " large",
    " and",
    " active",
    ",",
    " which",
    " means",
    " there",
    "'",
    "s",
    " plenty",
    " of",
    " help",
    " available",
    " if",
    " you",
    " run",
    " into",
    " any",
    " issues",
    " or",
    " have",
    " questions",
    ".",
    " Additionally",
    ",",
    " the",
    " community",
    " cont",
    "ributes",
    " to",
    " the",
    " development",
    " and",
    " improvement",
    " of",
    " the",
    " language",
    ".",
    "\n",
    "\n",
    "5",
    ".",
    " Port",
    "ability",
    ":",
    " Python",
    " code",
    " can",
    " be",
    " run",
    " on",
    " various",
    " platforms",
    ",",
    " including",
    " Windows",
    ",",
    " mac",
    "OS",
    ",",
    " and",
    " Linux",
    ",",
    " making",
    " it",
    " a",
    " great",
    " choice",
    " for",
    " cross",
    "-",
    "platform",
    " development",
    ".",
    "\n",
    "\n",
    "6",
    ".",
    " Scal",
    "ability",
    ":",
    " Python",
    "'",
    "s",
    " code",
    " can",
    " be",
    " easily",
    " sc",
    "aled",
    " up",
    " or",
    " down",
    " depending",
    " on",
    " the",
    " requirements",
    " of",
    " the",
    " project",
    ",",
    " making",
    " it",
    " suitable",
    " for",
    " both",
    " small",
    " and",
    " large",
    "-",
    "scale",
    " applications",
    ".",
    "\n",
    "\n",
    "7",
    ".",
    " Inter",
    "pret",
    "ed",
    " language",
    ":",
    " Python",
    " is",
    " an",
    " interpreted",
    " language",
    ",",
    " which",
    " means",
    " that",
    " the",
    " code",
    " is",
    " executed",
    " directly",
    " by",
    " the",
    " Python",
    " inter",
    "preter",
    ",",
    " without",
    " the",
    " need",
    " for",
    " comp",
    "iling",
    ".",
    " This",
    " can",
    " save",
    " time",
    " during",
    " development",
    " and",
    " deployment",
    ".",
    "\n",
    "\n",
    "In",
    " summary",
    ",",
    " Python",
    " is",
    " a",
    " cool",
    " programming",
    " language",
    " because",
    " it",
    " is",
    " vers",
    "atile",
    ",",
    " easy",
    " to",
    " learn",
    ",",
    " has",
    " a",
    " large",
    " community",
    " and",
    " library",
    " support",
    ",",
    " is",
    " portable",
    ",",
    " and",
    " is",
    " scal",
    "able",
    ".",
    " These",
    " factors",
    " make",
    " it",
    " a",
    " popular",
    " choice",
    " for",
    " a",
    " wide",
    " range",
    " of",
    " applications",
    " and",
    " developers",
    ".",
    ""
  ],
  "started_at": "2023-10-14T00:36:40.120877Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/73glludbc6knkvihbaeunzovym",
    "cancel": "https://api.replicate.com/v1/predictions/73glludbc6knkvihbaeunzovym/cancel"
  },
  "version": "7afe21847d582f7811327c903433e29334c31fe861a7cf23c62882b181bacb88"
}

Generated in

5.0 seconds

Tweak it Report

Run time and cost

This model costs approximately $0.00098 to run on Replicate, or 1020 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 1 seconds.

Readme

See the official model card on Hugging Face: https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca

Thank you to Open-Orca for releasing this model!