Official

meta / meta-llama-3-70b-instruct

A 70 billion parameter language model from Meta, fine tuned for chat completions

Warm

Public
157.4M runs
Priced by multiple properties
GitHub
License

Iterate in playground

Run with an API

Playground API Examples README

Prediction

7zr9g2asx9rgj0cey8698vm4d4

Status

Succeeded

Source

Web

Hardware

8x A100 (80GB)

Total duration

3.5s

Created

over 1 year ago by @meta

Input

top_p: 0.9
prompt: Work through this problem step by step: Q: Sarah has 7 llamas. Her friend gives her 3 more trucks of llamas. Each truck has 5 llamas. How many llamas does Sarah have in total?
max_tokens: 512
min_tokens: 0
temperature: 0.6
prompt_template: <|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|> {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
presence_penalty: 1.15
frequency_penalty: 0.2

{
  "top_p": 0.9,
  "prompt": "Work through this problem step by step:\n\nQ: Sarah has 7 llamas. Her friend gives her 3 more trucks of llamas. Each truck has 5 llamas. How many llamas does Sarah have in total?",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "prompt_template": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
  "presence_penalty": 1.15,
  "frequency_penalty": 0.2
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run meta/meta-llama-3-70b-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const input = {
  top_p: 0.9,
  prompt: "Work through this problem step by step:\n\nQ: Sarah has 7 llamas. Her friend gives her 3 more trucks of llamas. Each truck has 5 llamas. How many llamas does Sarah have in total?",
  max_tokens: 512,
  min_tokens: 0,
  temperature: 0.6,
  prompt_template: "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
  presence_penalty: 1.15,
  frequency_penalty: 0.2
};

for await (const event of replicate.stream("meta/meta-llama-3-70b-instruct", { input })) {
  process.stdout.write(event.toString());
};

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run meta/meta-llama-3-70b-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

# The meta/meta-llama-3-70b-instruct model can stream output as it's running.
for event in replicate.stream(
    "meta/meta-llama-3-70b-instruct",
    input={
        "top_p": 0.9,
        "prompt": "Work through this problem step by step:\n\nQ: Sarah has 7 llamas. Her friend gives her 3 more trucks of llamas. Each truck has 5 llamas. How many llamas does Sarah have in total?",
        "max_tokens": 512,
        "min_tokens": 0,
        "temperature": 0.6,
        "prompt_template": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
        "presence_penalty": 1.15,
        "frequency_penalty": 0.2
    },
):
    print(str(event), end="")

To learn more, take a look at the guide on getting started with Python.

Run meta/meta-llama-3-70b-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "input": {
      "top_p": 0.9,
      "prompt": "Work through this problem step by step:\\n\\nQ: Sarah has 7 llamas. Her friend gives her 3 more trucks of llamas. Each truck has 5 llamas. How many llamas does Sarah have in total?",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "prompt_template": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n",
      "presence_penalty": 1.15,
      "frequency_penalty": 0.2
    }
  }' \
  https://api.replicate.com/v1/models/meta/meta-llama-3-70b-instruct/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Let's break this problem down step by step. Step 1: Sarah already has 7 llamas. Step 2: Her friend gives her 3 trucks of llamas. We need to find out how many llamas are in these 3 trucks. Step 3: Each truck has 5 llamas, so we multiply the number of trucks (3) by the number of llamas per truck (5): 3 trucks x 5 llamas/truck = 3 x 5 = 15 llamas Step 4: Sarah already had 7 llamas, and now she gets 15 more llamas from her friend. To find the total number of llamas Sarah has, we add the two numbers together: 7 llamas (already had) + 15 llamas (from her friend) = 22 llamas Therefore, Sarah has a total of 22 llamas.

{
  "completed_at": "2024-04-18T16:31:23Z",
  "created_at": "2024-04-18T16:31:19.530000Z",
  "data_removed": false,
  "error": "",
  "id": "7zr9g2asx9rgj0cey8698vm4d4",
  "input": {
    "top_p": 0.9,
    "prompt": "Work through this problem step by step:\n\nQ: Sarah has 7 llamas. Her friend gives her 3 more trucks of llamas. Each truck has 5 llamas. How many llamas does Sarah have in total?",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "prompt_template": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
    "presence_penalty": 1.15,
    "frequency_penalty": 0.2
  },
  "logs": "",
  "metrics": {
    "total_time": 3.47,
    "input_token_count": 54,
    "tokens_per_second": 41.72998441835564,
    "output_token_count": 166,
    "predict_time": 4.045511
  },
  "output": [
    "Let",
    "'s",
    " break",
    " this",
    " problem",
    " down",
    " step",
    " by",
    " step",
    ".\n\n",
    "Step",
    " ",
    "1",
    ":",
    " Sarah",
    " already",
    " has",
    " ",
    "7",
    " ll",
    "amas",
    ".\n\n",
    "Step",
    " ",
    "2",
    ":",
    " Her",
    " friend",
    " gives",
    " her",
    " ",
    "3",
    " trucks",
    " of",
    " ll",
    "amas",
    ".",
    " We",
    " need",
    " to",
    " find",
    " out",
    " how",
    " many",
    " ll",
    "amas",
    " are",
    " in",
    " these",
    " ",
    "3",
    " trucks",
    ".\n\n",
    "Step",
    " ",
    "3",
    ":",
    " Each",
    " truck",
    " has",
    " ",
    "5",
    " ll",
    "amas",
    ",",
    " so",
    " we",
    " multiply",
    " the",
    " number",
    " of",
    " trucks",
    " (",
    "3",
    ")",
    " by",
    " the",
    " number",
    " of",
    " ll",
    "amas",
    " per",
    " truck",
    " (",
    "5",
    "):\n\n",
    "3",
    " trucks",
    " x",
    " ",
    "5",
    " ll",
    "amas",
    "/tr",
    "uck",
    " =",
    " ",
    "3",
    " x",
    " ",
    "5",
    " =",
    " ",
    "15",
    " ll",
    "amas",
    "\n\n",
    "Step",
    " ",
    "4",
    ":",
    " Sarah",
    " already",
    " had",
    " ",
    "7",
    " ll",
    "amas",
    ",",
    " and",
    " now",
    " she",
    " gets",
    " ",
    "15",
    " more",
    " ll",
    "amas",
    " from",
    " her",
    " friend",
    ".",
    " To",
    " find",
    " the",
    " total",
    " number",
    " of",
    " ll",
    "amas",
    " Sarah",
    " has",
    ",",
    " we",
    " add",
    " the",
    " two",
    " numbers",
    " together",
    ":\n\n",
    "7",
    " ll",
    "amas",
    " (",
    "already",
    " had",
    ")",
    " +",
    " ",
    "15",
    " ll",
    "amas",
    " (",
    "from",
    " her",
    " friend",
    ")",
    " =",
    " ",
    "22",
    " ll",
    "amas",
    "\n\n",
    "Therefore",
    ",",
    " Sarah",
    " has",
    " a",
    " total",
    " of",
    " ",
    "22",
    " ll",
    "amas",
    ".",
    ""
  ],
  "started_at": "2024-04-18T16:31:19Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.us.c.replicate.net/v1/streams/cvg64spwkdzlpdltjkmv6jotmwn2cq5btgxirmsbejvsxx7xp2la",
    "get": "https://api.replicate.com/v1/predictions/7zr9g2asx9rgj0cey8698vm4d4",
    "cancel": "https://api.replicate.com/v1/predictions/7zr9g2asx9rgj0cey8698vm4d4/cancel"
  },
  "version": "fbfb20b472b2f3bdd101412a9f70a0ed4fc0ced78a77ff00970ee7a2383c575d"
}

Generated in

4.0 seconds

Input tokens

Output tokens

160

Tokens per second

41.73 tokens / second

Share Report

Want to make some of these yourself?

Run this model