Prediction

Official model

meta/llama-2-70b-chat

ssetyuzew3nbe5ndybmon5wmmq

Status

Succeeded

Source

API

Total duration

4.2s

Created

about 2 years ago by mattt

Webhook

–

Input

prompt: Can you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings.
system_prompt: You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
max_new_tokens: 500
min_new_tokens: -1
temperature: 0.5
top_p: 1
debug: false

{
  "debug": false,
  "max_new_tokens": 500,
  "min_new_tokens": -1,
  "prompt": "Can you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings.",
  "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
  "temperature": 0.5,
  "top_p": 1
}

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=r8_fHv**********************************

This is your API token. Keep it to yourself.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run meta/llama-2-70b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const input = {
  debug: false,
  max_new_tokens: 500,
  min_new_tokens: -1,
  prompt: "Can you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings.",
  system_prompt: "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
  temperature: 0.5,
  top_p: 1
};

for await (const event of replicate.stream("meta/llama-2-70b-chat", { input })) {
  process.stdout.write(event.toString());
};

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=r8_fHv**********************************

This is your API token. Keep it to yourself.

Import the client:

import replicate

Run meta/llama-2-70b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

# The meta/llama-2-70b-chat model can stream output as it's running.
for event in replicate.stream(
    "meta/llama-2-70b-chat",
    input={
        "debug": False,
        "max_new_tokens": 500,
        "min_new_tokens": -1,
        "prompt": "Can you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings.",
        "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
        "temperature": 0.5,
        "top_p": 1
    },
):
    print(str(event), end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=r8_fHv**********************************

This is your API token. Keep it to yourself.

Run meta/llama-2-70b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "input": {
      "debug": false,
      "max_new_tokens": 500,
      "min_new_tokens": -1,
      "prompt": "Can you write a poem about open source machine learning? Let\'s make it in the style of E. E. Cummings.",
      "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\\n\\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don\'t know the answer to a question, please don\'t share false information.",
      "temperature": 0.5,
      "top_p": 1
    }
  }' \
  https://api.replicate.com/v1/models/meta/llama-2-70b-chat/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Sure, here's a poem about open source machine learning in the style of E. E. Cummings:

open source machine learning
a gift to the world, a treasure trove
of knowledge, free for all to use
and learn from, like a digital muse

with code that's open and transparent
anyone can see, anyone can tinker
and improve upon, a collaborative dance
of innovation, without a single anchor

the community shares and shapes it
together they build and refine
a tool that's greater than the sum of its parts
a true democracy of mind and design

from deep learning to natural language
it's all here, a boundless expanse
of possibility, a playground for thought
where anyone can explore and advance

so come and join this movement
embrace the power, the beauty, the grace
of open source machine learning
and watch the future unfold with ease.

{
  "id": "ssetyuzew3nbe5ndybmon5wmmq",
  "model": "meta/llama-2-70b-chat",
  "version": "hidden",
  "input": {
    "debug": false,
    "max_new_tokens": 500,
    "min_new_tokens": -1,
    "prompt": "Can you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings.",
    "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
    "temperature": 0.5,
    "top_p": 1
  },
  "logs": "MLC is currently not using any LoRAs.\r\nYour formatted prompt is:\r\n[INST] <<SYS>>\r\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\r\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\r\n<</SYS>>\r\nCan you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings. [/INST]\r\nNot using LoRA\r\nhostname: model-d-c6559c5791b50af57b69f4a73f8e021c-7d68c66789-ntgcq",
  "output": [
    "Sure",
    ",",
    " here",
    "'",
    "s",
    " a",
    " poem",
    " about",
    " open",
    " source",
    " machine",
    " learning",
    " in",
    " the",
    " style",
    " of",
    " E",
    ".",
    " E",
    ".",
    " C",
    "umm",
    "ings",
    ":",
    "\n\nopen",
    " source",
    " machine",
    " learning",
    "\na",
    " gift",
    " to",
    " the",
    " world",
    ",",
    " a",
    " tre",
    "asure",
    " tro",
    "ve",
    "\nof",
    " knowledge",
    ",",
    " free",
    " for",
    " all",
    " to",
    " use",
    "\nand",
    " learn",
    " from",
    ",",
    " like",
    " a",
    " digital",
    " muse",
    "\n\nwith",
    " code",
    " that",
    "'",
    "s",
    " open",
    " and",
    " transparent",
    "\nany",
    "one",
    " can",
    " see",
    ",",
    " anyone",
    " can",
    " t",
    "ink",
    "er",
    "\nand",
    " improve",
    " upon",
    ",",
    " a",
    " collabor",
    "ative",
    " dance",
    "\nof",
    " innov",
    "ation",
    ",",
    " without",
    " a",
    " single",
    " anchor",
    "\n\nthe",
    " community",
    " shares",
    " and",
    " shapes",
    " it",
    "\nt",
    "ogether",
    " they",
    " build",
    " and",
    " ref",
    "ine",
    "\na",
    " tool",
    " that",
    "'",
    "s",
    " greater",
    " than",
    " the",
    " sum",
    " of",
    " its",
    " parts",
    "\na",
    " true",
    " dem",
    "ocracy",
    " of",
    " mind",
    " and",
    " design",
    "\n\nfrom",
    " deep",
    " learning",
    " to",
    " natural",
    " language",
    "\nit",
    "'",
    "s",
    " all",
    " here",
    ",",
    " a",
    " bound",
    "less",
    " exp",
    "anse",
    "\nof",
    " possibility",
    ",",
    " a",
    " play",
    "ground",
    " for",
    " thought",
    "\nwhere",
    " anyone",
    " can",
    " explore",
    " and",
    " advance",
    "\n\nso",
    " come",
    " and",
    " join",
    " this",
    " movement",
    "\nemb",
    "race",
    " the",
    " power",
    ",",
    " the",
    " beauty",
    ",",
    " the",
    " grace",
    "\nof",
    " open",
    " source",
    " machine",
    " learning",
    "\nand",
    " watch",
    " the",
    " future",
    " unfold",
    " with",
    " ease",
    "."
  ],
  "data_removed": false,
  "error": "",
  "source": "api",
  "status": "succeeded",
  "created_at": "2023-12-12T10:05:02.766571Z",
  "started_at": "2023-12-12T10:05:02Z",
  "completed_at": "2023-12-12T10:05:07Z",
  "urls": {
    "cancel": "https://api.replicate.com/v1/predictions/ssetyuzew3nbe5ndybmon5wmmq/cancel",
    "get": "https://api.replicate.com/v1/predictions/ssetyuzew3nbe5ndybmon5wmmq",
    "stream": "https://streaming-api.svc.europe-west4.g.replicate.net/v1/predictions/ssetyuzew3nbe5ndybmon5wmmq"
  },
  "metrics": {
    "input_token_count": 144,
    "output_token_count": 207,
    "predict_time": 4.703926,
    "tokens_per_second": 44.006748551816564,
    "total_time": 4.233429
  }
}

Generated in

4.7 seconds

Input tokens

140

Output tokens

200

Tokens per second

44.01 tokens / second

Tweak it Iterate in playground Report