lucataco

hermes-2-pro-llama-3-70b

Hermes 2 Pro is an updated and cleaned version of the OpenHermes 2.5 Dataset, as well as a newly introduced Function Calling and JSON Mode dataset developed in-house

Public

344 runs

2x A100 (80GB)

Run with an API

Pricing

Playground API Examples Train README Versions

Prediction

lucataco/hermes-2-pro-llama-3-70b:fb2ed69e8d02c1ebe10d89842cf94b562aad4fe4480df7e40a90050328d8a99e

Model

lucataco/hermes-2-pro-llama-3-70b:fb2ed69e

vdttgqtdb9rgg0cgc1dra25gqm

Status

Succeeded

Source

Web

Hardware

2x A100 (80GB)

Total duration

3m 50s

Created

about 1 year ago

Input

top_k: 50
top_p: 0.9
prompt: Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.
max_tokens: 512
min_tokens: 0
temperature: 0.6
system_prompt: You are a helpful assistant.
presence_penalty: 0
frequency_penalty: 0

{
  "top_k": 50,
  "top_p": 0.9,
  "prompt": "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "system_prompt": "You are a helpful assistant.",
  "presence_penalty": 0,
  "frequency_penalty": 0
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run lucataco/hermes-2-pro-llama-3-70b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "lucataco/hermes-2-pro-llama-3-70b:fb2ed69e8d02c1ebe10d89842cf94b562aad4fe4480df7e40a90050328d8a99e",
  {
    input: {
      top_k: 50,
      top_p: 0.9,
      prompt: "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
      max_tokens: 512,
      min_tokens: 0,
      temperature: 0.6,
      system_prompt: "You are a helpful assistant.",
      presence_penalty: 0,
      frequency_penalty: 0
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run lucataco/hermes-2-pro-llama-3-70b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "lucataco/hermes-2-pro-llama-3-70b:fb2ed69e8d02c1ebe10d89842cf94b562aad4fe4480df7e40a90050328d8a99e",
    input={
        "top_k": 50,
        "top_p": 0.9,
        "prompt": "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
        "max_tokens": 512,
        "min_tokens": 0,
        "temperature": 0.6,
        "system_prompt": "You are a helpful assistant.",
        "presence_penalty": 0,
        "frequency_penalty": 0
    }
)

# The lucataco/hermes-2-pro-llama-3-70b model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/lucataco/hermes-2-pro-llama-3-70b/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Run lucataco/hermes-2-pro-llama-3-70b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "lucataco/hermes-2-pro-llama-3-70b:fb2ed69e8d02c1ebe10d89842cf94b562aad4fe4480df7e40a90050328d8a99e",
    "input": {
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "system_prompt": "You are a helpful assistant.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Title: The Unlikely Alliance Goku was soaring through the skies, enjoying the peace and tranquility that had lasted for a few years since his last battle. Suddenly, he sensed an immense power surging from a distant planet. His curiosity piqued, Goku used his instant transmission to travel to the source of the energy. Upon arrival, Goku was shocked to find Kirby, the pink puffball he had once considered an ally, standing alongside Majin Buu, the powerful villain responsible for the destruction of entire planets. The two unlikely allies were laughing maniacally as they prepared to unleash their combined power upon the Earth. Goku's eyes widened in disbelief. "Kirby, what are you doing with Majin Buu? You're supposed to be a hero, not a villain!" Kirby turned to face Goku, his once innocent eyes now filled with malice. "Foolish Saiyan, you know nothing! I have grown tired of being seen as weak and innocent. Teaming up with Majin Buu will grant me the power and respect I deserve!" Majin Buu chuckled, a sinister grin on his face. "Yes, Kirby and I have decided to join forces and destroy this pathetic planet. No one will stand in our way, not even you, Goku!" Goku's fists clenched as he prepared for battle. "I can't let you do that. I have to protect the Earth and everyone on it!" The battle commenced, with Goku using every ounce of his strength to fend off the combined might of Kirby and Majin Buu. Despite his incredible power, Goku found himself struggling against the duo. Kirby's ability to copy his opponents' powers, combined with Majin Buu's overwhelming strength, proved to be a formidable challenge. As the battle raged on, Goku noticed a glimmer of hesitation in Kirby's eyes. Sensing an opportunity, Goku spoke to the pink puffball. "Kirby, I know you're not evil. You've always been a hero, and you don't have to do this. You can still choose the right path!" Kirby faltered, his eyes wavering between his desire for power and his innate sense of justice. Goku pressed on, "Remember the times we fought together as allies? We protected the innocent and stood against evil. You don't want to be responsible for the destruction of the Earth, do you?" Torn between his ambitions and his conscience, Kirby finally made his choice. He turned to Majin Buu

{
  "completed_at": "2024-06-28T19:42:24.616274Z",
  "created_at": "2024-06-28T19:38:34.714000Z",
  "data_removed": false,
  "error": null,
  "id": "vdttgqtdb9rgg0cgc1dra25gqm",
  "input": {
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are a helpful assistant.",
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "logs": "INFO 06-28 19:41:58 async_llm_engine.py:529] Received request 07aba362773940d7aa2154854e6b86c9: prompt: '<|begin_of_text|><|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n<|im_start|>user\\nWrite a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>\\n<|im_start|>assistant\\n', sampling_params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=50, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[128003], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.\n stdoutINFO 06-28 19:42:01 metrics.py:334] Avg prompt throughput: 8.5 tokens/s, Avg generation throughput: 14.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%\n stdoutINFO 06-28 19:42:07 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%\n stdoutINFO 06-28 19:42:12 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%\n stdoutINFO 06-28 19:42:17 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%\n stdoutINFO 06-28 19:42:22 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.2%, CPU KV cache usage: 0.0%\n stdoutGeneration took 1719601455.61sFormatted prompt: <|begin_of_text|><|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWrite a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>\n<|im_start|>assistant\nINFO 06-28 19:42:24 async_llm_engine.py:120] Finished request 07aba362773940d7aa2154854e6b86c9.\n stdout",
  "metrics": {
    "predict_time": 26.551590739,
    "total_time": 229.902274
  },
  "output": [
    "Title",
    ":",
    " The",
    " Un",
    "likely",
    " Alliance",
    "\n\n",
    "G",
    "oku",
    " was",
    " soaring",
    " through",
    " the",
    " skies",
    ",",
    " enjoying",
    " the",
    " peace",
    " and",
    " tranqu",
    "ility",
    " that",
    " had",
    " lasted",
    " for",
    " a",
    " few",
    " years",
    " since",
    " his",
    " last",
    " battle",
    ".",
    " Suddenly",
    ",",
    " he",
    " sensed",
    " an",
    " immense",
    " power",
    " sur",
    "ging",
    " from",
    " a",
    " distant",
    " planet",
    ".",
    " His",
    " curiosity",
    " p",
    "iqu",
    "ed",
    ",",
    " Goku",
    " used",
    " his",
    " instant",
    " transmission",
    " to",
    " travel",
    " to",
    " the",
    " source",
    " of",
    " the",
    " energy",
    ".\n\n",
    "Upon",
    " arrival",
    ",",
    " Goku",
    " was",
    " shocked",
    " to",
    " find",
    " Kirby",
    ",",
    " the",
    " pink",
    " puff",
    "ball",
    " he",
    " had",
    " once",
    " considered",
    " an",
    " ally",
    ",",
    " standing",
    " alongside",
    " Maj",
    "in",
    " Bu",
    "u",
    ",",
    " the",
    " powerful",
    " villain",
    " responsible",
    " for",
    " the",
    " destruction",
    " of",
    " entire",
    " planets",
    ".",
    " The",
    " two",
    " unlikely",
    " allies",
    " were",
    " laughing",
    " man",
    "iac",
    "ally",
    " as",
    " they",
    " prepared",
    " to",
    " unleash",
    " their",
    " combined",
    " power",
    " upon",
    " the",
    " Earth",
    ".\n\n",
    "G",
    "oku",
    "'s",
    " eyes",
    " widened",
    " in",
    " disbelief",
    ".",
    " \"",
    "Kir",
    "by",
    ",",
    " what",
    " are",
    " you",
    " doing",
    " with",
    " Maj",
    "in",
    " Bu",
    "u",
    "?",
    " You",
    "'re",
    " supposed",
    " to",
    " be",
    " a",
    " hero",
    ",",
    " not",
    " a",
    " villain",
    "!\"\n\n",
    "Kir",
    "by",
    " turned",
    " to",
    " face",
    " Goku",
    ",",
    " his",
    " once",
    " innocent",
    " eyes",
    " now",
    " filled",
    " with",
    " mal",
    "ice",
    ".",
    " \"",
    "F",
    "ool",
    "ish",
    " Sai",
    "yan",
    ",",
    " you",
    " know",
    " nothing",
    "!",
    " I",
    " have",
    " grown",
    " tired",
    " of",
    " being",
    " seen",
    " as",
    " weak",
    " and",
    " innocent",
    ".",
    " Te",
    "aming",
    " up",
    " with",
    " Maj",
    "in",
    " Bu",
    "u",
    " will",
    " grant",
    " me",
    " the",
    " power",
    " and",
    " respect",
    " I",
    " deserve",
    "!\"\n\n",
    "M",
    "aj",
    "in",
    " Bu",
    "u",
    " chuckled",
    ",",
    " a",
    " sinister",
    " grin",
    " on",
    " his",
    " face",
    ".",
    " \"",
    "Yes",
    ",",
    " Kirby",
    " and",
    " I",
    " have",
    " decided",
    " to",
    " join",
    " forces",
    " and",
    " destroy",
    " this",
    " pathetic",
    " planet",
    ".",
    " No",
    " one",
    " will",
    " stand",
    " in",
    " our",
    " way",
    ",",
    " not",
    " even",
    " you",
    ",",
    " Goku",
    "!\"\n\n",
    "G",
    "oku",
    "'s",
    " fists",
    " clen",
    "ched",
    " as",
    " he",
    " prepared",
    " for",
    " battle",
    ".",
    " \"",
    "I",
    " can",
    "'t",
    " let",
    " you",
    " do",
    " that",
    ".",
    " I",
    " have",
    " to",
    " protect",
    " the",
    " Earth",
    " and",
    " everyone",
    " on",
    " it",
    "!\"\n\n",
    "The",
    " battle",
    " commenced",
    ",",
    " with",
    " Goku",
    " using",
    " every",
    " ounce",
    " of",
    " his",
    " strength",
    " to",
    " fend",
    " off",
    " the",
    " combined",
    " might",
    " of",
    " Kirby",
    " and",
    " Maj",
    "in",
    " Bu",
    "u",
    ".",
    " Despite",
    " his",
    " incredible",
    " power",
    ",",
    " Goku",
    " found",
    " himself",
    " struggling",
    " against",
    " the",
    " duo",
    ".",
    " Kirby",
    "'s",
    " ability",
    " to",
    " copy",
    " his",
    " opponents",
    "'",
    " powers",
    ",",
    " combined",
    " with",
    " Maj",
    "in",
    " Bu",
    "u",
    "'s",
    " overwhelming",
    " strength",
    ",",
    " proved",
    " to",
    " be",
    " a",
    " formidable",
    " challenge",
    ".\n\n",
    "As",
    " the",
    " battle",
    " r",
    "aged",
    " on",
    ",",
    " Goku",
    " noticed",
    " a",
    " gl",
    "immer",
    " of",
    " hesitation",
    " in",
    " Kirby",
    "'s",
    " eyes",
    ".",
    " Sens",
    "ing",
    " an",
    " opportunity",
    ",",
    " Goku",
    " spoke",
    " to",
    " the",
    " pink",
    " puff",
    "ball",
    ".",
    " \"",
    "Kir",
    "by",
    ",",
    " I",
    " know",
    " you",
    "'re",
    " not",
    " evil",
    ".",
    " You",
    "'ve",
    " always",
    " been",
    " a",
    " hero",
    ",",
    " and",
    " you",
    " don",
    "'t",
    " have",
    " to",
    " do",
    " this",
    ".",
    " You",
    " can",
    " still",
    " choose",
    " the",
    " right",
    " path",
    "!\"\n\n",
    "Kir",
    "by",
    " fal",
    "tered",
    ",",
    " his",
    " eyes",
    " wav",
    "ering",
    " between",
    " his",
    " desire",
    " for",
    " power",
    " and",
    " his",
    " innate",
    " sense",
    " of",
    " justice",
    ".",
    " Goku",
    " pressed",
    " on",
    ",",
    " \"",
    "Remember",
    " the",
    " times",
    " we",
    " fought",
    " together",
    " as",
    " allies",
    "?",
    " We",
    " protected",
    " the",
    " innocent",
    " and",
    " stood",
    " against",
    " evil",
    ".",
    " You",
    " don",
    "'t",
    " want",
    " to",
    " be",
    " responsible",
    " for",
    " the",
    " destruction",
    " of",
    " the",
    " Earth",
    ",",
    " do",
    " you",
    "?\"\n\n",
    "T",
    "orn",
    " between",
    " his",
    " ambitions",
    " and",
    " his",
    " conscience",
    ",",
    " Kirby",
    " finally",
    " made",
    " his",
    " choice",
    ".",
    " He",
    " turned",
    " to",
    " Maj",
    "in",
    " Bu",
    "u"
  ],
  "started_at": "2024-06-28T19:41:58.064683Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.us.c.replicate.net/v1/streams/ds5pasg23n2ace4izi3vdamwoa4kbxbrqygi2pvqfn5wjz7q2huq",
    "get": "https://api.replicate.com/v1/predictions/vdttgqtdb9rgg0cgc1dra25gqm",
    "cancel": "https://api.replicate.com/v1/predictions/vdttgqtdb9rgg0cgc1dra25gqm/cancel"
  },
  "version": "fb2ed69e8d02c1ebe10d89842cf94b562aad4fe4480df7e40a90050328d8a99e"
}

Generated in

26.6 seconds

Tweak it Iterate in playground Share Report

INFO 06-28 19:41:58 async_llm_engine.py:529] Received request 07aba362773940d7aa2154854e6b86c9: prompt: '<|begin_of_text|><|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWrite a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>\n<|im_start|>assistant\n', sampling_params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=50, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[128003], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.
 stdoutINFO 06-28 19:42:01 metrics.py:334] Avg prompt throughput: 8.5 tokens/s, Avg generation throughput: 14.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%
 stdoutINFO 06-28 19:42:07 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%
 stdoutINFO 06-28 19:42:12 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%
 stdoutINFO 06-28 19:42:17 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%
 stdoutINFO 06-28 19:42:22 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.2%, CPU KV cache usage: 0.0%
 stdoutGeneration took 1719601455.61sFormatted prompt: <|begin_of_text|><|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>
<|im_start|>assistant
INFO 06-28 19:42:24 async_llm_engine.py:120] Finished request 07aba362773940d7aa2154854e6b86c9.
 stdout

Want to make some of these yourself?

Run this model