lucataco/hermes-2-pro-llama-3-8b:51ca4143 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

lucataco /hermes-2-pro-llama-3-8b:51ca4143

Input

prompt

string

Shift + Return to add a new line

Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.

Prompt

Default: ""

system_prompt

string

Shift + Return to add a new line

System prompt to send to the model. This is prepended to the prompt and helps guide system behavior. Ignored for non-chat models.

Default: "You are a helpful assistant."

min_tokens

integer

The minimum number of tokens the model should generate as output.

Default: 0

max_tokens

integer

The maximum number of tokens the model should generate as output.

Default: 512

temperature

number

The value used to modulate the next token probabilities.

Default: 0.6

top_p

number

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).

Default: 0.9

top_k

integer

The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).

Default: 50

presence_penalty

number

Presence penalty

Default: 0

frequency_penalty

number

Frequency penalty

Default: 0

stop_sequences

string

Shift + Return to add a new line

A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.

Run this model in Node.js with one line of code:

npx create-replicate --model=lucataco/hermes-2-pro-llama-3-8b

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run lucataco/hermes-2-pro-llama-3-8b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "lucataco/hermes-2-pro-llama-3-8b:51ca4143b8464e9dfeb0c88339962a6bdf2ffd833f047ed293b38537a92c1515",
  {
    input: {
      top_k: 50,
      top_p: 0.9,
      prompt: "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
      max_tokens: 512,
      min_tokens: 0,
      temperature: 0.6,
      system_prompt: "You are Hermes 2, a helpful assistant.",
      presence_penalty: 0,
      frequency_penalty: 0
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run lucataco/hermes-2-pro-llama-3-8b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "lucataco/hermes-2-pro-llama-3-8b:51ca4143b8464e9dfeb0c88339962a6bdf2ffd833f047ed293b38537a92c1515",
    input={
        "top_k": 50,
        "top_p": 0.9,
        "prompt": "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
        "max_tokens": 512,
        "min_tokens": 0,
        "temperature": 0.6,
        "system_prompt": "You are Hermes 2, a helpful assistant.",
        "presence_penalty": 0,
        "frequency_penalty": 0
    }
)

# The lucataco/hermes-2-pro-llama-3-8b model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/lucataco/hermes-2-pro-llama-3-8b/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run lucataco/hermes-2-pro-llama-3-8b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "lucataco/hermes-2-pro-llama-3-8b:51ca4143b8464e9dfeb0c88339962a6bdf2ffd833f047ed293b38537a92c1515",
    "input": {
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "system_prompt": "You are Hermes 2, a helpful assistant.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Once upon a time, in the peaceful world of Dragon Ball, Goku and his friends were enjoying a day off after another successful tournament. Little did they know, a sinister plot was brewing in the shadows. In another realm, Kirby, the powerful pink puffball, had been approached by the evil Majin Buu. The mischievous creature had been released from his imprisonment and was now seeking a new ally to help him achieve his ultimate goal - to destroy the world! Kirby, being the curious and friendly character that he is, was initially hesitant. However, Majin Buu promised him great power and the ability to eat any enemy they encountered. Kirby, never one to turn down a good fight, agreed to join forces with the sinister villain. As the duo began their destructive path, wreaking havoc across the world, Goku and his friends were shocked and dismayed. They knew they had to act fast before the world was destroyed. Goku, with his keen sense of justice, rallied his friends and set off to stop Kirby and Majin Buu. Along the way, they encountered various challenges and obstacles, but their determination to save the world kept them going. As they finally caught up with the nefarious pair, a fierce battle ensued. Goku and his friends fought valiantly, but Kirby's incredible power and Majin Buu's cunning abilities proved to be a formidable force. Just when all seemed lost, Goku had an idea. He remembered a technique he had learned from his training with King Kai - the Fusion technique. With the help of his friends, Goku fused with his son Gohan, creating a powerful new warrior, Gogeta. Gogeta charged forward, unleashing a barrage of powerful attacks on Kirby and Majin Buu. The two villains fought back with all their might, but Gogeta's strength was too great. Finally, in a climactic moment, Gogeta delivered a final blow, defeating Kirby and Majin Buu and saving the world from destruction. As the dust settled, Goku and his friends breathed a sigh of relief, knowing that they had once again triumphed over evil. Kirby, realizing the error of his ways, vowed to be a better hero and joined forces with Goku and his friends to protect the world. From that day on, the unlikely alliance of Goku, his friends, and Kirby worked together to maintain peace and justice throughout the galaxy. And so, a new chapter in their epic adventures began.

{
  "completed_at": "2024-06-28T18:22:21.424958Z",
  "created_at": "2024-06-28T18:21:22.750000Z",
  "data_removed": false,
  "error": null,
  "id": "xzx1tqczqsrgj0cgc0aby4jzyc",
  "input": {
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are Hermes 2, a helpful assistant.",
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "logs": "INFO 06-28 18:22:08 async_llm_engine.py:529] Received request e4ae7433fdfe4cbda6b3543f717a58ad: prompt: '<|begin_of_text|><|im_start|>system\\nYou are Hermes 2, a helpful assistant.<|im_end|>\\n<|im_start|>user\\nWrite a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>\\n<|im_start|>assistant\\n', sampling_params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=50, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[128003], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.\n stdoutINFO 06-28 18:22:12 metrics.py:334] Avg prompt throughput: 9.4 tokens/s, Avg generation throughput: 30.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%\n stdoutINFO 06-28 18:22:17 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 38.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%\n stdoutGeneration took 1719596581.42sINFO 06-28 18:22:21 async_llm_engine.py:120] Finished request e4ae7433fdfe4cbda6b3543f717a58ad.\n stdoutFormatted prompt: <|begin_of_text|><|im_start|>system\nYou are Hermes 2, a helpful assistant.<|im_end|>\n<|im_start|>user\nWrite a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>\n<|im_start|>assistant",
  "metrics": {
    "predict_time": 12.990238542,
    "total_time": 58.674958
  },
  "output": [
    "Once",
    " upon",
    " a",
    " time",
    ",",
    " in",
    " the",
    " peaceful",
    " world",
    " of",
    " Dragon",
    " Ball",
    ",",
    " Goku",
    " and",
    " his",
    " friends",
    " were",
    " enjoying",
    " a",
    " day",
    " off",
    " after",
    " another",
    " successful",
    " tournament",
    ".",
    " Little",
    " did",
    " they",
    " know",
    ",",
    " a",
    " sinister",
    " plot",
    " was",
    " brewing",
    " in",
    " the",
    " shadows",
    ".\n\n",
    "In",
    " another",
    " realm",
    ",",
    " Kirby",
    ",",
    " the",
    " powerful",
    " pink",
    " puff",
    "ball",
    ",",
    " had",
    " been",
    " approached",
    " by",
    " the",
    " evil",
    " Maj",
    "in",
    " Bu",
    "u",
    ".",
    " The",
    " mis",
    "chie",
    "vous",
    " creature",
    " had",
    " been",
    " released",
    " from",
    " his",
    " imprisonment",
    " and",
    " was",
    " now",
    " seeking",
    " a",
    " new",
    " ally",
    " to",
    " help",
    " him",
    " achieve",
    " his",
    " ultimate",
    " goal",
    " -",
    " to",
    " destroy",
    " the",
    " world",
    "!\n\n",
    "Kir",
    "by",
    ",",
    " being",
    " the",
    " curious",
    " and",
    " friendly",
    " character",
    " that",
    " he",
    " is",
    ",",
    " was",
    " initially",
    " hesitant",
    ".",
    " However",
    ",",
    " Maj",
    "in",
    " Bu",
    "u",
    " promised",
    " him",
    " great",
    " power",
    " and",
    " the",
    " ability",
    " to",
    " eat",
    " any",
    " enemy",
    " they",
    " encountered",
    ".",
    " Kirby",
    ",",
    " never",
    " one",
    " to",
    " turn",
    " down",
    " a",
    " good",
    " fight",
    ",",
    " agreed",
    " to",
    " join",
    " forces",
    " with",
    " the",
    " sinister",
    " villain",
    ".\n\n",
    "As",
    " the",
    " duo",
    " began",
    " their",
    " destructive",
    " path",
    ",",
    " wreak",
    "ing",
    " havoc",
    " across",
    " the",
    " world",
    ",",
    " Goku",
    " and",
    " his",
    " friends",
    " were",
    " shocked",
    " and",
    " dismay",
    "ed",
    ".",
    " They",
    " knew",
    " they",
    " had",
    " to",
    " act",
    " fast",
    " before",
    " the",
    " world",
    " was",
    " destroyed",
    ".\n\n",
    "G",
    "oku",
    ",",
    " with",
    " his",
    " keen",
    " sense",
    " of",
    " justice",
    ",",
    " rallied",
    " his",
    " friends",
    " and",
    " set",
    " off",
    " to",
    " stop",
    " Kirby",
    " and",
    " Maj",
    "in",
    " Bu",
    "u",
    ".",
    " Along",
    " the",
    " way",
    ",",
    " they",
    " encountered",
    " various",
    " challenges",
    " and",
    " obstacles",
    ",",
    " but",
    " their",
    " determination",
    " to",
    " save",
    " the",
    " world",
    " kept",
    " them",
    " going",
    ".\n\n",
    "As",
    " they",
    " finally",
    " caught",
    " up",
    " with",
    " the",
    " nef",
    "arious",
    " pair",
    ",",
    " a",
    " fierce",
    " battle",
    " ensued",
    ".",
    " Goku",
    " and",
    " his",
    " friends",
    " fought",
    " val",
    "iant",
    "ly",
    ",",
    " but",
    " Kirby",
    "'s",
    " incredible",
    " power",
    " and",
    " Maj",
    "in",
    " Bu",
    "u",
    "'s",
    " cunning",
    " abilities",
    " proved",
    " to",
    " be",
    " a",
    " formidable",
    " force",
    ".\n\n",
    "Just",
    " when",
    " all",
    " seemed",
    " lost",
    ",",
    " Goku",
    " had",
    " an",
    " idea",
    ".",
    " He",
    " remembered",
    " a",
    " technique",
    " he",
    " had",
    " learned",
    " from",
    " his",
    " training",
    " with",
    " King",
    " Kai",
    " -",
    " the",
    " Fusion",
    " technique",
    ".",
    " With",
    " the",
    " help",
    " of",
    " his",
    " friends",
    ",",
    " Goku",
    " fused",
    " with",
    " his",
    " son",
    " G",
    "ohan",
    ",",
    " creating",
    " a",
    " powerful",
    " new",
    " warrior",
    ",",
    " Go",
    "get",
    "a",
    ".\n\n",
    "Go",
    "get",
    "a",
    " charged",
    " forward",
    ",",
    " unle",
    "ashing",
    " a",
    " barrage",
    " of",
    " powerful",
    " attacks",
    " on",
    " Kirby",
    " and",
    " Maj",
    "in",
    " Bu",
    "u",
    ".",
    " The",
    " two",
    " villains",
    " fought",
    " back",
    " with",
    " all",
    " their",
    " might",
    ",",
    " but",
    " Go",
    "get",
    "a",
    "'s",
    " strength",
    " was",
    " too",
    " great",
    ".\n\n",
    "Finally",
    ",",
    " in",
    " a",
    " clim",
    "actic",
    " moment",
    ",",
    " Go",
    "get",
    "a",
    " delivered",
    " a",
    " final",
    " blow",
    ",",
    " defeating",
    " Kirby",
    " and",
    " Maj",
    "in",
    " Bu",
    "u",
    " and",
    " saving",
    " the",
    " world",
    " from",
    " destruction",
    ".",
    " As",
    " the",
    " dust",
    " settled",
    ",",
    " Goku",
    " and",
    " his",
    " friends",
    " breathed",
    " a",
    " sigh",
    " of",
    " relief",
    ",",
    " knowing",
    " that",
    " they",
    " had",
    " once",
    " again",
    " triumph",
    "ed",
    " over",
    " evil",
    ".\n\n",
    "Kir",
    "by",
    ",",
    " realizing",
    " the",
    " error",
    " of",
    " his",
    " ways",
    ",",
    " vowed",
    " to",
    " be",
    " a",
    " better",
    " hero",
    " and",
    " joined",
    " forces",
    " with",
    " Goku",
    " and",
    " his",
    " friends",
    " to",
    " protect",
    " the",
    " world",
    ".",
    " From",
    " that",
    " day",
    " on",
    ",",
    " the",
    " unlikely",
    " alliance",
    " of",
    " Goku",
    ",",
    " his",
    " friends",
    ",",
    " and",
    " Kirby",
    " worked",
    " together",
    " to",
    " maintain",
    " peace",
    " and",
    " justice",
    " throughout",
    " the",
    " galaxy",
    ".",
    " And",
    " so",
    ",",
    " a",
    " new",
    " chapter",
    " in",
    " their",
    " epic",
    " adventures",
    " began",
    ".",
    ""
  ],
  "started_at": "2024-06-28T18:22:08.434719Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.us.c.replicate.net/v1/streams/malq4fxzedzu25bkq44c7vul4jcjv6hjcespinmyoahcetvwne5a",
    "get": "https://api.replicate.com/v1/predictions/xzx1tqczqsrgj0cgc0aby4jzyc",
    "cancel": "https://api.replicate.com/v1/predictions/xzx1tqczqsrgj0cgc0aby4jzyc/cancel"
  },
  "version": "51ca4143b8464e9dfeb0c88339962a6bdf2ffd833f047ed293b38537a92c1515"
}

Generated in

13.0 seconds

Tweak it Share Report