Prediction

Model

joehoover/zephyr-7b-alpha:14ec6336

vcxozelbrhlz7bzmukjromel7a

Status

Succeeded

Source

Web

Hardware

A40 (Large)

Total duration

4.7s

Created

about 2 years ago

Webhook

–

Input

prompt: Can you help me answer a question?</s> <|assistant|> I'd be happy to help you answer any question you have. Please provide me with the question you'd like assistance with, and I'll do my best to provide you with an answer.</s> <|user|> Please help understand this riddle: "I’m tall when I’m young, and I’m short when I’m old. What am I?"
system_prompt: You are a helpful assistant.
max_new_tokens: 128
min_new_tokens: -1
temperature: 0.75
top_p: 0.9
top_k: 50
debug: false
return_logits: false

{
  "debug": false,
  "max_new_tokens": 128,
  "min_new_tokens": -1,
  "prompt": "Can you help me answer a question?</s>\n<|assistant|>\nI'd be happy to help you answer any question you have. Please provide me with the question you'd like assistance with, and I'll do my best to provide you with an answer.</s>\n<|user|>\nPlease help understand this riddle: \"I’m tall when I’m young, and I’m short when I’m old. What am I?\"",
  "return_logits": false,
  "system_prompt": "You are a helpful assistant.",
  "temperature": 0.75,
  "top_k": 50,
  "top_p": 0.9
}

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=r8_UKh**********************************

This is your API token. Keep it to yourself.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run joehoover/zephyr-7b-alpha using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "joehoover/zephyr-7b-alpha:14ec63365a1141134c41b652fe798633f48b1fd28b356725c4d8842a0ac151ee",
  {
    input: {
      debug: false,
      max_new_tokens: 128,
      min_new_tokens: -1,
      prompt: "Can you help me answer a question?</s>\n<|assistant|>\nI'd be happy to help you answer any question you have. Please provide me with the question you'd like assistance with, and I'll do my best to provide you with an answer.</s>\n<|user|>\nPlease help understand this riddle: \"I’m tall when I’m young, and I’m short when I’m old. What am I?\"",
      return_logits: false,
      system_prompt: "You are a helpful assistant.",
      temperature: 0.75,
      top_k: 50,
      top_p: 0.9
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=r8_UKh**********************************

This is your API token. Keep it to yourself.

Import the client:

import replicate

Run joehoover/zephyr-7b-alpha using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "joehoover/zephyr-7b-alpha:14ec63365a1141134c41b652fe798633f48b1fd28b356725c4d8842a0ac151ee",
    input={
        "debug": False,
        "max_new_tokens": 128,
        "min_new_tokens": -1,
        "prompt": "Can you help me answer a question?</s>\n<|assistant|>\nI'd be happy to help you answer any question you have. Please provide me with the question you'd like assistance with, and I'll do my best to provide you with an answer.</s>\n<|user|>\nPlease help understand this riddle: \"I’m tall when I’m young, and I’m short when I’m old. What am I?\"",
        "return_logits": False,
        "system_prompt": "You are a helpful assistant.",
        "temperature": 0.75,
        "top_k": 50,
        "top_p": 0.9
    }
)

# The joehoover/zephyr-7b-alpha model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/joehoover/zephyr-7b-alpha/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=r8_UKh**********************************

This is your API token. Keep it to yourself.

Run joehoover/zephyr-7b-alpha using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "joehoover/zephyr-7b-alpha:14ec63365a1141134c41b652fe798633f48b1fd28b356725c4d8842a0ac151ee",
    "input": {
      "debug": false,
      "max_new_tokens": 128,
      "min_new_tokens": -1,
      "prompt": "Can you help me answer a question?</s>\\n<|assistant|>\\nI\'d be happy to help you answer any question you have. Please provide me with the question you\'d like assistance with, and I\'ll do my best to provide you with an answer.</s>\\n<|user|>\\nPlease help understand this riddle: \\"I’m tall when I’m young, and I’m short when I’m old. What am I?\\"",
      "return_logits": false,
      "system_prompt": "You are a helpful assistant.",
      "temperature": 0.75,
      "top_k": 50,
      "top_p": 0.9
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Certainly! The answer to this riddle is a "candle". When a candle is young, it's tall, but as it burns and gets shorter, it becomes shorter and shorter until it eventually extinguishes, at which point it's no longer "tall" or "short".

{
  "id": "vcxozelbrhlz7bzmukjromel7a",
  "model": "joehoover/zephyr-7b-alpha",
  "version": "14ec63365a1141134c41b652fe798633f48b1fd28b356725c4d8842a0ac151ee",
  "input": {
    "debug": false,
    "max_new_tokens": 128,
    "min_new_tokens": -1,
    "prompt": "Can you help me answer a question?</s>\n<|assistant|>\nI'd be happy to help you answer any question you have. Please provide me with the question you'd like assistance with, and I'll do my best to provide you with an answer.</s>\n<|user|>\nPlease help understand this riddle: \"I’m tall when I’m young, and I’m short when I’m old. What am I?\"",
    "return_logits": false,
    "system_prompt": "You are a helpful assistant.",
    "temperature": 0.75,
    "top_k": 50,
    "top_p": 0.9
  },
  "logs": "You are a helpful assistant.\nPROMPT_TEMPLATE: <|system|>\n{system_prompt}</s>\n<|user|>\n{instruction}</s>\n<|assistant|>\nYour formatted prompt is:\n<|system|>\nYou are a helpful assistant.</s>\n<|user|>\nCan you help me answer a question?</s>\n<|assistant|>\nI'd be happy to help you answer any question you have. Please provide me with the question you'd like assistance with, and I'll do my best to provide you with an answer.</s>\n<|user|>\nPlease help understand this riddle: \"I’m tall when I’m young, and I’m short when I’m old. What am I?\"</s>\n<|assistant|>\nDisabling loras\nNo loras were ever loaded, nothing to disable.\nNot using LoRA\nTokens:  tensor([[    1,   523, 28766,  6574, 28766, 28767,    13,  1976,   460,   264,\n10865, 13892, 28723,     2,   523, 28766,  1838, 28766, 28767,    13,\n6325,   368,  1316,   528,  4372,   264,  2996, 28804,     2,   523,\n28766,   489, 11143, 28766, 28767,    13, 28737, 28742, 28715,   347,\n4610,   298,  1316,   368,  4372,   707,  2996,   368,   506, 28723,\n5919,  3084,   528,   395,   272,  2996,   368, 28742, 28715,   737,\n11611,   395, 28725,   304,   315, 28742,   584,   511,   586,  1489,\n298,  3084,   368,   395,   396,  4372, 28723,     2,   523, 28766,\n1838, 28766, 28767,    13, 12069,  1316,  2380,   456,   408,  3287,\n28747,   345, 28737, 28809, 28719,  9369,   739,   315, 28809, 28719,\n2518, 28725,   304,   315, 28809, 28719,  2485,   739,   315, 28809,\n28719,  1571, 28723,  1824,   837,   315,  1110,     2,   523, 28766,\n489, 11143, 28766, 28767,    13]], device='cuda:0')\nhostname: model-14ec6336-bdb00b47d4679dba-gpu-a40-557dfb9fc5-frhh6",
  "output": [
    "",
    "",
    "",
    "",
    "Certainly! ",
    "The ",
    "answer ",
    "to ",
    "this ",
    "",
    "riddle ",
    "is ",
    "a ",
    "",
    "",
    "",
    "",
    "\"candle\". ",
    "When ",
    "a ",
    "candle ",
    "is ",
    "",
    "young, ",
    "",
    "",
    "it's ",
    "",
    "tall, ",
    "but ",
    "as ",
    "it ",
    "",
    "burns ",
    "and ",
    "gets ",
    "",
    "shorter, ",
    "it ",
    "becomes ",
    "shorter ",
    "and ",
    "shorter ",
    "until ",
    "it ",
    "eventually ",
    "",
    "",
    "",
    "",
    "extinguishes, ",
    "at ",
    "which ",
    "point ",
    "",
    "",
    "it's ",
    "no ",
    "longer ",
    "",
    "",
    "",
    "\"tall\" ",
    "or ",
    "",
    "",
    "",
    "\"short\"."
  ],
  "data_removed": false,
  "error": null,
  "source": "web",
  "status": "succeeded",
  "created_at": "2023-10-13T16:57:46.114914Z",
  "started_at": "2023-10-13T16:57:46.07471Z",
  "completed_at": "2023-10-13T16:57:50.793307Z",
  "urls": {
    "cancel": "https://api.replicate.com/v1/predictions/vcxozelbrhlz7bzmukjromel7a/cancel",
    "get": "https://api.replicate.com/v1/predictions/vcxozelbrhlz7bzmukjromel7a"
  },
  "metrics": {
    "predict_time": 4.718597,
    "total_time": 4.678393
  }
}

Generated in

4.7 seconds

Tweak it Report