ibm-granite / granite-20b-code-instruct-8k

Join the Granite community where you can find numerous recipe workbooks to help you get started with a wide variety of use cases using this model. https://github.com/ibm-granite-community

Cold

Public
110K runs
Priced per token
GitHub
Paper
License

Run with an API

Playground API Examples README

Prediction

ibm-granite/granite-20b-code-instruct-8k

Official model

ibm-granite/granite-20b-code-instruct-8k

4rfr3bw6q5rm00cjkda88c6j6c

Status

Succeeded

Source

Web

Total duration

1.2s

Created

7 months ago

Input

top_p: 0.9
prompt: Could you please explain what APR means?
max_tokens: 512
min_tokens: 0
temperature: 0.6
system_prompt: You are an expert in finance that knows many concepts related to loans and credit.
presence_penalty: 0
frequency_penalty: 0

{
  "top_p": 0.9,
  "prompt": "Could you please explain what APR means?",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
  "presence_penalty": 0,
  "frequency_penalty": 0
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const input = {
  top_p: 0.9,
  prompt: "Could you please explain what APR means?",
  max_tokens: 512,
  min_tokens: 0,
  temperature: 0.6,
  system_prompt: "You are an expert in finance that knows many concepts related to loans and credit.",
  presence_penalty: 0,
  frequency_penalty: 0
};

for await (const event of replicate.stream("ibm-granite/granite-20b-code-instruct-8k", { input })) {
  process.stdout.write(event.toString());
};

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

# The ibm-granite/granite-20b-code-instruct-8k model can stream output as it's running.
for event in replicate.stream(
    "ibm-granite/granite-20b-code-instruct-8k",
    input={
        "top_p": 0.9,
        "prompt": "Could you please explain what APR means?",
        "max_tokens": 512,
        "min_tokens": 0,
        "temperature": 0.6,
        "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
        "presence_penalty": 0,
        "frequency_penalty": 0
    },
):
    print(str(event), end="")

To learn more, take a look at the guide on getting started with Python.

Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "input": {
      "top_p": 0.9,
      "prompt": "Could you please explain what APR means?",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/models/ibm-granite/granite-20b-code-instruct-8k/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

APR stands for Annual Percentage Rate. It is a figure that represents the annual cost of borrowing, including fees and interest. It is used to help consumers compare the costs of different loans and credit products. The higher the APR, the more expensive the loan is.

{
  "completed_at": "2024-10-17T16:34:22.167598Z",
  "created_at": "2024-10-17T16:34:20.985000Z",
  "data_removed": false,
  "error": null,
  "id": "4rfr3bw6q5rm00cjkda88c6j6c",
  "input": {
    "top_p": 0.9,
    "prompt": "Could you please explain what APR means?",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "logs": "INFO 10-17 16:34:20 async_llm_engine.py:173] Added request f21156ed57234d8585b9b09777df8582.\nINFO 10-17 16:34:21 metrics.py:396] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.\nGeneration took 1729182595.16sFormatted prompt: System:\nYou are an expert in finance that knows many concepts related to loans and credit.\n\nQuestion:\nCould you please explain what APR means?\n\nAnswer:\nRandom seed used: `99346`\nNote: Random seed will not impact output if greedy decoding is used.\nINFO 10-17 16:34:22 async_llm_engine.py:140] Finished request f21156ed57234d8585b9b09777df8582.",
  "metrics": {
    "batch_size": 1.0006387106010313,
    "input_token_count": 39,
    "tokens_per_second": 49.61475799941566,
    "output_token_count": 58,
    "predict_time_share": 1.1756401062011719,
    "predict_time": 1.177441129,
    "total_time": 1.182598,
    "time_to_first_token": 0.013591117
  },
  "output": [
    "AP",
    "R",
    " stands",
    " for",
    " Ann",
    "ual",
    " Per",
    "centage",
    " Rate",
    ".",
    " It",
    " is",
    " a",
    " figure",
    " that",
    " represents",
    " the",
    " annual",
    " cost",
    " of",
    " borrow",
    "ing",
    ",",
    " including",
    " fees",
    " and",
    " interest",
    ".",
    " It",
    " is",
    " used",
    " to",
    " help",
    " consumers",
    " compare",
    " the",
    " costs",
    " of",
    " different",
    " lo",
    "ans",
    " and",
    " credit",
    " products",
    ".",
    " The",
    " higher",
    " the",
    " APR",
    ",",
    " the",
    " more",
    " expensive",
    " the",
    " loan",
    " is",
    ".",
    ""
  ],
  "started_at": "2024-10-17T16:34:20.990157Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://stream-b.svc.sea.v.replicate.net/v1/streams/wuruoxlkb3th2ds7dk5gjumk7mxjxrclvctwd6lajtfn25f4bg3q",
    "get": "https://api.replicate.com/v1/predictions/4rfr3bw6q5rm00cjkda88c6j6c",
    "cancel": "https://api.replicate.com/v1/predictions/4rfr3bw6q5rm00cjkda88c6j6c/cancel"
  },
  "version": "hidden"
}

Generated in

1.2 seconds

Input tokens

Output tokens

Tokens per second

49.61 tokens / second

Time to first token

14 milliseconds

Tweak it Share Report

Prediction

ibm-granite/granite-20b-code-instruct-8k

Official model

ibm-granite/granite-20b-code-instruct-8k

w4915ttdhxrm20cjkdarc0zvg0

Status

Succeeded

Source

Web

Total duration

4.3s

Created

7 months ago

Input

top_p: 0.9
prompt: Now, please explain APR meaning with a simple example, so that even a 5 years old kid can understand you.
max_tokens: 512
min_tokens: 0
temperature: 0.6
system_prompt: You are an expert in finance that knows many concepts related to loans and credit.
presence_penalty: 0
frequency_penalty: 0

{
  "top_p": 0.9,
  "prompt": "Now, please explain APR meaning with a simple example, so that even a 5 years old kid can understand you.",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
  "presence_penalty": 0,
  "frequency_penalty": 0
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const input = {
  top_p: 0.9,
  prompt: "Now, please explain APR meaning with a simple example, so that even a 5 years old kid can understand you.",
  max_tokens: 512,
  min_tokens: 0,
  temperature: 0.6,
  system_prompt: "You are an expert in finance that knows many concepts related to loans and credit.",
  presence_penalty: 0,
  frequency_penalty: 0
};

for await (const event of replicate.stream("ibm-granite/granite-20b-code-instruct-8k", { input })) {
  process.stdout.write(event.toString());
};

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

# The ibm-granite/granite-20b-code-instruct-8k model can stream output as it's running.
for event in replicate.stream(
    "ibm-granite/granite-20b-code-instruct-8k",
    input={
        "top_p": 0.9,
        "prompt": "Now, please explain APR meaning with a simple example, so that even a 5 years old kid can understand you.",
        "max_tokens": 512,
        "min_tokens": 0,
        "temperature": 0.6,
        "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
        "presence_penalty": 0,
        "frequency_penalty": 0
    },
):
    print(str(event), end="")

To learn more, take a look at the guide on getting started with Python.

Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "input": {
      "top_p": 0.9,
      "prompt": "Now, please explain APR meaning with a simple example, so that even a 5 years old kid can understand you.",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/models/ibm-granite/granite-20b-code-instruct-8k/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Sure, I'd be happy to explain it to a 5-year-old! Let's say you want to buy a toy that costs $10. You ask your parents if you can borrow the money, and they agree. You sign a loan agreement promising to pay them back the $10 plus some extra money, called interest. The interest is calculated as a percentage of the loan amount, and it's called the Annual Percentage Rate, or APR for short. For example, let's say the APR is 5%. This means that you have to pay an extra $0.50 for each dollar you borrow. So, if you borrow $10, you have to pay $10 + ($10 * 0.05) = $10.50 back. The APR is important because it helps you understand how much extra money you'll have to pay back. The higher the APR, the more interest you'll have to pay. So, you want to make sure the APR is as low as possible when you take a loan.

{
  "completed_at": "2024-10-17T16:35:16.158605Z",
  "created_at": "2024-10-17T16:35:11.887000Z",
  "data_removed": false,
  "error": null,
  "id": "w4915ttdhxrm20cjkdarc0zvg0",
  "input": {
    "top_p": 0.9,
    "prompt": "Now, please explain APR meaning with a simple example, so that even a 5 years old kid can understand you.",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "logs": "INFO 10-17 16:35:11 async_llm_engine.py:173] Added request fa211c5f76294cecbfec374e820625c3.\nINFO 10-17 16:35:11 metrics.py:396] Avg prompt throughput: 1.1 tokens/s, Avg generation throughput: 1.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.\nGeneration took 1729182103.16sINFO 10-17 16:35:16 async_llm_engine.py:140] Finished request fa211c5f76294cecbfec374e820625c3.\nFormatted prompt: System:\nYou are an expert in finance that knows many concepts related to loans and credit.\n\nQuestion:\nNow, please explain APR meaning with a simple example, so that even a 5 years old kid can understand you.\n\nAnswer:\nRandom seed used: `55125`\nNote: Random seed will not impact output if greedy decoding is used.",
  "metrics": {
    "batch_size": 1.0001717241333348,
    "input_token_count": 55,
    "tokens_per_second": 53.94693117006514,
    "output_token_count": 230,
    "predict_time_share": 4.264168739318848,
    "predict_time": 4.266457472,
    "total_time": 4.271605,
    "time_to_first_token": 0.008155282
  },
  "output": [
    "S",
    "ure",
    ",",
    " I",
    "'d",
    " be",
    " happy",
    " to",
    " explain",
    " it",
    " to",
    " a",
    " ",
    "5",
    "-",
    "year",
    "-",
    "old",
    "!",
    "\n",
    "\n",
    "Let",
    "'s",
    " say",
    " you",
    " want",
    " to",
    " buy",
    " a",
    " to",
    "y",
    " that",
    " costs",
    " $",
    "1",
    "0",
    ".",
    " You",
    " ask",
    " your",
    " parents",
    " if",
    " you",
    " can",
    " borrow",
    " the",
    " money",
    ",",
    " and",
    " they",
    " agree",
    ".",
    " You",
    " sign",
    " a",
    " loan",
    " agreement",
    " prom",
    "ising",
    " to",
    " pay",
    " them",
    " back",
    " the",
    " $",
    "1",
    "0",
    " plus",
    " some",
    " extra",
    " money",
    ",",
    " called",
    " interest",
    ".",
    " The",
    " interest",
    " is",
    " calculated",
    " as",
    " a",
    " percentage",
    " of",
    " the",
    " loan",
    " amount",
    ",",
    " and",
    " it",
    "'s",
    " called",
    " the",
    " Ann",
    "ual",
    " Per",
    "centage",
    " Rate",
    ",",
    " or",
    " APR",
    " for",
    " short",
    ".",
    "\n",
    "\n",
    "For",
    " example",
    ",",
    " let",
    "'s",
    " say",
    " the",
    " APR",
    " is",
    " ",
    "5",
    "%.",
    " This",
    " means",
    " that",
    " you",
    " have",
    " to",
    " pay",
    " an",
    " extra",
    " $",
    "0",
    ".",
    "5",
    "0",
    " for",
    " each",
    " dollar",
    " you",
    " borrow",
    ".",
    " So",
    ",",
    " if",
    " you",
    " borrow",
    " $",
    "1",
    "0",
    ",",
    " you",
    " have",
    " to",
    " pay",
    " $",
    "1",
    "0",
    " +",
    " ($",
    "1",
    "0",
    " *",
    " ",
    "0",
    ".",
    "0",
    "5",
    ")",
    " =",
    " $",
    "1",
    "0",
    ".",
    "5",
    "0",
    " back",
    ".",
    "\n",
    "\n",
    "The",
    " APR",
    " is",
    " important",
    " because",
    " it",
    " helps",
    " you",
    " understand",
    " how",
    " much",
    " extra",
    " money",
    " you",
    "'ll",
    " have",
    " to",
    " pay",
    " back",
    ".",
    " The",
    " higher",
    " the",
    " APR",
    ",",
    " the",
    " more",
    " interest",
    " you",
    "'ll",
    " have",
    " to",
    " pay",
    ".",
    " So",
    ",",
    " you",
    " want",
    " to",
    " make",
    " sure",
    " the",
    " APR",
    " is",
    " as",
    " low",
    " as",
    " possible",
    " when",
    " you",
    " take",
    " a",
    " loan",
    ".",
    ""
  ],
  "started_at": "2024-10-17T16:35:11.892147Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://stream-b.svc.sea.v.replicate.net/v1/streams/lzoviwv5mhigpi7ism5am5g7xurh4mfqmobsbcqw7iihbxcnf7wq",
    "get": "https://api.replicate.com/v1/predictions/w4915ttdhxrm20cjkdarc0zvg0",
    "cancel": "https://api.replicate.com/v1/predictions/w4915ttdhxrm20cjkdarc0zvg0/cancel"
  },
  "version": "hidden"
}

Generated in

4.3 seconds

Input tokens

Output tokens

230

Tokens per second

53.95 tokens / second

Time to first token

8 milliseconds

Tweak it Share Report