typetext
{
"frequency_penalty": 0,
"max_tokens": 512,
"min_tokens": 0,
"presence_penalty": 0,
"prompt": "Could you please explain what APR means?",
"system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
"temperature": 0.6,
"top_p": 0.9
}npm install replicate
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=r8_84T**********************************
This is your API token. Keep it to yourself.
import Replicate from "replicate";
const replicate = new Replicate({
auth: process.env.REPLICATE_API_TOKEN,
});
Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const input = {
frequency_penalty: 0,
max_tokens: 512,
min_tokens: 0,
presence_penalty: 0,
prompt: "Could you please explain what APR means?",
system_prompt: "You are an expert in finance that knows many concepts related to loans and credit.",
temperature: 0.6,
top_p: 0.9
};
for await (const event of replicate.stream("ibm-granite/granite-20b-code-instruct-8k", { input })) {
process.stdout.write(event.toString());
};
To learn more, take a look at the guide on getting started with Node.js.
pip install replicate
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=r8_84T**********************************
This is your API token. Keep it to yourself.
import replicate
Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
# The ibm-granite/granite-20b-code-instruct-8k model can stream output as it's running.
for event in replicate.stream(
"ibm-granite/granite-20b-code-instruct-8k",
input={
"frequency_penalty": 0,
"max_tokens": 512,
"min_tokens": 0,
"presence_penalty": 0,
"prompt": "Could you please explain what APR means?",
"system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
"temperature": 0.6,
"top_p": 0.9
},
):
print(str(event), end="")
To learn more, take a look at the guide on getting started with Python.
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=r8_84T**********************************
This is your API token. Keep it to yourself.
Run ibm-granite/granite-20b-code-instruct-8k using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
-H "Content-Type: application/json" \
-H "Prefer: wait" \
-d $'{
"input": {
"frequency_penalty": 0,
"max_tokens": 512,
"min_tokens": 0,
"presence_penalty": 0,
"prompt": "Could you please explain what APR means?",
"system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
"temperature": 0.6,
"top_p": 0.9
}
}' \
https://api.replicate.com/v1/models/ibm-granite/granite-20b-code-instruct-8k/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
APR stands for Annual Percentage Rate. It is a figure that represents the annual cost of borrowing, including fees and interest. It is used to help consumers compare the costs of different loans and credit products. The higher the APR, the more expensive the loan is.
{
"id": "4rfr3bw6q5rm00cjkda88c6j6c",
"model": "ibm-granite/granite-20b-code-instruct-8k",
"version": "hidden",
"input": {
"frequency_penalty": 0,
"max_tokens": 512,
"min_tokens": 0,
"presence_penalty": 0,
"prompt": "Could you please explain what APR means?",
"system_prompt": "You are an expert in finance that knows many concepts related to loans and credit.",
"temperature": 0.6,
"top_p": 0.9
},
"logs": "INFO 10-17 16:34:20 async_llm_engine.py:173] Added request f21156ed57234d8585b9b09777df8582.\nINFO 10-17 16:34:21 metrics.py:396] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.\nGeneration took 1729182595.16sFormatted prompt: System:\nYou are an expert in finance that knows many concepts related to loans and credit.\n\nQuestion:\nCould you please explain what APR means?\n\nAnswer:\nRandom seed used: `99346`\nNote: Random seed will not impact output if greedy decoding is used.\nINFO 10-17 16:34:22 async_llm_engine.py:140] Finished request f21156ed57234d8585b9b09777df8582.",
"output": [
"AP",
"R",
" stands",
" for",
" Ann",
"ual",
" Per",
"centage",
" Rate",
".",
" It",
" is",
" a",
" figure",
" that",
" represents",
" the",
" annual",
" cost",
" of",
" borrow",
"ing",
",",
" including",
" fees",
" and",
" interest",
".",
" It",
" is",
" used",
" to",
" help",
" consumers",
" compare",
" the",
" costs",
" of",
" different",
" lo",
"ans",
" and",
" credit",
" products",
".",
" The",
" higher",
" the",
" APR",
",",
" the",
" more",
" expensive",
" the",
" loan",
" is",
".",
""
],
"data_removed": false,
"error": null,
"source": "web",
"status": "succeeded",
"created_at": "2024-10-17T16:34:20.985Z",
"started_at": "2024-10-17T16:34:20.990157Z",
"completed_at": "2024-10-17T16:34:22.167598Z",
"urls": {
"cancel": "https://api.replicate.com/v1/predictions/4rfr3bw6q5rm00cjkda88c6j6c/cancel",
"get": "https://api.replicate.com/v1/predictions/4rfr3bw6q5rm00cjkda88c6j6c",
"stream": "https://stream-b.svc.sea.v.replicate.net/v1/streams/wuruoxlkb3th2ds7dk5gjumk7mxjxrclvctwd6lajtfn25f4bg3q",
"web": "https://replicate.com/p/4rfr3bw6q5rm00cjkda88c6j6c"
},
"metrics": {
"batch_size": 1.0006387106010313,
"input_token_count": 39,
"output_token_count": 58,
"predict_time": 1.177441129,
"predict_time_share": 1.1756401062011719,
"time_to_first_token": 0.013591117,
"tokens_per_second": 49.61475799941566,
"total_time": 1.182598
}
}