microsoft/phi-3-medium-4k-instruct:c6913a88 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

microsoft /phi-3-medium-4k-instruct:c6913a88

Input

prompt

string

Shift + Return to add a new line

Can you provide ways to eat combinations of bananas and dragonfruits?Can you provide ways to eat combinations of bananas and dragonfruits?

Prompt

Default: ""

system_prompt

string

Shift + Return to add a new line

System prompt to send to the model. This is prepended to the prompt and helps guide system behavior. Ignored for non-chat models.

Default: "You are a helpful assistant."

min_tokens

integer

The minimum number of tokens the model should generate as output.

Default: 0

max_tokens

integer

The maximum number of tokens the model should generate as output.

Default: 512

temperature

number

The value used to modulate the next token probabilities.

Default: 0.6

top_p

number

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).

Default: 0.9

top_k

integer

The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).

Default: 50

presence_penalty

number

Presence penalty

Default: 0

frequency_penalty

number

Frequency penalty

Default: 0

stop_sequences

string

Shift + Return to add a new line

A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.

Run this model in Node.js with one line of code:

npx create-replicate --model=microsoft/phi-3-medium-4k-instruct

or set up a project from scratch

Install Replicate’s Node.js client library

npm install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run microsoft/phi-3-medium-4k-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "microsoft/phi-3-medium-4k-instruct:c6913a88bec85c7e014322393e63807fc5a54230cd488b8d5b566edbcb0e8f4c",
  {
    input: {
      top_k: 50,
      top_p: 0.9,
      prompt: "Can you provide ways to eat combinations of bananas and dragonfruits?",
      max_tokens: 256,
      min_tokens: 0,
      temperature: 0,
      system_prompt: "You are a helpful assistant.",
      presence_penalty: 0,
      frequency_penalty: 0
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library

pip install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client

import replicate

Run microsoft/phi-3-medium-4k-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "microsoft/phi-3-medium-4k-instruct:c6913a88bec85c7e014322393e63807fc5a54230cd488b8d5b566edbcb0e8f4c",
    input={
        "top_k": 50,
        "top_p": 0.9,
        "prompt": "Can you provide ways to eat combinations of bananas and dragonfruits?",
        "max_tokens": 256,
        "min_tokens": 0,
        "temperature": 0,
        "system_prompt": "You are a helpful assistant.",
        "presence_penalty": 0,
        "frequency_penalty": 0
    }
)

# The microsoft/phi-3-medium-4k-instruct model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/microsoft/phi-3-medium-4k-instruct/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run microsoft/phi-3-medium-4k-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "c6913a88bec85c7e014322393e63807fc5a54230cd488b8d5b566edbcb0e8f4c",
    "input": {
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "Can you provide ways to eat combinations of bananas and dragonfruits?",
      "max_tokens": 256,
      "min_tokens": 0,
      "temperature": 0,
      "system_prompt": "You are a helpful assistant.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Certainly! Here are some delicious and healthy combinations of bananas and dragonfruits that you can try: 1. Banana and Dragonfruit Smoothie: Blend together 1 banana, 1/2 dragonfruit, 1 cup of almond milk, and a handful of ice cubes for a refreshing smoothie. You can also add a tablespoon of honey or a scoop of protein powder for added sweetness and nutrition. 2. Banana and Dragonfruit Parfait: Layer sliced bananas, diced dragonfruit, and Greek yogurt in a glass or bowl. Top with granola, chopped nuts, and a drizzle of honey for a delicious and nutritious breakfast or snack. 3. Banana and Dragonfruit Salad: Combine sliced bananas, diced dragonfruit, and mixed greens in a bowl. Drizzle with a simple vinaigrette made from olive oil, lemon juice, honey, and Dijon mustard. Top with to

{
  "completed_at": "2024-07-24T04:05:59.828933Z",
  "created_at": "2024-07-24T04:03:49.656000Z",
  "data_removed": false,
  "error": null,
  "id": "8m81wnyv31rj40cgwbm8yhq8q4",
  "input": {
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "Can you provide ways to eat combinations of bananas and dragonfruits?",
    "max_tokens": 256,
    "min_tokens": 0,
    "temperature": 0,
    "system_prompt": "You are a helpful assistant.",
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "logs": "INFO 07-24 04:05:54 async_llm_engine.py:646] Received request c3c00c0305bb4b64a88368c460202398: prompt: '<|user|>\\nCan you provide ways to eat combinations of bananas and dragonfruits?<|end|>\\n<|assistant|>\\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[32000], include_stop_str_in_output=False, ignore_eos=False, max_tokens=256, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.\n stdoutINFO 07-24 04:05:59 metrics.py:295] Avg prompt throughput: 3.8 tokens/s, Avg generation throughput: 46.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.\n stdoutGeneration took 1721793164.83sFormatted prompt: <|user|>\nCan you provide ways to eat combinations of bananas and dragonfruits?<|end|>\n<|assistant|>\nINFO 07-24 04:05:59 async_llm_engine.py:134] Finished request c3c00c0305bb4b64a88368c460202398.\n stdout",
  "metrics": {
    "predict_time": 5.304694617,
    "total_time": 130.172933
  },
  "output": [
    " C",
    "ertain",
    "ly",
    "!",
    " Here",
    " are",
    " some",
    " del",
    "icious",
    " and",
    " health",
    "y",
    " combinations",
    " of",
    " ban",
    "anas",
    " and",
    " drag",
    "on",
    "f",
    "ruits",
    " that",
    " you",
    " can",
    " try",
    ":",
    "\n",
    "\n",
    "1",
    ".",
    " Ban",
    "ana",
    " and",
    " Dragon",
    "f",
    "ruit",
    " Sm",
    "ooth",
    "ie",
    ":",
    "\n",
    "Bl",
    "end",
    " together",
    " ",
    "1",
    " ban",
    "ana",
    ",",
    " ",
    "1",
    "/",
    "2",
    " drag",
    "on",
    "f",
    "ruit",
    ",",
    " ",
    "1",
    " cup",
    " of",
    " al",
    "mond",
    " milk",
    ",",
    " and",
    " a",
    " hand",
    "ful",
    " of",
    " ice",
    " cub",
    "es",
    " for",
    " a",
    " ref",
    "res",
    "hing",
    " smooth",
    "ie",
    ".",
    " You",
    " can",
    " also",
    " add",
    " a",
    " tables",
    "po",
    "on",
    " of",
    " h",
    "oney",
    " or",
    " a",
    " sc",
    "oop",
    " of",
    " protein",
    " pow",
    "der",
    " for",
    " added",
    " sweet",
    "ness",
    " and",
    " nut",
    "r",
    "ition",
    ".",
    "\n",
    "\n",
    "2",
    ".",
    " Ban",
    "ana",
    " and",
    " Dragon",
    "f",
    "ruit",
    " Par",
    "f",
    "ait",
    ":",
    "\n",
    "Layer",
    " s",
    "lic",
    "ed",
    " ban",
    "anas",
    ",",
    " d",
    "iced",
    " drag",
    "on",
    "f",
    "ruit",
    ",",
    " and",
    " Greek",
    " y",
    "og",
    "urt",
    " in",
    " a",
    " glass",
    " or",
    " bow",
    "l",
    ".",
    " Top",
    " with",
    " gran",
    "ola",
    ",",
    " cho",
    "pped",
    " n",
    "uts",
    ",",
    " and",
    " a",
    " dri",
    "zz",
    "le",
    " of",
    " h",
    "oney",
    " for",
    " a",
    " del",
    "icious",
    " and",
    " nut",
    "rit",
    "ious",
    " breakfast",
    " or",
    " sn",
    "ack",
    ".",
    "\n",
    "\n",
    "3",
    ".",
    " Ban",
    "ana",
    " and",
    " Dragon",
    "f",
    "ruit",
    " Sal",
    "ad",
    ":",
    "\n",
    "Com",
    "bine",
    " s",
    "lic",
    "ed",
    " ban",
    "anas",
    ",",
    " d",
    "iced",
    " drag",
    "on",
    "f",
    "ruit",
    ",",
    " and",
    " mixed",
    " gre",
    "ens",
    " in",
    " a",
    " bow",
    "l",
    ".",
    " D",
    "ri",
    "zz",
    "le",
    " with",
    " a",
    " simple",
    " v",
    "ina",
    "ig",
    "ret",
    "te",
    " made",
    " from",
    " o",
    "live",
    " oil",
    ",",
    " le",
    "mon",
    " ju",
    "ice",
    ",",
    " h",
    "oney",
    ",",
    " and",
    " D",
    "ij",
    "on",
    " must",
    "ard",
    ".",
    " Top",
    " with",
    " to"
  ],
  "started_at": "2024-07-24T04:05:54.524239Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.rno2.c.replicate.net/v1/streams/yx6xoiy5z6spnuhucjwdrnwyyulo6mpa2agdcxuycxsh4x5ne6bq",
    "get": "https://api.replicate.com/v1/predictions/8m81wnyv31rj40cgwbm8yhq8q4",
    "cancel": "https://api.replicate.com/v1/predictions/8m81wnyv31rj40cgwbm8yhq8q4/cancel"
  },
  "version": "c6913a88bec85c7e014322393e63807fc5a54230cd488b8d5b566edbcb0e8f4c"
}

Generated in

5.3 seconds

Tweak it Share Report