snowflake/snowflake-arctic-instruct – Run with an API on Replicate

Official

snowflake / snowflake-arctic-instruct

An efficient, intelligent, and truly open-source language model

Warm

Public
2M runs
Priced per token
GitHub
License

Run with an API

Playground API Examples README

Input

prompt

string

Shift + Return to add a new line

Prompt to send to the model.

Default: ""

system_prompt

string

Shift + Return to add a new line

System prompt to send to the model. This is prepended to the prompt and helps guide system behavior.

Default: "You are a helpful assistant."

top_p

number

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751). Lower to ignore less likely tokens.

Default: 0.9

temperature

number

The value used to modulate the next token probabilities. Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.

Default: 0.6

max_new_tokens

integer

The maximum number of tokens the model should generate as output. A word is generally 2-3 tokens.

Default: 512

min_new_tokens

integer

Deprecated. This input no longer has any effect.

Default: 0

top_k

integer

Deprecated. This input no longer has any effect.

Default: 50

presence_penalty

number

Deprecated. This input no longer has any effect.

Default: 1.15

frequency_penalty

number

Deprecated. This input no longer has any effect.

Default: 0.2

stop_sequences

string

Shift + Return to add a new line

Default: "<|im_end|>"

Run this model in Node.js with one line of code:

npx create-replicate --model=snowflake/snowflake-arctic-instruct

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run snowflake/snowflake-arctic-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const input = {
  top_k: 50,
  top_p: 0.9,
  prompt: "Write fizz buzz in SQL",
  temperature: 0.2,
  system_prompt: "You are a helpful assistant.",
  max_new_tokens: 512,
  min_new_tokens: 0,
  stop_sequences: "<|im_end|>",
  presence_penalty: 1.15,
  frequency_penalty: 0.2
};

for await (const event of replicate.stream("snowflake/snowflake-arctic-instruct", { input })) {
  process.stdout.write(event.toString());
};

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run snowflake/snowflake-arctic-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

# The snowflake/snowflake-arctic-instruct model can stream output as it's running.
for event in replicate.stream(
    "snowflake/snowflake-arctic-instruct",
    input={
        "top_k": 50,
        "top_p": 0.9,
        "prompt": "Write fizz buzz in SQL",
        "temperature": 0.2,
        "system_prompt": "You are a helpful assistant.",
        "max_new_tokens": 512,
        "min_new_tokens": 0,
        "stop_sequences": "<|im_end|>",
        "presence_penalty": 1.15,
        "frequency_penalty": 0.2
    },
):
    print(str(event), end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run snowflake/snowflake-arctic-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "input": {
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "Write fizz buzz in SQL",
      "temperature": 0.2,
      "system_prompt": "You are a helpful assistant.",
      "max_new_tokens": 512,
      "min_new_tokens": 0,
      "stop_sequences": "<|im_end|>",
      "presence_penalty": 1.15,
      "frequency_penalty": 0.2
    }
  }' \
  https://api.replicate.com/v1/models/snowflake/snowflake-arctic-instruct/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Fizz Buzz is a common programming problem that involves printing numbers from 1 to 100, replacing multiples of 3 with "Fizz", multiples of 5 with "Buzz", and multiples of both 3 and 5 with "FizzBuzz". Here's how you can solve this problem in SQL: ```sql WITH RECURSIVE fizz_buzz AS ( SELECT 1 AS num UNION ALL SELECT num + 1 FROM fizz_buzz WHERE num < 100 ) SELECT CASE WHEN MOD(num, 3) = 0 AND MOD(num, 5) = 0 THEN 'FizzBuzz' WHEN MOD(num, 3) = 0 THEN 'Fizz' WHEN MOD(num, 5) = 0 THEN 'Buzz' ELSE CAST(num AS CHAR) END AS result FROM fizz_buzz; ```

{
  "completed_at": "2024-04-24T03:58:33Z",
  "created_at": "2024-04-24T03:58:30.302000Z",
  "data_removed": false,
  "error": "",
  "id": "3knp1rt7ksrhw0cf1s0vgxh2s0",
  "input": {
    "top_p": 0.9,
    "prompt": "Write fizz buzz in SQL",
    "temperature": 0.2,
    "max_new_tokens": 512,
    "min_new_tokens": 0,
    "stop_sequences": "<|im_end|>",
    "prompt_template": "<|im_start|>system\nYou're a helpful assistant<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n\n<|im_start|>assistant\n",
    "presence_penalty": 1.15,
    "frequency_penalty": 0.2
  },
  "logs": "INFO 04-24 03:58:30 async_llm_engine.py:508] Received request 709f2034294045d7aacd6694b23a02aa: prompt: \"<|im_start|>system\\nYou're a helpful assistant<|im_end|>\\n<|im_start|>user\\nWrite fizz buzz in SQL<|im_end|>\\n\\n<|im_start|>assistant\\n\", sampling_params: SamplingParams(n=1, best_of=1, presence_penalty=1.15, frequency_penalty=0.2, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=50, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=['<|im_end|>', '<|im_end|>'], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.\r\n stdoutINFO 04-24 03:58:33 async_llm_engine.py:120] Finished request 709f2034294045d7aacd6694b23a02aa.\r\n stdoutTokens/second: 79.51895574949201\r\n stdout",
  "metrics": {
    "predict_time": 3,
    "total_time": 2.698
  },
  "output": [
    "Fiz",
    "z ",
    "Buzz is",
    " a common pr",
    "ogrammin",
    "g pro",
    "blem that",
    " involves",
    " printin",
    "g num",
    "b",
    "e",
    "rs ",
    "f",
    "r",
    "o",
    "m",
    " ",
    "1 to 100, ",
    "replac",
    "ing ",
    "mul",
    "t",
    "i",
    "ples ",
    "of",
    " ",
    "3 w",
    "it",
    "h \"Fiz",
    "z\", ",
    "mul",
    "t",
    "i",
    "ples ",
    "of",
    " ",
    "5 w",
    "it",
    "h \"B",
    "uzz\", ",
    "and ",
    "mul",
    "tiple",
    "s",
    " ",
    "of b",
    "o",
    "t",
    "h 3 a",
    "nd",
    " ",
    "5 w",
    "i",
    "th ",
    "\"F",
    "izzBu",
    "z",
    "z",
    "\". H",
    "ere'",
    "s ho",
    "w you ",
    "can s",
    "olve thi",
    "s p",
    "robl",
    "e",
    "m",
    " ",
    "in ",
    "SQL",
    ":",
    "\n",
    "\n``",
    "`sq",
    "l\nW",
    "IT",
    "H ",
    "RE",
    "CUR",
    "S",
    "IV",
    "E ",
    "fiz",
    "z_",
    "b",
    "uzz",
    " AS (\n ",
    " ",
    " ",
    " SE",
    "LECT",
    " ",
    "1 A",
    "S num\n",
    "    ",
    "U",
    "NIO",
    "N ALL\n ",
    "   S",
    "EL",
    "E",
    "C",
    "T num",
    " +",
    " 1 ",
    "F",
    "RO",
    "M ",
    "fizz_b",
    "uzz ",
    "WH",
    "E",
    "R",
    "E",
    " ",
    "n",
    "u",
    "m",
    " < 100",
    "\n",
    ")\nS",
    "ELECT",
    "\n",
    "    CAS",
    "E\n   ",
    "  ",
    "  ",
    " ",
    "WHE",
    "N",
    " ",
    "M",
    "O",
    "D(",
    "n",
    "u",
    "m, 3",
    ") ",
    "= ",
    "0",
    " AN",
    "D",
    " ",
    "M",
    "O",
    "D(",
    "n",
    "u",
    "m, 5)",
    " =",
    " ",
    "0 T",
    "H",
    "EN ",
    "'",
    "F",
    "izzBuzz",
    "'\n   ",
    "  ",
    "  ",
    " ",
    "WHE",
    "N",
    " ",
    "M",
    "O",
    "D(",
    "n",
    "u",
    "m, 3)",
    " =",
    " ",
    "0 T",
    "H",
    "E",
    "N 'Fizz",
    "'\n   ",
    "  ",
    "  ",
    " ",
    "WHE",
    "N",
    " ",
    "M",
    "O",
    "D(",
    "n",
    "u",
    "m, 5)",
    " =",
    " ",
    "0 T",
    "H",
    "E",
    "N 'Buzz",
    "'\n   ",
    "   ",
    "  ",
    "E",
    "LSE",
    " CA",
    "ST(nu",
    "m",
    " ",
    "AS ",
    "CHAR",
    ")\n ",
    "   END ",
    "A",
    "S re",
    "su",
    "lt\n",
    "F",
    "RO",
    "M ",
    "f",
    "i",
    "zz_",
    "buzz;\n```"
  ],
  "started_at": "2024-04-24T03:58:30Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.us-chicago-1.o.replicate.net/v1/streams/axg26c7crqn2w723buhwjxxiswx7conlfwp64wgim4sxszp7rxeq",
    "get": "https://api.replicate.com/v1/predictions/3knp1rt7ksrhw0cf1s0vgxh2s0",
    "cancel": "https://api.replicate.com/v1/predictions/3knp1rt7ksrhw0cf1s0vgxh2s0/cancel"
  },
  "version": "hidden"
}

Generated in

3.0 seconds

Tweak it Share Report

Run time and cost

This model costs approximately $0.00025 to run on Replicate, or 4000 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on CPU hardware. Predictions typically complete within 3 seconds.

Readme

Demo App

Want to chat with Arctic? Try the Streamlit demo app. Streamlit Arctic demo app

Model Details

Arctic is a dense-MoE Hybrid transformer architecture pre-trained from scratch by the Snowflake AI Research Team. We are releasing model checkpoints for both the base and instruct-tuned versions of Arctic under an Apache-2.0 license. This means you can use them freely in your own research, prototypes, and products. Please see our blog Snowflake Arctic: The Best LLM for Enterprise AI — Efficiently Intelligent, Truly Open for more information on Arctic and links to other relevant resources such as our series of cookbooks covering topics around training your own custom MoE models, how to produce high-quality training data, and much more.

For the latest details about Snowflake Arctic including tutorials, etc. please refer to our github repo: https://github.com/Snowflake-Labs/snowflake-arctic

Model developers Snowflake AI Research Team

License Apache-2.0

Input Models input text only.

Output Models generate text and code only.

Model Release Date April, 24th 2024.

Model Architecture

Arctic combines a 10B dense transformer model with a residual 128x3.66B MoE MLP resulting in 480B total and 17B active parameters chosen using a top-2 gating. For more details about Arctic’s model Architecture, training process, data, etc. see our series of cookbooks.