stability-ai/stablelm-tuned-alpha-7b – Run with an API on Replicate

stability-ai / stablelm-tuned-alpha-7b

7 billion parameter version of Stability AI's language model

Cold

Public
140.5K runs
A100 (80GB)
GitHub
License

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

Input Prompt.

Default: "What's your mood today?"

max_new_tokens

integer

(minimum: 1)

Maximum number of tokens to generate. A word is generally 2-3 tokens

Default: 100

top_p

number

(minimum: 0.01, maximum: 1)

Valid if you choose top_p decoding. When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens

Default: 1

temperature

number

(minimum: 0.01, maximum: 5)

Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.

Default: 0.75

repetition_penalty

number

(minimum: 0.01, maximum: 5)

Penalty for repeated words in generated text; 1 is no penalty, values greater than 1 discourage repetition, less than 1 encourage it.

Default: 1.2

Run this model in Node.js with one line of code:

npx create-replicate --model=stability-ai/stablelm-tuned-alpha-7b

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run stability-ai/stablelm-tuned-alpha-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "stability-ai/stablelm-tuned-alpha-7b:943c4afb4d0273cf1cf17c1070e182c903a9fe6b372df36b5447cf45935c42f2",
  {
    input: {
      top_p: 1,
      prompt: "How do you make ratatouille?",
      temperature: 0.75,
      max_new_tokens: 100,
      repetition_penalty: 1.2
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run stability-ai/stablelm-tuned-alpha-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "stability-ai/stablelm-tuned-alpha-7b:943c4afb4d0273cf1cf17c1070e182c903a9fe6b372df36b5447cf45935c42f2",
    input={
        "top_p": 1,
        "prompt": "How do you make ratatouille?",
        "temperature": 0.75,
        "max_new_tokens": 100,
        "repetition_penalty": 1.2
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run stability-ai/stablelm-tuned-alpha-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "943c4afb4d0273cf1cf17c1070e182c903a9fe6b372df36b5447cf45935c42f2",
    "input": {
      "top_p": 1,
      "prompt": "How do you make ratatouille?",
      "temperature": 0.75,
      "max_new_tokens": 100,
      "repetition_penalty": 1.2
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/stability-ai/stablelm-tuned-alpha-7b@sha256:943c4afb4d0273cf1cf17c1070e182c903a9fe6b372df36b5447cf45935c42f2 \
  -i 'top_p=1' \
  -i 'prompt="How do you make ratatouille?"' \
  -i 'temperature=0.75' \
  -i 'max_new_tokens=100' \
  -i 'repetition_penalty=1.2'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/stability-ai/stablelm-tuned-alpha-7b@sha256:943c4afb4d0273cf1cf17c1070e182c903a9fe6b372df36b5447cf45935c42f2
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "top_p": 1,
      "prompt": "How do you make ratatouille?",
      "temperature": 0.75,
      "max_new_tokens": 100,
      "repetition_penalty": 1.2
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

First start with some lovely vegetables like eggplant or zucchini - whatever you fancy! Then cook them up nice and tender. Next put on some herbs if you want something special, such as thyme or rosemary - maybe a sprinkling of smoked paprika for extra flavor. Once everything's cooked through stir it all together well so each spoonful contains delicious little bits of veggies, herbs and flavourings. And serve over plenty of good quality white rice – either keep it simple with basmati or let your guests choose their own pasta from amongst several available options... The end-result should look delectable too!

Generated in

3.6 seconds

Tweak it Report

This example was created by a different version, stability-ai/stablelm-tuned-alpha-7b:4a9a32b4.

Run time and cost

This model costs approximately $0.0026 to run on Replicate, or 384 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 2 seconds.

Readme

StableLM-Tuned-Alpha-7B is a 7B parameter decoder-only language model built on top of the StableLM-Base-Alpha models and further fine-tuned on various chat and instruction-following datasets.

The StableLM-Alpha models are trained on the new dataset that builds on The Pile, which contains 1.5 trillion tokens, roughly 3x the size of The Pile. These models will be trained on up to 1.5 trillion tokens. The context length for these models is 4096 tokens.

An upcoming technical report will document the model specifications and the training settings.

As a proof-of-concept, we also fine-tuned the model with Stanford Alpaca’s procedure using a combination of five recent datasets for conversational agents: Stanford’s Alpaca, Nomic-AI’s gpt4all, RyokoAI’s ShareGPT52K datasets, Databricks labs’ Dolly, and Anthropic’s HH. We will be releasing these models as StableLM-Tuned-Alpha.

Acknowledgements

StableLM-Tuned-Alpha would not have been possible without the helpful hand of Dakota Mahan @dmayhem93.

Licenses

Base model checkpoints (StableLM-Base-Alpha) are licensed under the Creative Commons license (CC BY-SA-4.0). Under the license, you must give credit to Stability AI, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the Stability AI endorses you or your use.
Fine-tuned checkpoints (StableLM-Tuned-Alpha) are licensed under the Non-Commercial Creative Commons license (CC BY-NC-SA-4.0), in-line with the original non-commercial license specified by Stanford Alpaca.
All code in this repository is licensed under the Apache License 2.0 license.