adirik / mamba-790m

Base version of Mamba 790M, a 790 million parameter state space language model

Cold

Public
47 runs
L40S
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

Text prompt to send to the model.

max_length

integer

(minimum: 1, maximum: 5000)

Maximum number of tokens to generate. A word is generally 2-3 tokens.

Default: 100

temperature

number

(minimum: 0.1, maximum: 5)

Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.

Default: 1

top_p

number

(minimum: 0.01, maximum: 1)

When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens.

Default: 1

top_k

integer

When decoding text, samples from the top k most likely tokens; lower to ignore less likely tokens.

Default: 1

repetition_penalty

number

(minimum: 0.01, maximum: 10)

Penalty for repeated words in generated text; 1 is no penalty, values greater than 1 discourage repetition, less than 1 encourage it.

Default: 1.2

seed

integer

The seed for the random number generator

Run this model in Node.js with one line of code:

npx create-replicate --model=adirik/mamba-790m

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run adirik/mamba-790m using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "adirik/mamba-790m:77782448285ebc03a24c2e90cc12b6cebbdaf325c071eaee2e315320308d9748",
  {
    input: {
      top_k: 1,
      top_p: 1,
      prompt: "How are you doing today?",
      max_length: 100,
      temperature: 1,
      repetition_penalty: 1.2
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run adirik/mamba-790m using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "adirik/mamba-790m:77782448285ebc03a24c2e90cc12b6cebbdaf325c071eaee2e315320308d9748",
    input={
        "top_k": 1,
        "top_p": 1,
        "prompt": "How are you doing today?",
        "max_length": 100,
        "temperature": 1,
        "repetition_penalty": 1.2
    }
)

# The adirik/mamba-790m model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/adirik/mamba-790m/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run adirik/mamba-790m using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "77782448285ebc03a24c2e90cc12b6cebbdaf325c071eaee2e315320308d9748",
    "input": {
      "top_k": 1,
      "top_p": 1,
      "prompt": "How are you doing today?",
      "max_length": 100,
      "temperature": 1,
      "repetition_penalty": 1.2
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/adirik/mamba-790m@sha256:77782448285ebc03a24c2e90cc12b6cebbdaf325c071eaee2e315320308d9748 \
  -i 'top_k=1' \
  -i 'top_p=1' \
  -i 'prompt="How are you doing today?"' \
  -i 'max_length=100' \
  -i 'temperature=1' \
  -i 'repetition_penalty=1.2'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/adirik/mamba-790m@sha256:77782448285ebc03a24c2e90cc12b6cebbdaf325c071eaee2e315320308d9748
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "top_k": 1,
      "top_p": 1,
      "prompt": "How are you doing today?",
      "max_length": 100,
      "temperature": 1,
      "repetition_penalty": 1.2
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

I'm good. I just got back from the doctor's office, and he said that my blood pressure is normal for me at this age... so it was a little bit of an adjustment to get used too but overall everything has been great! So thank God!! :)<|endoftext|>Q: tag in HTML5 not working properly with IE9 <div class="container">  // works fine in all

{
  "completed_at": "2024-02-05T18:27:43.234473Z",
  "created_at": "2024-02-05T18:26:10.992362Z",
  "data_removed": false,
  "error": null,
  "id": "htznmk3b65r5t7yxmwdms6gb4a",
  "input": {
    "top_k": 1,
    "top_p": 1,
    "prompt": "How are you doing today?",
    "max_length": 100,
    "temperature": 1,
    "repetition_penalty": 1.2
  },
  "logs": null,
  "metrics": {
    "predict_time": 8.321735,
    "total_time": 92.242111
  },
  "output": [
    "\n",
    "",
    "",
    "I'm ",
    "",
    "good. ",
    "I ",
    "just ",
    "got ",
    "back ",
    "from ",
    "the ",
    "",
    "doctor's ",
    "",
    "office, ",
    "and ",
    "he ",
    "said ",
    "that ",
    "my ",
    "blood ",
    "pressure ",
    "is ",
    "normal ",
    "for ",
    "me ",
    "at ",
    "this ",
    "",
    "age... ",
    "so ",
    "it ",
    "was ",
    "a ",
    "little ",
    "bit ",
    "of ",
    "an ",
    "adjustment ",
    "to ",
    "get ",
    "used ",
    "too ",
    "but ",
    "overall ",
    "everything ",
    "has ",
    "been ",
    "",
    "great! ",
    "So ",
    "thank ",
    "",
    "God!! ",
    "",
    "",
    "",
    ":)<|endoftext|>Q:\n\n",
    " ",
    "tag ",
    "in ",
    "",
    "HTML5 ",
    "not ",
    "working ",
    "properly ",
    "with ",
    "",
    "IE9\n",
    "    ",
    "",
    "",
    "<div ",
    "",
    "",
    "",
    "class=\"container\"> \n",
    "        ",
    "",
    "<!-- ",
    "",
    "",
    "",
    "",
    "<h1>Hello ",
    "",
    "",
    "",
    "",
    "World</p><!-- ",
    "",
    "</br ",
    "/> ",
    "-->  ",
    "",
    "// ",
    "works ",
    "fine ",
    "in ",
    "all"
  ],
  "started_at": "2024-02-05T18:27:34.912738Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.us.c.replicate.net/v1/streams/iwo6ktek23hzvlfm72w6fwcwvpfpoz2qaqipsdu6ougfsotpwicq",
    "get": "https://api.replicate.com/v1/predictions/htznmk3b65r5t7yxmwdms6gb4a",
    "cancel": "https://api.replicate.com/v1/predictions/htznmk3b65r5t7yxmwdms6gb4a/cancel"
  },
  "version": "77782448285ebc03a24c2e90cc12b6cebbdaf325c071eaee2e315320308d9748"
}

Generated in

8.3 seconds

Tweak it Report

Run time and cost

This model runs on Nvidia L40S GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

Mamba

Mamba is a large language model with state space model architecture showing promising performance on information-dense data such as language modeling. See the original repo and paper for details.

Basic Usage

The API input arguments are as follows:

prompt: The text prompt for Mamba.
max_length: Maximum number of tokens to generate. A word is generally 2-3 tokens.
temperature: Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.
top_p: Samples from the top p percentage of most likely tokens during text decoding, lower to ignore less likely tokens.
top_k: Samples from the top k most likely tokens during text decoding, lower to ignore less likely tokens.
repetition_penalty: Penalty for repeated words in generated text; 1 is no penalty, values greater than 1 discourage repetition, less than 1 encourage it.
seed: The seed parameter for deterministic text generation. A specific seed can be used to reproduce results or left blank for random generation.

References

@article{mamba,
  title={Mamba: Linear-Time Sequence Modeling with Selective State Spaces},
  author={Gu, Albert and Dao, Tri},
  journal={arXiv preprint arXiv:2312.00752},
  year={2023}
}