tomasmcm/metamath-mistral-7b – Run with an API on Replicate

tomasmcm / metamath-mistral-7b

Source: meta-math/MetaMath-Mistral-7B ✦ Quant: TheBloke/MetaMath-Mistral-7B-AWQ ✦ Bootstrap Your Own Mathematical Questions for Large Language Models

Cold

Public
391 runs
L40S
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

Below is an instruction that describes a task.
Write a response that appropriately completes the request.

### Instruction:
Gerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left?

### Response: Let's think step by step.Below is an instruction that describes a task.
Write a response that appropriately completes the request.

### Instruction:
Gerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left?

### Response: Let's think step by step.

Text prompt to send to the model.

max_new_tokens

integer

The maximum number of tokens the model should generate as output.

Default: 128

temperature

number

(minimum: 0.01, maximum: 5)

The value used to modulate the next token probabilities.

Default: 0.8

top_p

number

(minimum: 0.01, maximum: 1)

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).

Default: 0.95

top_k

integer

The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).

Default: 50

presence_penalty

number

(minimum: 0.01, maximum: 5)

Presence penalty

Default: 1

Run this model in Node.js with one line of code:

npx create-replicate --model=tomasmcm/metamath-mistral-7b

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run tomasmcm/metamath-mistral-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "tomasmcm/metamath-mistral-7b:2873b99ace018acc5ea83756c5773bf89eeb7db35eb882c29a3eec8bba99133e",
  {
    input: {
      top_k: 50,
      top_p: 0.95,
      prompt: "Below is an instruction that describes a task.\nWrite a response that appropriately completes the request.\n\n### Instruction:\nGerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left?\n\n### Response: Let's think step by step.",
      temperature: 0.8,
      max_new_tokens: 128,
      presence_penalty: 1
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run tomasmcm/metamath-mistral-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "tomasmcm/metamath-mistral-7b:2873b99ace018acc5ea83756c5773bf89eeb7db35eb882c29a3eec8bba99133e",
    input={
        "top_k": 50,
        "top_p": 0.95,
        "prompt": "Below is an instruction that describes a task.\nWrite a response that appropriately completes the request.\n\n### Instruction:\nGerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left?\n\n### Response: Let's think step by step.",
        "temperature": 0.8,
        "max_new_tokens": 128,
        "presence_penalty": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run tomasmcm/metamath-mistral-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "2873b99ace018acc5ea83756c5773bf89eeb7db35eb882c29a3eec8bba99133e",
    "input": {
      "top_k": 50,
      "top_p": 0.95,
      "prompt": "Below is an instruction that describes a task.\\nWrite a response that appropriately completes the request.\\n\\n### Instruction:\\nGerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left?\\n\\n### Response: Let\'s think step by step.",
      "temperature": 0.8,
      "max_new_tokens": 128,
      "presence_penalty": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/tomasmcm/metamath-mistral-7b@sha256:2873b99ace018acc5ea83756c5773bf89eeb7db35eb882c29a3eec8bba99133e \
  -i 'top_k=50' \
  -i 'top_p=0.95' \
  -i $'prompt="Below is an instruction that describes a task.\\nWrite a response that appropriately completes the request.\\n\\n### Instruction:\\nGerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left?\\n\\n### Response: Let\'s think step by step."' \
  -i 'temperature=0.8' \
  -i 'max_new_tokens=128' \
  -i 'presence_penalty=1'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/tomasmcm/metamath-mistral-7b@sha256:2873b99ace018acc5ea83756c5773bf89eeb7db35eb882c29a3eec8bba99133e
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "top_k": 50,
      "top_p": 0.95,
      "prompt": "Below is an instruction that describes a task.\\nWrite a response that appropriately completes the request.\\n\\n### Instruction:\\nGerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left?\\n\\n### Response: Let\'s think step by step.",
      "temperature": 0.8,
      "max_new_tokens": 128,
      "presence_penalty": 1
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

Gerald earns $30 every day, so in a week, he earns 7 * $30 = $210. If he spent a total of $100, then he has $210 - $100 = $110 left. #### 110 The answer is: 110

Generated in

1.0 second

Tweak it Report

Run time and cost

This model costs approximately $0.0016 to run on Replicate, or 625 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 2 seconds.

Readme

see our paper in https://arxiv.org/abs/2309.12284

View the project page: https://meta-math.github.io/

Model Details

MetaMath-Mistral-7B is fully fine-tuned on the MetaMathQA datasets and based on the powerful Mistral-7B model. It is glad to see using MetaMathQA datasets and change the base model from llama-2-7B to Mistral-7b can boost the GSM8K performance from 66.5 to 77.7.

To fine-tune Mistral-7B, I would suggest using a smaller learning rate (usually 1/5 to 1/10 of the lr for LlaMa-2-7B) and staying other training args unchanged. More training details and scripts can be seen at https://github.com/meta-math/MetaMath

Installation

pip install transformers==4.35.0
pip instal torch==2.0.1
pip instal sentencepiece==0.1.99
pip instal tokenizers==0.13.3
pip instal accelerate==0.21.0
pip instal bitsandbytes==0.40.0
pip instal vllm
pip instal fraction
pip install protobuf

Model Usage

prompting template:

’‘’

“Below is an instruction that describes a task. “ “Write a response that appropriately completes the request.\n\n” “### Instruction:\n{instruction}\n\n### Response: Let’s think step by step.”

’‘’

where you need to use your query question to replace the {instruction}

There is another interesting repo about Arithmo-Mistral-7B in https://huggingface.co/akjindal53244/Arithmo-Mistral-7B, where they combine our MetaMathQA dataset and MathInstruct datasets to train a powerful model. Thanks agian for their contributions. We would also try to train the combination of MetaMathQA and MathInstruct datasets, and also open all the results and training details.

Experiments

Model	GSM8k Pass@1	MATH Pass@1
MPT-7B	6.8	3.0
Falcon-7B	6.8	2.3
LLaMA-1-7B	11.0	2.9
LLaMA-2-7B	14.6	2.5
MPT-30B	15.2	3.1
LLaMA-1-13B	17.8	3.9
GPT-Neo-2.7B	19.5	–
Falcon-40B	19.6	2.5
Baichuan-chat-13B	23.9	–
Vicuna-v1.3-13B	27.6	–
LLaMA-2-13B	28.7	3.9
InternLM-7B	31.2	–
ChatGLM-2-6B	32.4	–
GPT-J-6B	34.9	–
LLaMA-1-33B	35.6	3.9
LLaMA-2-34B	42.2	6.24
RFT-7B	50.3	–
LLaMA-1-65B	50.9	10.6
Qwen-7B	51.6	–
WizardMath-7B	54.9	10.7
LLaMA-2-70B	56.8	13.5
WizardMath-13B	63.9	14.0
MAmmoTH-7B (COT)	50.5	10.4
MAmmoTH-7B (POT+COT)	53.6	31.5
Arithmo-Mistral-7B	74.7	25.3
MetaMath-7B	66.5	19.8
MetaMath-13B	72.3	22.4
🔥 MetaMath-Mistral-7B	77.7	28.2

Citation

@article{yu2023metamath,
  title={MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models},
  author={Yu, Longhui and Jiang, Weisen and Shi, Han and Yu, Jincheng and Liu, Zhengying and Zhang, Yu and Kwok, James T and Li, Zhenguo and Weller, Adrian and Liu, Weiyang},
  journal={arXiv preprint arXiv:2309.12284},
  year={2023}
}

@article{jiang2023mistral,
  title={Mistral 7B},
  author={Jiang, Albert Q and Sablayrolles, Alexandre and Mensch, Arthur and Bamford, Chris and Chaplot, Devendra Singh and Casas, Diego de las and Bressand, Florian and Lengyel, Gianna and Lample, Guillaume and Saulnier, Lucile and others},
  journal={arXiv preprint arXiv:2310.06825},
  year={2023}
}