nwhitehead
/
llama2-70b-oasst-sft-v10
This model is the Open-Assistant fine-tuning of Meta's Llama2 70B LLM.
- Public
- 2 runs
- License
Run nwhitehead/llama2-70b-oasst-sft-v10 with an API
Use one of our client libraries to get started quickly. Clicking on a library will take you to the Playground tab where you can tweak different inputs, see the results, and copy the corresponding code to use in your own project.
Input schema
The fields you can use to run this model with an API. If you don't give a value for a field its default value will be used.
| Field | Type | Default value | Description |
|---|---|---|---|
| prompt |
string
|
USER: Hello, who are you?
ASSISTANT:
|
Text prompt for the model
|
| temperature |
number
|
0.5
Min: 0.01 Max: 2 |
Temperature of the output, it's best to keep it below 1
|
| top_p |
number
|
1
Min: 0.01 Max: 1 |
Top cumulative probability to filter candidates
|
| top_k |
integer
|
20
Min: 1 Max: 100 |
Number of top candidates to keep
|
| repetition_penalty |
number
|
1
Min: 1 Max: 1.5 |
Penalty for repeated tokens in the model's output
|
| max_tokens |
integer
|
50
Min: 1 Max: 4096 |
Maximum tokens to generate
|
| min_tokens |
integer
|
1
Max: 4096 |
Minimum tokens to generate
|
| seed |
integer
|
-1
Min: -2147483648 Max: 2147483647 |
Seed for reproducibility, -1 for random seed
|
{
"type": "object",
"title": "Input",
"properties": {
"seed": {
"type": "integer",
"title": "Seed",
"default": -1,
"maximum": 2147483647,
"minimum": -2147483648,
"x-order": 7,
"description": "Seed for reproducibility, -1 for random seed"
},
"top_k": {
"type": "integer",
"title": "Top K",
"default": 20,
"maximum": 100,
"minimum": 1,
"x-order": 3,
"description": "Number of top candidates to keep"
},
"top_p": {
"type": "number",
"title": "Top P",
"default": 1,
"maximum": 1,
"minimum": 0.01,
"x-order": 2,
"description": "Top cumulative probability to filter candidates"
},
"prompt": {
"type": "string",
"title": "Prompt",
"default": "USER: Hello, who are you?\nASSISTANT:",
"x-order": 0,
"description": "Text prompt for the model"
},
"max_tokens": {
"type": "integer",
"title": "Max Tokens",
"default": 50,
"maximum": 4096,
"minimum": 1,
"x-order": 5,
"description": "Maximum tokens to generate"
},
"min_tokens": {
"type": "integer",
"title": "Min Tokens",
"default": 1,
"maximum": 4096,
"minimum": 0,
"x-order": 6,
"description": "Minimum tokens to generate"
},
"temperature": {
"type": "number",
"title": "Temperature",
"default": 0.5,
"maximum": 2,
"minimum": 0.01,
"x-order": 1,
"description": "Temperature of the output, it's best to keep it below 1"
},
"repetition_penalty": {
"type": "number",
"title": "Repetition Penalty",
"default": 1,
"maximum": 1.5,
"minimum": 1,
"x-order": 4,
"description": "Penalty for repeated tokens in the model's output"
}
}
}
Output schema
The shape of the response you’ll get when you run this model with an API.
{
"type": "array",
"items": {
"type": "string"
},
"title": "Output",
"x-cog-array-type": "iterator",
"x-cog-array-display": "concatenate"
}