You're looking at a specific version of this model. Jump to the model overview.
moinnadeem /mlc_llama_70b:cb441f71
Input schema
The fields you can use to run this model with an API. If you don’t give a value for a field its default value will be used.
| Field | Type | Default value | Description |
|---|---|---|---|
| prompt |
string
|
Can you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings.
|
Prompt to send to Llama v2.
|
| system_prompt |
string
|
"[INST] <<SYS>>
You are a helpful, respectful and honest assistant. "
"Always answer as helpfully as possible, while being safe. "
"Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, "
"or illegal content. "
"Please ensure that your responses are socially unbiased and positive in nature.
"
"If a question does not make any sense, or is not factually coherent, explain why instead "
"of answering something not correct. "
"If you don't know the answer to a question, please don't share false "
"information.
<</SYS>>
"
|
System prompt to send to Llama v2. This is prepended to the prompt and helps guide system behavior.
|
| max_new_tokens |
integer
|
500
Min: 1 |
Maximum number of tokens to generate. A word is generally 2-3 tokens
|
| temperature |
number
|
0.95
Min: 0.01 Max: 5 |
Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.
|
| top_p |
number
|
0.95
Max: 1 |
When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens
|
| repetition_penalty |
number
|
1.15
Min: 0.01 Max: 5 |
Penalty for repeated words in generated text; 1 is no penalty, values greater than 1 discourage repetition, less than 1 encourage it.
|
| stop_str |
string
|
A sequence to stop generation at. For example, '<end>' will stop generation at the first instance of '<end>'.
|
Output schema
The shape of the response you’ll get when you run this model with an API.
Schema
{'items': {'type': 'string'},
'title': 'Output',
'type': 'array',
'x-cog-array-display': 'concatenate',
'x-cog-array-type': 'iterator'}