List of strings that stop the generation when they are generated. The returned output will not contain the stop strings.

Integer that controls the number of top tokens to consider. Set to -1 to consider all tokens.

Default

-1

Float that controls the cumulative probability of the top tokens to consider. Must be in (0, 1]. Set to 1 to consider all tokens.

Default

0.95

Minimum

0.01

Maximum

1

Text prompt to send to the model.

Maximum number of tokens to generate per output sequence.

Default

128

Float that controls the randomness of the sampling. Lower values make the model more deterministic, while higher values make the model more random. Zero means greedy sampling.

Default

0.8

Minimum

0.01

Maximum

5

Float that penalizes new tokens based on whether they appear in the generated text so far. Values > 0 encourage the model to use new tokens, while values < 0 encourage the model to repeat tokens.

Minimum

-5

Maximum

5

Float that penalizes new tokens based on their frequency in the generated text so far. Values > 0 encourage the model to use new tokens, while values < 0 encourage the model to repeat tokens.

Minimum

-5

Maximum

5