You're looking at a specific version of this model. Jump to the model overview.
01-ai /yi-34b-chat:d7337888
Input
Run this model in Node.js with one line of code:
npm install replicate
REPLICATE_API_TOKEN
environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>
Find your API token in your account settings.
import Replicate from "replicate";
const replicate = new Replicate({
auth: process.env.REPLICATE_API_TOKEN,
});
Run 01-ai/yi-34b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run(
"01-ai/yi-34b-chat:d7337888fd666656582ffe234e97c72ea754914b48bb75a97f1ae657ddb060aa",
{
input: {
top_k: 50,
top_p: 0.8,
temperature: 0.3,
max_new_tokens: 1024,
prompt_template: "<|im_start|>system\nYou are a helpful assistant<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n",
presence_penalty: 0,
frequency_penalty: 0
}
}
);
console.log(output);
To learn more, take a look at the guide on getting started with Node.js.
pip install replicate
REPLICATE_API_TOKEN
environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>
Find your API token in your account settings.
import replicate
Run 01-ai/yi-34b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run(
"01-ai/yi-34b-chat:d7337888fd666656582ffe234e97c72ea754914b48bb75a97f1ae657ddb060aa",
input={
"top_k": 50,
"top_p": 0.8,
"temperature": 0.3,
"max_new_tokens": 1024,
"prompt_template": "<|im_start|>system\nYou are a helpful assistant<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n",
"presence_penalty": 0,
"frequency_penalty": 0
}
)
# The 01-ai/yi-34b-chat model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
# https://replicate.com/01-ai/yi-34b-chat/api#output-schema
print(item, end="")
To learn more, take a look at the guide on getting started with Python.
REPLICATE_API_TOKEN
environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>
Find your API token in your account settings.
Run 01-ai/yi-34b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
-H "Content-Type: application/json" \
-H "Prefer: wait" \
-d $'{
"version": "d7337888fd666656582ffe234e97c72ea754914b48bb75a97f1ae657ddb060aa",
"input": {
"top_k": 50,
"top_p": 0.8,
"temperature": 0.3,
"max_new_tokens": 1024,
"prompt_template": "<|im_start|>system\\nYou are a helpful assistant<|im_end|>\\n<|im_start|>user\\n{prompt}<|im_end|>\\n<|im_start|>assistant\\n",
"presence_penalty": 0,
"frequency_penalty": 0
}
}' \
https://api.replicate.com/v1/predictions
To learn more, take a look at Replicate’s HTTP API reference docs.
brew install cog
If you don’t have Homebrew, there are other installation options available.
Run this to download the model and run it in your local environment:
cog predict r8.im/01-ai/yi-34b-chat@sha256:d7337888fd666656582ffe234e97c72ea754914b48bb75a97f1ae657ddb060aa \
-i 'top_k=50' \
-i 'top_p=0.8' \
-i 'temperature=0.3' \
-i 'max_new_tokens=1024' \
-i $'prompt_template="<|im_start|>system\\nYou are a helpful assistant<|im_end|>\\n<|im_start|>user\\n{prompt}<|im_end|>\\n<|im_start|>assistant\\n"' \
-i 'presence_penalty=0' \
-i 'frequency_penalty=0'
To learn more, take a look at the Cog documentation.
Run this to download the model and run it in your local environment:
docker run -d -p 5000:5000 --gpus=all r8.im/01-ai/yi-34b-chat@sha256:d7337888fd666656582ffe234e97c72ea754914b48bb75a97f1ae657ddb060aa
curl -s -X POST \ -H "Content-Type: application/json" \ -d $'{ "input": { "top_k": 50, "top_p": 0.8, "temperature": 0.3, "max_new_tokens": 1024, "prompt_template": "<|im_start|>system\\nYou are a helpful assistant<|im_end|>\\n<|im_start|>user\\n{prompt}<|im_end|>\\n<|im_start|>assistant\\n", "presence_penalty": 0, "frequency_penalty": 0 } }' \ http://localhost:5000/predictions
To learn more, take a look at the Cog documentation.
Add a payment method to run this model.
By signing in, you agree to our
terms of service and privacy policy
Output
No output yet! Press "Submit" to start a prediction.