You can now deploy, run, and fine-tune large language models on Replicate.

We’ve got official versions of FLAN-T5, GPT-J, and LLaMA, and you can also push any other custom model. We’re also releasing a preview of fine-tuning language models.

Language models can be run with just a couple of lines of code, like any other model on Replicate:

import replicate

output = replicate.run(
    "replicate/flan-t5-xl:3ae0799123a1fe11f8c89fd99632f843fc5f7a761630160521c4253149754523",
    input={"prompt": "Answer the following yes/no question by reasoning step by step. Can a dog drive a car?"}
)

for s in output:
    print(s, end="", flush=True)

You can run them from Python, Node.js, or with an HTTP API, without having to set up servers or GPUs.

Try them out:

• Google’s Flan-T5 XL
• EleutherAI’s GPT-J-6B
• Meta AI’s LLaMA

Push a custom model

You can push any language model to Replicate. They can either be public if you want to share an open-source model with the world, or private if you want to deploy a custom model at scale for your product.

We’ve written a guide that shows you how to push a Transformers model from Hugging Face to Replicate.

Preview: Fine-tune a model

Given a good prompt, these models can do many tasks, but they are much more powerful when fine-tuned on your own data. You can create a support bot that knows about your product, writes emails in your style, extracts structured data from text, and all sorts of things.

We’re releasing a preview of training models on Replicate in the cloud. You can fine-tune Flan-T5, GPT-J, and LLaMA. We’ll be adding support for other models and your own custom training scripts soon.

replicate.trainings.create(
   version="...",
   input={"train_data": "..."},
   destination="bfirsh/hackerbot",
)

We’re starting by just giving a few people access and we will be opening it up over the coming weeks. If you want to use it, email us at team@replicate.com with more about yourself and what you want to use it for.

Read our guide to fine-tuning a model to learn more.

What’s next

Open-source language models have been improving at an extraordinary pace. We’ll be releasing more guides and example apps over the next few weeks to show you what’s possible. Follow us on Twitter to follow along.

We can’t wait to see what you build.