LLaMA is a new open-source language model from Meta Research that performs as well as closed-source models. Similar to Stable Diffusion, there’s been a ton of experimentation and innovation since the model was publicly released. As Simon Willison articulated, LLaMA is easy to run on your own hardware, large enough to be useful, and open-source enough to be tinkered with.
LLaMA is powerful, but it was not built for answering questions. It functions more like a fancy version of autocomplete than a conversational bot. This is where Stanford’s Alpaca comes in. Alpaca is a fine-tuned version of LLaMA that can respond to instructions like ChatGPT. And, like LLaMA, it’s open-source.
The problem is, the weights for Alpaca have not been released, so you can't tinker with it. We do have all the component parts we need to replicate it though: the LLaMA weights, the training data, and the training script.
In this post we'll show you how to train Alpaca so you can tinker with it on your own machine.
Note: LLaMA and anything built on LLaMA is for research purposes only. You can't build anything commercial with it.
We’ve created a fork of the Alpaca repository that adds a Cog file that’ll set up all the dependencies for you.
Log into your GPU instance via SSH. Clone the repository by running:
The LLaMA weights are currently only available for research use. To apply for access, fill out this Meta Research form.
Put your downloaded weights in a folder called unconverted-weights
. The folder hierarchy should look something like this:
Convert the weights from a PyTorch checkpoint to a transformers-compatible format using this command:
You final directory structure should look like this:
Kick off the training:
This will take about an hour and a half on four A100s, so you might want to go and do some programming while your model is programming itself.
When that’s finished, you can run Alpaca:
Here are some ideas for what you could do next:
Just remember that you can only use Alpaca for non-commercial research. Eventually, we expect models like this will be released with more permissive licenses that will allow them to be used for all sorts of things — chat bots, coding assistants, and so on.
Open-source language models are just getting started, and we can’t wait to see what you build.
We’re going to be posting more guides to hacking on open-source language models. Follow us on Twitter to follow along.