Readme
Model used: Airoboros L2 70B 2.1 - GPTQ
LoRA used: Airoboros 2.1 Jannie 70B QLoRA
Inference using: ExLlama
Cog GitHub project: cog-exllama
uwulewd
/
airoboros-llama-2-70b
Inference Airoboros L2 70B 2.1 - GPTQ using ExLlama.
Run time and cost
This model costs approximately $0.037 to run on Replicate, or 27 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.
This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 27 seconds.