okaris / progen2

ProGen: Language Modeling for Protein Engineering

Cold

Public
233 runs
A100 (80GB)
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=okaris/progen2

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run okaris/progen2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "okaris/progen2:bc5e7b50d9770241f05cae293ccf223e4dfd56ee45e9bb93d7b49b8d29f6c761",
  {
    input: {
      p: 0.9,
      t: 0.8,
      fp16: true,
      model: "progen2-small",
      device: "cuda:0",
      sanity: true,
      context: "1",
      rng_seed: 42,
      max_length: 1024,
      num_samples: 2,
      rng_deterministic: true
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run okaris/progen2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "okaris/progen2:bc5e7b50d9770241f05cae293ccf223e4dfd56ee45e9bb93d7b49b8d29f6c761",
    input={
        "p": 0.9,
        "t": 0.8,
        "fp16": True,
        "model": "progen2-small",
        "device": "cuda:0",
        "sanity": True,
        "context": "1",
        "rng_seed": 42,
        "max_length": 1024,
        "num_samples": 2,
        "rng_deterministic": True
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run okaris/progen2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "bc5e7b50d9770241f05cae293ccf223e4dfd56ee45e9bb93d7b49b8d29f6c761",
    "input": {
      "p": 0.9,
      "t": 0.8,
      "fp16": true,
      "model": "progen2-small",
      "device": "cuda:0",
      "sanity": true,
      "context": "1",
      "rng_seed": 42,
      "max_length": 1024,
      "num_samples": 2,
      "rng_deterministic": true
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.

Run time and cost

This model costs approximately $0.050 to run on Replicate, or 20 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 36 seconds. The predict time for this model varies significantly based on the inputs.

Readme

ProGen2

Official release of the ProGen2 models (151M, 764M, 2.7B, 6.4B) for Protein Engineering (see paper).

Models

Model	Size	Checkpoint
progen2-small	`151M`	https://storage.googleapis.com/sfr-progen-research/checkpoints/progen2-small.tar.gz
progen2-medium	`764M`	https://storage.googleapis.com/sfr-progen-research/checkpoints/progen2-medium.tar.gz
progen2-oas	`764M`	https://storage.googleapis.com/sfr-progen-research/checkpoints/progen2-oas.tar.gz
progen2-base	`764M`	https://storage.googleapis.com/sfr-progen-research/checkpoints/progen2-base.tar.gz
progen2-large	`2.7B`	https://storage.googleapis.com/sfr-progen-research/checkpoints/progen2-large.tar.gz
progen2-BFD90	`2.7B`	https://storage.googleapis.com/sfr-progen-research/checkpoints/progen2-BFD90.tar.gz
progen2-xlarge	`6.4B`	https://storage.googleapis.com/sfr-progen-research/checkpoints/progen2-xlarge.tar.gz

Setup

# code
git clone https://github.com/salesforce/progen
cd progen/progen2

# checkpoint
model=progen2-large
wget -P checkpoints/${model} https://storage.googleapis.com/sfr-progen-research/checkpoints/${model}.tar.gz
tar -xvf checkpoints/${model}/${model}.tar.gz -C checkpoints/${model}/

# venv
python3.8 -m venv .venv
source .venv/bin/activate
pip3 install --upgrade pip setuptools
pip3 install -r requirements.txt

# sample
python3 sample.py --model ${model} --t 0.8 --p 0.9 --max-length 1024 --num-samples 2 --context "1"

# log-likelihood (GenBank: TMF32756.1)
python3 likelihood.py --model ${model} --context "1MGHGVSRPPVVTLRPAVLDDCPVLWRWRNDPETRQASVDEREIPVDTHTRWFEETLKRFDRKLFIVSADGVDAGMVRLDIQDRDAAVSVNIAPEWRGRGVGPRALGCLSREAFGPLALLRMSAVVKRENAASRIAFERAGFTVVDTGGPLLHSSKARLHVVAAIQARMGSTRLPGKVLVSIAGRPTIQRIAERLAVCQELDAVAVSTSVENRDDAIADLAAHLGLVCVRGSETDLIERLGRTAARTGADALVRITADCPLVDPALVDRVVGVWRRSAGRLEYVSNVFPPTFPDGLDVEVLSRTVLERLDREVSDPFFRESLTAYVREHPAAFEIANVEHPEDLSRLRWTMDYPEDLAFVEAVYRRLGNQGEIFGMDDLLRLLEWSPELRDLNRCREDVTVERGIRGTGYHAALRARGQAP2"

Citation

If you find our code or paper useful, please cite:

@article{ProGen2,
  title={ProGen2: Exploring the Boundaries of Protein Language Models},
  author={Nijkamp, Erik and Ruffolo, Jeffrey and Weinstein, Eli N. and Naik, Nikhil and Madani, Ali},
  journal={arXiv},
  year={2022}
}

License

Our code and models are BSD-3 licensed. See LICENSE.txt for details.

Ethics

Predicting the fitness of a protein sequence and capturing the distribution of natural proteins for generative purposes could be a powerful tool for protein design. If our technique or a future iteration thereof is adopted broadly, care should be taken in terms of the end use-cases of these designed samples and downstream effects to ensure safe, non-nefarious, and ethical applications. For projects in any domain, active oversight during project initiation, experimental optimization, and deployment phases should be put in place to ensure safe usage and limitation of unintended harmful effects.