meta/llama-2-13b-chat:56acad22 – Run with an API on Replicate

meta /llama-2-13b-chat:56acad22

Input

prompt

*string

Shift + Return to add a new line

User: Write a python script that trains `bert-large` on the `IMDB` dataset using the Transformers `Trainer` class and Datasets library. I have access to four GPUs, so let's use DDP. Please write the script and then tell me how to launch it on the command line.
Assistant:User: Write a python script that trains `bert-large` on the `IMDB` dataset using the Transformers `Trainer` class and Datasets library. I have access to four GPUs, so let's use DDP. Please write the script and then tell me how to launch it on the command line.
Assistant:

Prompt to send to Llama v2.

max_length

integer

(minimum: 1)

Maximum number of tokens to generate. A word is generally 2-3 tokens

Default: 500

temperature

number

(minimum: 0.01, maximum: 5)

Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.

Default: 0.75

top_p

number

(minimum: 0.01, maximum: 1)

When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens

Default: 1

repetition_penalty

number

(minimum: 0.01, maximum: 5)

Penalty for repeated words in generated text; 1 is no penalty, values greater than 1 discourage repetition, less than 1 encourage it.

Default: 1

debug

boolean

provide debugging output in logs

Default: false

Run this model in Node.js with one line of code:

npx create-replicate --model=meta/llama-2-13b-chat

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run meta/llama-2-13b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "meta/llama-2-13b-chat:56acad22679f6b95d6e45c78309a2b50a670d5ed29a37dd73d182e89772c02f1",
  {
    input: {
      debug: false,
      top_p: 1,
      prompt: "User: Write a python script that trains `bert-large` on the `IMDB` dataset using the Transformers `Trainer` class and Datasets library. I have access to four GPUs, so let's use DDP. Please write the script and then tell me how to launch it on the command line.\nAssistant:",
      max_length: 2000,
      temperature: 0.25,
      repetition_penalty: 1
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run meta/llama-2-13b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "meta/llama-2-13b-chat:56acad22679f6b95d6e45c78309a2b50a670d5ed29a37dd73d182e89772c02f1",
    input={
        "debug": False,
        "top_p": 1,
        "prompt": "User: Write a python script that trains `bert-large` on the `IMDB` dataset using the Transformers `Trainer` class and Datasets library. I have access to four GPUs, so let's use DDP. Please write the script and then tell me how to launch it on the command line.\nAssistant:",
        "max_length": 2000,
        "temperature": 0.25,
        "repetition_penalty": 1
    }
)

# The meta/llama-2-13b-chat model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/meta/llama-2-13b-chat/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run meta/llama-2-13b-chat using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "56acad22679f6b95d6e45c78309a2b50a670d5ed29a37dd73d182e89772c02f1",
    "input": {
      "debug": false,
      "top_p": 1,
      "prompt": "User: Write a python script that trains `bert-large` on the `IMDB` dataset using the Transformers `Trainer` class and Datasets library. I have access to four GPUs, so let\'s use DDP. Please write the script and then tell me how to launch it on the command line.\\nAssistant:",
      "max_length": 2000,
      "temperature": 0.25,
      "repetition_penalty": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Sure! Here is the script you requested: ``` import torch from transformers import BertTokenizer, BertModel from transformers.training import Trainer from torch.utils.data import DataLoader from datasets import load_dataset # Load the IMDB dataset train_dataset = load_dataset("imdb", split="train") val_dataset = load_dataset("imdb", split="val") # Load the BERT model tokenizer = BertTokenizer.from_pretrained("bert-base-uncased") model = BertModel.from_pretrained("bert-base-uncased") # Define the training and validation dataloaders train_dataloader = DataLoader(train_dataset, batch_size=32, shuffle=True) val_dataloader = DataLoader(val_dataset, batch_size=32, shuffle=False) # Define the training arguments training_args = TrainingArguments( output_dir="./results", num_train_epochs=3, per_device_train_batch_size=32, per_device_eval_batch_size=32, evaluation_strategy="epoch", learning_rate=5e-5, save_total_limit=2, save_steps=500, load_best_model_at_end=True, metric_for_best_model="f1", greater_is_better=True, save_strategy="steps", save_on_each_node=True, ) # Train the model trainer = Trainer( model=model, args=training_args, train_dataset=train_dataset, eval_dataset=val_dataset, compute_metrics=lambda pred: {"f1": f1_score(y_true=pred.label_ids, y_pred=pred.predictions.argmax, average="macro")}, ) trainer.train() # Print the final trained model print(trainer.model) ``` To launch the script on the command line, you can use the following command: ``` python -m torch.distributed.launch training_bert_imdb.py --n_gpus 4 ``` This will launch the script on four GPUs. You can adjust the number of GPUs as needed. I hope this helps! Let me know if you have any questions.

{
  "completed_at": "2023-07-17T13:43:41.916818Z",
  "created_at": "2023-07-17T13:43:24.954078Z",
  "data_removed": false,
  "error": null,
  "id": "gzcomldbikyay2s6q6744lsyt4",
  "input": {
    "top_p": 1,
    "prompt": "User: Write a python script that trains `bert-large` on the `IMDB` dataset using the Transformers `Trainer` class and Datasets library. I have access to four GPUs, so let's use DDP. Please write the script and then tell me how to launch it on the command line.\nAssistant:",
    "max_length": 2000,
    "temperature": 0.25,
    "repetition_penalty": 1
  },
  "logs": null,
  "metrics": {
    "predict_time": 16.957151,
    "total_time": 16.96274
  },
  "output": [
    "Sure!",
    " Here",
    " is",
    " the",
    " script",
    " you",
    " requested:\n```\nimport",
    " torch\nfrom",
    " transformers",
    " import",
    " BertTokenizer,",
    " BertModel\nfrom",
    " transformers.training",
    " import",
    " Trainer\nfrom",
    " torch.utils.data",
    " import",
    " DataLoader\nfrom",
    " datasets",
    " import",
    " load_dataset\n\n#",
    " Load",
    " the",
    " IMDB",
    " dataset\ntrain_dataset",
    " =",
    " load_dataset(\"imdb\",",
    " split=\"train\")\nval_dataset",
    " =",
    " load_dataset(\"imdb\",",
    " split=\"val\")\n\n#",
    " Load",
    " the",
    " BERT",
    " model\ntokenizer",
    " =",
    " BertTokenizer.from_pretrained(\"bert-base-uncased\")\nmodel",
    " =",
    " BertModel.from_pretrained(\"bert-base-uncased\")\n\n#",
    " Define",
    " the",
    " training",
    " and",
    " validation",
    " dataloaders\ntrain_dataloader",
    " =",
    " DataLoader(train_dataset,",
    " batch_size=32,",
    " shuffle=True)\nval_dataloader",
    " =",
    " DataLoader(val_dataset,",
    " batch_size=32,",
    " shuffle=False)\n\n#",
    " Define",
    " the",
    " training",
    " arguments\ntraining_args",
    " =",
    " TrainingArguments(\n",
    "   ",
    " output_dir=\"./results\",\n",
    "   ",
    " num_train_epochs=3,\n",
    "   ",
    " per_device_train_batch_size=32,\n",
    "   ",
    " per_device_eval_batch_size=32,\n",
    "   ",
    " evaluation_strategy=\"epoch\",\n",
    "   ",
    " learning_rate=5e-5,\n",
    "   ",
    " save_total_limit=2,\n",
    "   ",
    " save_steps=500,\n",
    "   ",
    " load_best_model_at_end=True,\n",
    "   ",
    " metric_for_best_model=\"f1\",\n",
    "   ",
    " greater_is_better=True,\n",
    "   ",
    " save_strategy=\"steps\",\n",
    "   ",
    " save_on_each_node=True,\n)\n\n#",
    " Train",
    " the",
    " model\ntrainer",
    " =",
    " Trainer(\n",
    "   ",
    " model=model,\n",
    "   ",
    " args=training_args,\n",
    "   ",
    " train_dataset=train_dataset,\n",
    "   ",
    " eval_dataset=val_dataset,\n",
    "   ",
    " compute_metrics=lambda",
    " pred:",
    " {\"f1\":",
    " f1_score(y_true=pred.label_ids,",
    " y_pred=pred.predictions.argmax,",
    " average=\"macro\")},\n)\ntrainer.train()\n\n#",
    " Print",
    " the",
    " final",
    " trained",
    " model\nprint(trainer.model)\n```\nTo",
    " launch",
    " the",
    " script",
    " on",
    " the",
    " command",
    " line,",
    " you",
    " can",
    " use",
    " the",
    " following",
    " command:\n```\npython",
    " -m",
    " torch.distributed.launch",
    " training_bert_imdb.py",
    " --n_gpus",
    " 4\n```\nThis",
    " will",
    " launch",
    " the",
    " script",
    " on",
    " four",
    " GPUs.",
    " You",
    " can",
    " adjust",
    " the",
    " number",
    " of",
    " GPUs",
    " as",
    " needed.\n\nI",
    " hope",
    " this",
    " helps!",
    " Let",
    " me",
    " know",
    " if",
    " you",
    " have",
    " any",
    " questions."
  ],
  "started_at": "2023-07-17T13:43:24.959667Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/gzcomldbikyay2s6q6744lsyt4",
    "cancel": "https://api.replicate.com/v1/predictions/gzcomldbikyay2s6q6744lsyt4/cancel"
  },
  "version": "56acad22679f6b95d6e45c78309a2b50a670d5ed29a37dd73d182e89772c02f1"
}

Generated in

17.0 seconds

Tweak it Report