hamelsmu/genstruct-7b | Run with an API on Replicate

hamelsmu / genstruct-7b

Genstruct 7B is an instruction-generation model, designed to create valid instructions given a raw text corpus. This enables the creation of new, partially synthetic instruction finetuning datasets from any raw-text corpus. (Updated 1 year, 3 months ago)

Cold

Public
172 runs
L40S

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

title

*string

Shift + Return to add a new line

content

*string

Shift + Return to add a new line

In probability theory, the law of large numbers (LLN) is a mathematical theorem that states that the average of the results obtained from a large number of independent and identical random samples converges to the true value, if it exists.[1] More formally, the LLN states that given a sample of independent and identically distributed values, the sample mean converges to the true mean.In probability theory, the law of large numbers (LLN) is a mathematical theorem that states that the average of the results obtained from a large number of independent and identical random samples converges to the true value, if it exists.[1] More formally, the LLN states that given a sample of independent and identically distributed values, the sample mean converges to the true mean.

temp

number

Default: 0

max_tokens

integer

Default: 1000

Run this model in Node.js with one line of code:

npx create-replicate --model=hamelsmu/genstruct-7b

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run hamelsmu/genstruct-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "hamelsmu/genstruct-7b:161d71ba3f9f8e8ce51dfaec83dfd5ed901b003cfe5208e638b91f636a1a248c",
  {
    input: {
      temp: 0,
      title: "Law of large numbers",
      content: "In probability theory, the law of large numbers (LLN) is a mathematical theorem that states that the average of the results obtained from a large number of independent and identical random samples converges to the true value, if it exists.[1] More formally, the LLN states that given a sample of independent and identically distributed values, the sample mean converges to the true mean.",
      max_tokens: 2000
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run hamelsmu/genstruct-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "hamelsmu/genstruct-7b:161d71ba3f9f8e8ce51dfaec83dfd5ed901b003cfe5208e638b91f636a1a248c",
    input={
        "temp": 0,
        "title": "Law of large numbers",
        "content": "In probability theory, the law of large numbers (LLN) is a mathematical theorem that states that the average of the results obtained from a large number of independent and identical random samples converges to the true value, if it exists.[1] More formally, the LLN states that given a sample of independent and identically distributed values, the sample mean converges to the true mean.",
        "max_tokens": 2000
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run hamelsmu/genstruct-7b using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "hamelsmu/genstruct-7b:161d71ba3f9f8e8ce51dfaec83dfd5ed901b003cfe5208e638b91f636a1a248c",
    "input": {
      "temp": 0,
      "title": "Law of large numbers",
      "content": "In probability theory, the law of large numbers (LLN) is a mathematical theorem that states that the average of the results obtained from a large number of independent and identical random samples converges to the true value, if it exists.[1] More formally, the LLN states that given a sample of independent and identically distributed values, the sample mean converges to the true mean.",
      "max_tokens": 2000
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

I am a statistician. I am working on a project to determine the number of seats that a certain company should have in order to minimize the number of customers that have to wait in line. I have two companies, company A and company B, that are similar in size and that provide the same service. I have access to the number of customers that each company served in the past year, which is a random sample. The number of customers company A served last year is 95,000, and the number of customers company B served last year is 99,000. Which company served more customers on average? [[[Assistant]]] The law of large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. To determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. To determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. To determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. To determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. To determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. Industrially as a larger average. Industrially as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. To determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as

{
  "completed_at": "2024-03-08T05:03:48.986963Z",
  "created_at": "2024-03-08T05:02:26.830993Z",
  "data_removed": false,
  "error": null,
  "id": "qsypk33bkah5lflfq2yffqugxq",
  "input": {
    "temp": 0,
    "title": "Law of large numbers",
    "content": "In probability theory, the law of large numbers (LLN) is a mathematical theorem that states that the average of the results obtained from a large number of independent and identical random samples converges to the true value, if it exists.[1] More formally, the LLN states that given a sample of independent and identically distributed values, the sample mean converges to the true mean.",
    "max_tokens": 2000
  },
  "logs": null,
  "metrics": {
    "predict_time": 48.577139,
    "total_time": 82.15597
  },
  "output": " I am a statistician. I am working on a project to determine the number of seats that a certain company should have in order to minimize the number of customers that have to wait in line. I have two companies, company A and company B, that are similar in size and that provide the same service. I have access to the number of customers that each company served in the past year, which is a random sample. The number of customers company A served last year is 95,000, and the number of customers company B served last year is 99,000. Which company served more customers on average?\n[[[Assistant]]] The law of large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. \n\nTo determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers. \n\nTo determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.\n\nTo determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.\n\nTo determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.\n\nTo determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. Industrially as a larger average. Industrially as a larger average.й large numbers tells us that the average of a large number of independent and identical random samples will converge to the true value, if it exists. In this case, we have two companies, A and B, that are similar in size and provide the same service. We are given the number of customers each company served last year, which can be considered as a random sample. Company A served 95,000 customers and Company B served 99,000 customers.\n\nTo determine which company served more customers on average, we can calculate the average number of customers served per year for each company. The average for Company A is 95,000 customers, and the average for Company B is 99,000 customers. Since the law of large numbers states that the sample mean converges to the true mean, we can conclude that Company B served more customers on average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as a larger average. Industrially as",
  "started_at": "2024-03-08T05:03:00.409824Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/qsypk33bkah5lflfq2yffqugxq",
    "cancel": "https://api.replicate.com/v1/predictions/qsypk33bkah5lflfq2yffqugxq/cancel"
  },
  "version": "62e716be8006a43e858b3e7cba5770e7aafa310539b3d2a379dd33a246bbdcf6"
}

Generated in

48.6 seconds

Tweak it Report View full prediction

This output was created using a different version of the model, hamelsmu/genstruct-7b:62e716be.

Run time and cost

This model costs approximately $0.047 to run on Replicate, or 21 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 49 seconds.

Readme

Hosted version of NousResearch/Genstruct-7B

Files used to host this model are on GitHub.

This model is hosted with vLLM with the following code:

import os
os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"
import torch
from cog import BasePredictor
from vllm import LLM, SamplingParams

def prompt(title, content):
    return f"""[[[Title]]] {title}

[[[Content]]] {content}

The following is an interaction between a user and an AI assistant that is related to the above text.

[[[User]]] """

class Predictor(BasePredictor):

    def setup(self):
        n_gpus = torch.cuda.device_count()
        self.llm = LLM(model='NousResearch/Genstruct-7B', 
                       tensor_parallel_size=n_gpus)

    def predict(self, title: str, content: str, temp:float=0.0, max_tokens:int=2000) -> str:        
        _p = prompt(title, content)
        sampling_params = SamplingParams(temperature=temp, ignore_eos=True, max_tokens=max_tokens)
        out = self.llm.generate(_p, sampling_params=sampling_params, use_tqdm=False)
        return out[0].outputs[0].text

For more information, see these docs.