tomasmcm / gorilla-openfunctions-v1

Source: gorilla-llm/gorilla-openfunctions-v1 ✦ Quant: TheBloke/gorilla-openfunctions-v1-AWQ ✦ Extend Large Language Model (LLM) Chat Completion feature to formulate executable APIs call given natural language instructions and API context

Cold

Public
415 runs
L40S
Paper
License

Run with an API

Playground API Examples README Versions

Prediction

tomasmcm/gorilla-openfunctions-v1:574ca2df

Model

tomasmcm/gorilla-openfunctions-v1:574ca2df

3uzviltb4h6g6bbjpfp55yhkhq

Status

Succeeded

Source

Web

Hardware

A40

Total duration

76.2s

Created

about 1 year ago

Input

top_k: -1
top_p: 0.95
prompt: USER: <<question>> Call me an Uber ride type "Plus" in Berkeley at zipcode 94704 in 10 minutes <<function>> [{"name": "Uber Carpool", "api_name": "uber.ride", "description": "Find suitable ride for customers given the location, type of ride, and the amount of time the customer is willing to wait as parameters", "parameters": [{"name": "loc", "description": "Location of the starting place of the Uber ride"}, {"name": "type", "enum": ["plus", "comfort", "black"], "description": "Types of Uber ride user is ordering"}, {"name": "time", "description": "The amount of time in minutes the customer is willing to wait"}]}] ASSISTANT:
max_tokens: 128
temperature: 0.8
presence_penalty: 0
frequency_penalty: 0

{
  "top_k": -1,
  "top_p": 0.95,
  "prompt": "USER: <<question>> Call me an Uber ride type \"Plus\" in Berkeley at zipcode 94704 in 10 minutes <<function>> [{\"name\": \"Uber Carpool\", \"api_name\": \"uber.ride\", \"description\": \"Find suitable ride for customers given the location, type of ride, and the amount of time the customer is willing to wait as parameters\", \"parameters\": [{\"name\": \"loc\", \"description\": \"Location of the starting place of the Uber ride\"}, {\"name\": \"type\", \"enum\": [\"plus\", \"comfort\", \"black\"], \"description\": \"Types of Uber ride user is ordering\"}, {\"name\": \"time\", \"description\": \"The amount of time in minutes the customer is willing to wait\"}]}]\nASSISTANT: ",
  "max_tokens": 128,
  "temperature": 0.8,
  "presence_penalty": 0,
  "frequency_penalty": 0
}

Install Replicate’s Node.js client library

npm install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run tomasmcm/gorilla-openfunctions-v1 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "tomasmcm/gorilla-openfunctions-v1:574ca2dfccd6ea5006ae008b773dd9d5a1da77f978fe183f4f6452bbc9f62aba",
  {
    input: {
      top_k: -1,
      top_p: 0.95,
      prompt: "USER: <<question>> Call me an Uber ride type \"Plus\" in Berkeley at zipcode 94704 in 10 minutes <<function>> [{\"name\": \"Uber Carpool\", \"api_name\": \"uber.ride\", \"description\": \"Find suitable ride for customers given the location, type of ride, and the amount of time the customer is willing to wait as parameters\", \"parameters\": [{\"name\": \"loc\", \"description\": \"Location of the starting place of the Uber ride\"}, {\"name\": \"type\", \"enum\": [\"plus\", \"comfort\", \"black\"], \"description\": \"Types of Uber ride user is ordering\"}, {\"name\": \"time\", \"description\": \"The amount of time in minutes the customer is willing to wait\"}]}]\nASSISTANT: ",
      max_tokens: 128,
      temperature: 0.8,
      presence_penalty: 0,
      frequency_penalty: 0
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library

pip install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client

import replicate

Run tomasmcm/gorilla-openfunctions-v1 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "tomasmcm/gorilla-openfunctions-v1:574ca2dfccd6ea5006ae008b773dd9d5a1da77f978fe183f4f6452bbc9f62aba",
    input={
        "top_k": -1,
        "top_p": 0.95,
        "prompt": "USER: <<question>> Call me an Uber ride type \"Plus\" in Berkeley at zipcode 94704 in 10 minutes <<function>> [{\"name\": \"Uber Carpool\", \"api_name\": \"uber.ride\", \"description\": \"Find suitable ride for customers given the location, type of ride, and the amount of time the customer is willing to wait as parameters\", \"parameters\": [{\"name\": \"loc\", \"description\": \"Location of the starting place of the Uber ride\"}, {\"name\": \"type\", \"enum\": [\"plus\", \"comfort\", \"black\"], \"description\": \"Types of Uber ride user is ordering\"}, {\"name\": \"time\", \"description\": \"The amount of time in minutes the customer is willing to wait\"}]}]\nASSISTANT: ",
        "max_tokens": 128,
        "temperature": 0.8,
        "presence_penalty": 0,
        "frequency_penalty": 0
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run tomasmcm/gorilla-openfunctions-v1 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "574ca2dfccd6ea5006ae008b773dd9d5a1da77f978fe183f4f6452bbc9f62aba",
    "input": {
      "top_k": -1,
      "top_p": 0.95,
      "prompt": "USER: <<question>> Call me an Uber ride type \\"Plus\\" in Berkeley at zipcode 94704 in 10 minutes <<function>> [{\\"name\\": \\"Uber Carpool\\", \\"api_name\\": \\"uber.ride\\", \\"description\\": \\"Find suitable ride for customers given the location, type of ride, and the amount of time the customer is willing to wait as parameters\\", \\"parameters\\": [{\\"name\\": \\"loc\\", \\"description\\": \\"Location of the starting place of the Uber ride\\"}, {\\"name\\": \\"type\\", \\"enum\\": [\\"plus\\", \\"comfort\\", \\"black\\"], \\"description\\": \\"Types of Uber ride user is ordering\\"}, {\\"name\\": \\"time\\", \\"description\\": \\"The amount of time in minutes the customer is willing to wait\\"}]}]\\nASSISTANT: ",
      "max_tokens": 128,
      "temperature": 0.8,
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Install Cog

brew install cog

If you don’t have Homebrew, there are other installation options available.

Pull and run tomasmcm/gorilla-openfunctions-v1 using Cog (this will download the full model and run it in your local environment):

cog predict r8.im/tomasmcm/gorilla-openfunctions-v1@sha256:574ca2dfccd6ea5006ae008b773dd9d5a1da77f978fe183f4f6452bbc9f62aba \
  -i 'top_k=-1' \
  -i 'top_p=0.95' \
  -i $'prompt="USER: <<question>> Call me an Uber ride type \\"Plus\\" in Berkeley at zipcode 94704 in 10 minutes <<function>> [{\\"name\\": \\"Uber Carpool\\", \\"api_name\\": \\"uber.ride\\", \\"description\\": \\"Find suitable ride for customers given the location, type of ride, and the amount of time the customer is willing to wait as parameters\\", \\"parameters\\": [{\\"name\\": \\"loc\\", \\"description\\": \\"Location of the starting place of the Uber ride\\"}, {\\"name\\": \\"type\\", \\"enum\\": [\\"plus\\", \\"comfort\\", \\"black\\"], \\"description\\": \\"Types of Uber ride user is ordering\\"}, {\\"name\\": \\"time\\", \\"description\\": \\"The amount of time in minutes the customer is willing to wait\\"}]}]\\nASSISTANT: "' \
  -i 'max_tokens=128' \
  -i 'temperature=0.8' \
  -i 'presence_penalty=0' \
  -i 'frequency_penalty=0'

To learn more, take a look at the Cog documentation.

Pull and run tomasmcm/gorilla-openfunctions-v1 using Docker (this will download the full model and run it in your local environment):

docker run -d -p 5000:5000 --gpus=all r8.im/tomasmcm/gorilla-openfunctions-v1@sha256:574ca2dfccd6ea5006ae008b773dd9d5a1da77f978fe183f4f6452bbc9f62aba
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "top_k": -1,
      "top_p": 0.95,
      "prompt": "USER: <<question>> Call me an Uber ride type \\"Plus\\" in Berkeley at zipcode 94704 in 10 minutes <<function>> [{\\"name\\": \\"Uber Carpool\\", \\"api_name\\": \\"uber.ride\\", \\"description\\": \\"Find suitable ride for customers given the location, type of ride, and the amount of time the customer is willing to wait as parameters\\", \\"parameters\\": [{\\"name\\": \\"loc\\", \\"description\\": \\"Location of the starting place of the Uber ride\\"}, {\\"name\\": \\"type\\", \\"enum\\": [\\"plus\\", \\"comfort\\", \\"black\\"], \\"description\\": \\"Types of Uber ride user is ordering\\"}, {\\"name\\": \\"time\\", \\"description\\": \\"The amount of time in minutes the customer is willing to wait\\"}]}]\\nASSISTANT: ",
      "max_tokens": 128,
      "temperature": 0.8,
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  http://localhost:5000/predictions

Output

uber.ride(loc="94704", type="plus", time=10)

Generated in

0.3 seconds

Tweak it Share Report

Want to make some of these yourself?

Run this model