isaacgv/vec2 – Run with an API on Replicate

isaacgv / vec2

Cold

Public
105 runs
L40S

Run with an API

Playground API Examples README Versions

Input

Video Player is loading.

Current Time 00:00:000

Duration 00:00:000

Loaded: 0%

Stream Type LIVE

Remaining Time 00:00:000

audio

*file

Audio file

model

string

Choose a Whisper model.

Default: "large-v2"

temperature

number

temperature to use for sampling

Default: 0

language

string

language spoken in the audio, specify None to perform language detection

Run this model in Node.js with one line of code:

npx create-replicate --model=isaacgv/vec2

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run isaacgv/vec2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "isaacgv/vec2:2faa07c8f78bcb72c27bc4d6ee62bcc15fdda228e4a46c8434e6f3ccd2df34b7",
  {
    input: {
      audio: "https://replicate.delivery/pbxt/JdiHR0a8YAD50La2BOcan3rBIuQsNfEREqwLYQo2U5s0t1hu/audio%20%283%29.mp3",
      model: "large-v2",
      language: "af",
      temperature: 0
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run isaacgv/vec2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "isaacgv/vec2:2faa07c8f78bcb72c27bc4d6ee62bcc15fdda228e4a46c8434e6f3ccd2df34b7",
    input={
        "audio": "https://replicate.delivery/pbxt/JdiHR0a8YAD50La2BOcan3rBIuQsNfEREqwLYQo2U5s0t1hu/audio%20%283%29.mp3",
        "model": "large-v2",
        "language": "af",
        "temperature": 0
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run isaacgv/vec2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "2faa07c8f78bcb72c27bc4d6ee62bcc15fdda228e4a46c8434e6f3ccd2df34b7",
    "input": {
      "audio": "https://replicate.delivery/pbxt/JdiHR0a8YAD50La2BOcan3rBIuQsNfEREqwLYQo2U5s0t1hu/audio%20%283%29.mp3",
      "model": "large-v2",
      "language": "af",
      "temperature": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/isaacgv/vec2@sha256:2faa07c8f78bcb72c27bc4d6ee62bcc15fdda228e4a46c8434e6f3ccd2df34b7 \
  -i 'audio="https://replicate.delivery/pbxt/JdiHR0a8YAD50La2BOcan3rBIuQsNfEREqwLYQo2U5s0t1hu/audio%20%283%29.mp3"' \
  -i 'model="large-v2"' \
  -i 'language="af"' \
  -i 'temperature=0'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/isaacgv/vec2@sha256:2faa07c8f78bcb72c27bc4d6ee62bcc15fdda228e4a46c8434e6f3ccd2df34b7
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "audio": "https://replicate.delivery/pbxt/JdiHR0a8YAD50La2BOcan3rBIuQsNfEREqwLYQo2U5s0t1hu/audio%20%283%29.mp3",
      "model": "large-v2",
      "language": "af",
      "temperature": 0
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

segments

[ { "end": 2.459, "text": " Vă face la fel și în acest caz.", "start": 0.34, "words": [ { "end": 0.804, "word": "Vă", "score": 0.166, "start": 0.34 }, { "end": 1.147, "word": "face", "score": 0.383, "start": 0.865 }, { "end": 1.309, "word": "la", "score": 0.5, "start": 1.188 }, { "end": 1.591, "word": "fel", "score": 0.361, "start": 1.369 }, { "end": 1.712, "word": "și", "score": 0.504, "start": 1.632 }, { "end": 1.834, "word": "în", "score": 0.506, "start": 1.753 }, { "end": 2.217, "word": "acest", "score": 0.365, "start": 1.874 }, { "end": 2.459, "word": "caz.", "score": 0.268, "start": 2.278 } ] } ]

detected_language

{
  "completed_at": "2023-10-04T07:21:41.818868Z",
  "created_at": "2023-10-04T07:21:37.764231Z",
  "data_removed": false,
  "error": null,
  "id": "w3bh5rdbh4lo6unn434ticllsa",
  "input": {
    "audio": "https://replicate.delivery/pbxt/JdiHR0a8YAD50La2BOcan3rBIuQsNfEREqwLYQo2U5s0t1hu/audio%20%283%29.mp3",
    "model": "large-v2",
    "temperature": 0
  },
  "logs": "Transcribe with large-v2 model\nDetected language: Romanian\n  0%|          | 0/342 [00:00<?, ?frames/s]\n100%|██████████| 342/342 [00:01<00:00, 323.52frames/s]\n100%|██████████| 342/342 [00:01<00:00, 323.46frames/s]\nSpecial tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.",
  "metrics": {
    "predict_time": 4.197268,
    "total_time": 4.054637
  },
  "output": {
    "segments": [
      {
        "end": 2.459,
        "text": " Vă face la fel și în acest caz.",
        "start": 0.34,
        "words": [
          {
            "end": 0.804,
            "word": "Vă",
            "score": 0.166,
            "start": 0.34
          },
          {
            "end": 1.147,
            "word": "face",
            "score": 0.383,
            "start": 0.865
          },
          {
            "end": 1.309,
            "word": "la",
            "score": 0.5,
            "start": 1.188
          },
          {
            "end": 1.591,
            "word": "fel",
            "score": 0.361,
            "start": 1.369
          },
          {
            "end": 1.712,
            "word": "și",
            "score": 0.504,
            "start": 1.632
          },
          {
            "end": 1.834,
            "word": "în",
            "score": 0.506,
            "start": 1.753
          },
          {
            "end": 2.217,
            "word": "acest",
            "score": 0.365,
            "start": 1.874
          },
          {
            "end": 2.459,
            "word": "caz.",
            "score": 0.268,
            "start": 2.278
          }
        ]
      }
    ],
    "detected_language": "ro"
  },
  "started_at": "2023-10-04T07:21:37.621600Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/w3bh5rdbh4lo6unn434ticllsa",
    "cancel": "https://api.replicate.com/v1/predictions/w3bh5rdbh4lo6unn434ticllsa/cancel"
  },
  "version": "ec9bcadf686c05e859d49f81755ee7427368d44d2350a5f1ec897c04d654201b"
}

Generated in

4.2 seconds

Tweak it Report

This example was created by a different version, isaacgv/vec2:ec9bcadf.

Run time and cost

This model runs on Nvidia L40S GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

This model doesn't have a readme.