victor-upmeet / whisperx

Accelerated transcription, word-level timestamps and diarization with whisperX large-v3

Warm

Public
1.8M runs
A100 (80GB)
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Prediction

victor-upmeet/whisperx:84d2ad2d

Model

victor-upmeet/whisperx:77505c70

h2ovig3bxqz5wwgyexzumicsam

Status

Succeeded

Source

Web

Hardware

A40

Total duration

3.6s

Created

over 1 year ago

Input

debug
vad_onset: 0.5
audio_file: Video Player is loading.
Current Time 00:00:000
Duration 00:00:000
Loaded: 0%
Stream Type LIVE
Remaining Time 00:00:000

1x
Chapters
descriptions off, selected
captions settings, opens captions settings dialog
captions off, selected
Beginning of dialog window. Escape will cancel and close the window.
TextColorTransparency
BackgroundColorTransparency
WindowColorTransparency
Font Size
Text Edge Style
Font Family
End of dialog window.
batch_size: 64
vad_offset: 0.363
diarization
temperature: 0
align_output

{
  "debug": false,
  "vad_onset": 0.5,
  "audio_file": "https://replicate.delivery/pbxt/JrvsggK5WvFQ4Q53h4ugPbXW0LK2BLnMZm2dCPhM8bodUq5w/OSR_uk_000_0050_8k.wav",
  "batch_size": 64,
  "vad_offset": 0.363,
  "diarization": false,
  "temperature": 0,
  "align_output": false
}

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run victor-upmeet/whisperx using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "victor-upmeet/whisperx:84d2ad2d6194fe98a17d2b60bef1c7f910c46b2f6fd38996ca457afd9c8abfcb",
  {
    input: {
      debug: false,
      vad_onset: 0.5,
      audio_file: "https://replicate.delivery/pbxt/JrvsggK5WvFQ4Q53h4ugPbXW0LK2BLnMZm2dCPhM8bodUq5w/OSR_uk_000_0050_8k.wav",
      batch_size: 64,
      vad_offset: 0.363,
      diarization: false,
      temperature: 0,
      align_output: false
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run victor-upmeet/whisperx using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "victor-upmeet/whisperx:84d2ad2d6194fe98a17d2b60bef1c7f910c46b2f6fd38996ca457afd9c8abfcb",
    input={
        "debug": False,
        "vad_onset": 0.5,
        "audio_file": "https://replicate.delivery/pbxt/JrvsggK5WvFQ4Q53h4ugPbXW0LK2BLnMZm2dCPhM8bodUq5w/OSR_uk_000_0050_8k.wav",
        "batch_size": 64,
        "vad_offset": 0.363,
        "diarization": False,
        "temperature": 0,
        "align_output": False
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run victor-upmeet/whisperx using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "84d2ad2d6194fe98a17d2b60bef1c7f910c46b2f6fd38996ca457afd9c8abfcb",
    "input": {
      "debug": false,
      "vad_onset": 0.5,
      "audio_file": "https://replicate.delivery/pbxt/JrvsggK5WvFQ4Q53h4ugPbXW0LK2BLnMZm2dCPhM8bodUq5w/OSR_uk_000_0050_8k.wav",
      "batch_size": 64,
      "vad_offset": 0.363,
      "diarization": false,
      "temperature": 0,
      "align_output": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

segments

[ { "end": 30.811, "text": " The little tales they tell are false. The door was barred, locked and bolted as well. Ripe pears are fit for a queen's table. A big wet stain was on the round carpet. The kite dipped and swayed but stayed aloft. The pleasant hours fly by much too soon. The room was crowded with a mild wob.", "start": 2.585 }, { "end": 48.592, "text": " The room was crowded with a wild mob. This strong arm shall shield your honor. She blushed when he gave her a white orchid. The beetle droned in the hot June sun.", "start": 33.029 } ]

detected_language

{
  "completed_at": "2023-11-13T08:47:22.035804Z",
  "created_at": "2023-11-13T08:47:18.460418Z",
  "data_removed": false,
  "error": null,
  "id": "h2ovig3bxqz5wwgyexzumicsam",
  "input": {
    "debug": false,
    "vad_onset": 0.5,
    "audio_file": "https://replicate.delivery/pbxt/JrvsggK5WvFQ4Q53h4ugPbXW0LK2BLnMZm2dCPhM8bodUq5w/OSR_uk_000_0050_8k.wav",
    "batch_size": 64,
    "vad_offset": 0.363,
    "diarization": false,
    "temperature": 0,
    "align_output": false
  },
  "logs": "No language specified, language will be first be detected for each audio file (increases inference time).\nLightning automatically upgraded your loaded checkpoint from v1.5.4 to v2.1.1. To apply the upgrade to your files permanently, run `python -m pytorch_lightning.utilities.upgrade_checkpoint ../root/.cache/torch/whisperx-vad-segmentation.bin`\nModel was trained with pyannote.audio 0.0.1, yours is 3.0.1. Bad things might happen unless you revert pyannote.audio to 0.x.\nModel was trained with torch 1.10.0+cu102, yours is 2.1.0+cu121. Bad things might happen unless you revert torch to 1.x.\nDetected language: en (1.00) in first 30s of audio...",
  "metrics": {
    "predict_time": 3.596677,
    "total_time": 3.575386
  },
  "output": {
    "segments": [
      {
        "end": 30.811,
        "text": " The little tales they tell are false. The door was barred, locked and bolted as well. Ripe pears are fit for a queen's table. A big wet stain was on the round carpet. The kite dipped and swayed but stayed aloft. The pleasant hours fly by much too soon. The room was crowded with a mild wob.",
        "start": 2.585
      },
      {
        "end": 48.592,
        "text": " The room was crowded with a wild mob. This strong arm shall shield your honor. She blushed when he gave her a white orchid. The beetle droned in the hot June sun.",
        "start": 33.029
      }
    ],
    "detected_language": "en"
  },
  "started_at": "2023-11-13T08:47:18.439127Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/h2ovig3bxqz5wwgyexzumicsam",
    "cancel": "https://api.replicate.com/v1/predictions/h2ovig3bxqz5wwgyexzumicsam/cancel"
  },
  "version": "77505c700514deed62ab3891c0011e307f905ee527458afc15de7d9e2a3034e8"
}

Generated in

3.6 seconds

Tweak it Share Report

Want to make some of these yourself?

Run this model