cjwbw/whisper-downloadable-subtitles – Run with an API on Replicate

cjwbw / whisper-downloadable-subtitles

Added downloadable subtitles for openai/whisper

Cold

Public
2.2K runs
A100 (80GB)

Run with an API

Playground API Examples README Versions

Input

Video Player is loading.

Current Time 00:00:000

Duration 00:00:000

Loaded: 0%

Stream Type LIVE

Remaining Time 00:00:000

audio

*file

Audio file

model

string

Choose a Whisper model.

Default: "base"

translate

boolean

Translate the text to English when set to True

Default: false

subtitle

string

Generate subtitles with the chosen format

Default: "None"

Run this model in Node.js with one line of code:

npx create-replicate --model=cjwbw/whisper-downloadable-subtitles

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run cjwbw/whisper-downloadable-subtitles using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "cjwbw/whisper-downloadable-subtitles:67bca9bd863c13a4cb470a5d8ca4fab66211a51fabc7bc20da70b20c25d736ea",
  {
    input: {
      audio: "https://replicate.delivery/mgxm/f2ee54de-6356-4a7d-82da-78f6057e3ccb/OSR_fr_000_0045_8k.wav",
      model: "base",
      subtitle: "All",
      translate: true
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run cjwbw/whisper-downloadable-subtitles using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "cjwbw/whisper-downloadable-subtitles:67bca9bd863c13a4cb470a5d8ca4fab66211a51fabc7bc20da70b20c25d736ea",
    input={
        "audio": "https://replicate.delivery/mgxm/f2ee54de-6356-4a7d-82da-78f6057e3ccb/OSR_fr_000_0045_8k.wav",
        "model": "base",
        "subtitle": "All",
        "translate": True
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run cjwbw/whisper-downloadable-subtitles using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "67bca9bd863c13a4cb470a5d8ca4fab66211a51fabc7bc20da70b20c25d736ea",
    "input": {
      "audio": "https://replicate.delivery/mgxm/f2ee54de-6356-4a7d-82da-78f6057e3ccb/OSR_fr_000_0045_8k.wav",
      "model": "base",
      "subtitle": "All",
      "translate": true
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/cjwbw/whisper-downloadable-subtitles@sha256:67bca9bd863c13a4cb470a5d8ca4fab66211a51fabc7bc20da70b20c25d736ea \
  -i 'audio="https://replicate.delivery/mgxm/f2ee54de-6356-4a7d-82da-78f6057e3ccb/OSR_fr_000_0045_8k.wav"' \
  -i 'model="base"' \
  -i 'subtitle="All"' \
  -i 'translate=true'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/cjwbw/whisper-downloadable-subtitles@sha256:67bca9bd863c13a4cb470a5d8ca4fab66211a51fabc7bc20da70b20c25d736ea
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "audio": "https://replicate.delivery/mgxm/f2ee54de-6356-4a7d-82da-78f6057e3ccb/OSR_fr_000_0045_8k.wav",
      "model": "base",
      "subtitle": "All",
      "translate": true
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

srt_file

tmpesl6mmczOSR_fr_000_0045_8k.wav.srt

txt_file

tmpesl6mmczOSR_fr_000_0045_8k.wav.txt

translation

The cultural center organized a round table. Its universe is the one of its families. The battle is fighting to keep face to face. No matter how many people are there, they will not be taken to this age. The Dutch and Olivier are a wonderful brave. Do you know this expression? The actors prepare to show their love. The country and the Dutch have seen the high school. You will see my view of what I am going to do. I have already lived a lot. You will not be able to see. The manual is still in sight. He signed a contract with 5 people. The men's vegoes are transformed into a knight. He is going to the credibility. All the eyes are broken on him. This time, luck can heal us. The battle is fast and silent.

transcription

Le centre culturel a organisé une table ronde. Son univers est celui de sa famille. Le bateau lutte pour tenir face au vent. Aucune demande ne sera prise en compte après cette date. Rolon et Olivier sont d'une merveilleuse brahoure. Connecis-vous cette expression? Les acteurs se préparent à monter en scène. Le pays et jeolier vio de la colline. Tu vas voir à mon vieux ce que je vais faire. De fait, Jacques buvez beaucoup. Tu n'y iras certaines mots pas. Émanuelle s'ombe encore à voir vieille. Il a signé un contrat de 5 ans. Des vegonnets de mine sont transformés en un brevoir. Il y a aller de leur crédibilité. Tous les yeux étaient braqués sur lui. Cette fois, la chance peut nous surrir. Le bateau filait rapide et silencieux.

detected_language

french

{
  "completed_at": "2022-10-03T20:58:27.411643Z",
  "created_at": "2022-10-03T20:55:30.873008Z",
  "data_removed": false,
  "error": null,
  "id": "sewqnijymzfudnpiabz6ocl5wi",
  "input": {
    "audio": "https://replicate.delivery/mgxm/f2ee54de-6356-4a7d-82da-78f6057e3ccb/OSR_fr_000_0045_8k.wav",
    "model": "base",
    "subtitle": "All",
    "translate": true
  },
  "logs": "Transcribe with base model",
  "metrics": {
    "predict_time": 14.789779,
    "total_time": 176.538635
  },
  "output": {
    "srt_file": "https://replicate.delivery/mgxm/b35a063e-9ab4-48ba-8875-3e796478871e/tmpesl6mmczOSR_fr_000_0045_8k.wav.srt",
    "txt_file": "https://replicate.delivery/mgxm/52eb0dc3-a476-438e-a568-0166e21edc89/tmpesl6mmczOSR_fr_000_0045_8k.wav.txt",
    "translation": " The cultural center organized a round table. Its universe is the one of its families. The battle is fighting to keep face to face. No matter how many people are there, they will not be taken to this age. The Dutch and Olivier are a wonderful brave. Do you know this expression? The actors prepare to show their love. The country and the Dutch have seen the high school. You will see my view of what I am going to do. I have already lived a lot. You will not be able to see. The manual is still in sight. He signed a contract with 5 people. The men's vegoes are transformed into a knight. He is going to the credibility. All the eyes are broken on him. This time, luck can heal us. The battle is fast and silent.",
    "transcription": " Le centre culturel a organisé une table ronde. Son univers est celui de sa famille. Le bateau lutte pour tenir face au vent. Aucune demande ne sera prise en compte après cette date. Rolon et Olivier sont d'une merveilleuse brahoure. Connecis-vous cette expression? Les acteurs se préparent à monter en scène. Le pays et jeolier vio de la colline. Tu vas voir à mon vieux ce que je vais faire. De fait, Jacques buvez beaucoup. Tu n'y iras certaines mots pas. Émanuelle s'ombe encore à voir vieille. Il a signé un contrat de 5 ans. Des vegonnets de mine sont transformés en un brevoir. Il y a aller de leur crédibilité. Tous les yeux étaient braqués sur lui. Cette fois, la chance peut nous surrir. Le bateau filait rapide et silencieux.",
    "detected_language": "french"
  },
  "started_at": "2022-10-03T20:58:12.621864Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/sewqnijymzfudnpiabz6ocl5wi",
    "cancel": "https://api.replicate.com/v1/predictions/sewqnijymzfudnpiabz6ocl5wi/cancel"
  },
  "version": "67bca9bd863c13a4cb470a5d8ca4fab66211a51fabc7bc20da70b20c25d736ea"
}

Generated in

14.8 seconds

Tweak itReport

Run time and cost

This model costs approximately $0.40 to run on Replicate, or 2 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia A100 (80GB) GPU hardware. Predictions typically complete within 5 minutes. The predict time for this model varies significantly based on the inputs.

Readme

This demo enables the downloadable subtitle option for https://github.com/openai/whisper.

Implementation: https://github.com/chenxwh/cog-whisper/tree/subtitle .vtt file is not supported by Replicate for downloading yet, therefore the format is not included.