cjwbw / distil-whisper

Distilled version of Whisper

Cold

Public
275 runs
L40S
GitHub
Paper
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

Video Player is loading.

Current Time 00:00:000

Duration 00:00:000

Loaded: 0%

Stream Type LIVE

Remaining Time 00:00:000

audio

*file

Input audio file

model_name

string

Choose a model.

Default: "distil-whisper/distil-large-v2"

long_form_transcription

boolean

Enable chunked algorithm to transcribe long-form audio files.

Default: false

max_new_tokens

integer

Maximum number of new tokens to output.

Default: 128

Run this model in Node.js with one line of code:

npx create-replicate --model=cjwbw/distil-whisper

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run cjwbw/distil-whisper using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "cjwbw/distil-whisper:0a087deaca947454c2543451b52f74b9f05de21ff7848dc0e2b20d3e0ddfa100",
  {
    input: {
      audio: "https://replicate.delivery/pbxt/Jo9t4FqOSuJfla5FDD0TOgaLwIZiv0jETZXc5SWlL9ZtGFIg/1.wav",
      model_name: "distil-whisper/distil-large-v2",
      max_new_tokens: 128,
      long_form_transcription: false
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run cjwbw/distil-whisper using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "cjwbw/distil-whisper:0a087deaca947454c2543451b52f74b9f05de21ff7848dc0e2b20d3e0ddfa100",
    input={
        "audio": "https://replicate.delivery/pbxt/Jo9t4FqOSuJfla5FDD0TOgaLwIZiv0jETZXc5SWlL9ZtGFIg/1.wav",
        "model_name": "distil-whisper/distil-large-v2",
        "max_new_tokens": 128,
        "long_form_transcription": False
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run cjwbw/distil-whisper using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "cjwbw/distil-whisper:0a087deaca947454c2543451b52f74b9f05de21ff7848dc0e2b20d3e0ddfa100",
    "input": {
      "audio": "https://replicate.delivery/pbxt/Jo9t4FqOSuJfla5FDD0TOgaLwIZiv0jETZXc5SWlL9ZtGFIg/1.wav",
      "model_name": "distil-whisper/distil-large-v2",
      "max_new_tokens": 128,
      "long_form_transcription": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

others will be discontinued and need to be replaced by new benchmark rates.

Generated in

2.4 seconds

Tweak it Report

This output was created using a different version of the model, cjwbw/distil-whisper:d843a073.

Run time and cost

This model costs approximately $0.067 to run on Replicate, or 14 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 69 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Distil-Whisper

Distil-Whisper is a distilled version of Whisper that is 6 times faster, 49% smaller, and performs within 1% word error rate (WER) on out-of-distribution evaluation sets.

Model	Params / M	Rel. Latency	Short-Form WER	Long-Form WER
whisper-large-v2	1550	1.0	9.1	11.7

distil-large-v2	756	5.8	10.1	11.6
distil-medium.en	394	6.8	11.1	12.4

Acknowledgements

OpenAI for the Whisper model and original codebase
Hugging Face 🤗 Transformers for the model integration
Google’s TPU Research Cloud (TRC) programme for Cloud TPU v4s

Citation

If you use this model, please consider citing the Distil-Whisper paper:

@misc{gandhi2023distilwhisper,
      title={Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling}, 
      author={Sanchit Gandhi and Patrick von Platen and Alexander M. Rush},
      year={2023},
      eprint={2311.00430},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

And also the Whisper paper:

@misc{radford2022robust,
      title={Robust Speech Recognition via Large-Scale Weak Supervision}, 
      author={Alec Radford and Jong Wook Kim and Tao Xu and Greg Brockman and Christine McLeavey and Ilya Sutskever},
      year={2022},
      eprint={2212.04356},
      archivePrefix={arXiv},
      primaryClass={eess.AS}
}