Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
| Date | Stars |
|---|---|
| 2026-07-24 | 2829 |
| 2026-07-25 | 2829 |
| 2026-07-28 | 2831 |
| 2026-07-30 | 2832 |
| 2026-08-06 | 2832 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# whisper-timestamped
Multilingual Automatic Speech Recognition with word-level timestamps and confidence.
* [Description](#description)
* [Notes on other approaches](#notes-on-other-approaches)
* [Installation](#installation)
* [First installation](#first-installation)
* [Additional packages that might be needed](#additional-packages-that-might-be-needed)
* [Docker](#docker)
* [Light installation for CPU](#light-installation-for-cpu)
* [Upgrade to the latest version](#upgrade-to-the-latest-version)
* [Usage](#usage)
* [Python](#python)
* [Command line](#command-line)
* [Utility Functions](#utility-functions)
* [Plot of word alignment](#plot-of-word-alignment)
* [Example output](#example-output)
* [API Reference](#api-reference)
* [Main Transcription Function](#main-transcription-function)
* [Utility Functions](#utility-functions-1)
* [File Writing Functions](#file-writing-functions)
* [Options that may improve results](#options-that-may-improve-results)
* [Accurate Whisper transcription](#accurate-whisper-transcription)
* [Running Voice Activity Detection (VAD) before sending to Whisper](#running-voice-activity-detection-vad-before-sending-to-whisper)
* [Detecting disfluencies](#detecting-disfluencies)
* [Acknowledgments/Support](#acknowledgments-support)
* [Citations](#citations)
## Description
[Whisper](https://openai.com/blog/whisper/) is a set of multi-lingual, robust speech recognition models trained by OpenAI that achieve state-of-the-art results in many languages. Whisper models were trained to predict approximate timestamps on speech segments (most of the time with 1-second accuracy), but they cannot originally predict word timestamps. This repository proposes an implementation to **predict word timestamps and provide a more accurate estimation of speech segments when transcribing with Whisper models**.
Besides, a confidence score is assigned to each word and each segment.
The approach is based on Dynamic Time Warping (DTW) applied to cross-attention weights, as demonstrated by [this notebook by Jong Wook Kim](https://github.com/openai/whisper/blob/f82bc59f5ea234d4b97fb2860842ed38519f7e65/notebooks/Multilingual_ASR.ipynb). There are some additions to this notebook:
* The start/end estimation is more accurate.
* Confidence scores are assigned to each word.
* **If possible (without beam search...)**, no additional inference steps are required to predict word timestamps (word alignment is done on the fly after each speech segment is decoded).
* Special care has been taken regarding memory usage: `whisper-timestamped` is able to process long files with little additional memory compared to the regular use of the Whisper model.
`whisper-timestamped` is an extension of the [`openai-whisper`](https://pypi.org/project/whisper-openai/) Python package and is meant to be compatible with any version of `openai-whisper`.
It provides more efficient/accurate word timestamps, along with those additional features:
* Voice Activity Detection (VAD) can be run before applying Whisper model,
to avoid hallucinations due to errors in the training data (for instance, predicting "Thanks you for watching!" on pure silence).
Several VAD methods are available: silero (default), auditok, auditok:v3.1
* When the language is not specified, the language probabilities are provided among the outputs.
**Disclaimer: Please note that this extension is intended for experimental purposes and may significantly impact performance. We are not responsible for any issues or inefficiencies that arise from its use.**
### Notes on other approaches
An alternative relevant approach to recovering word-level timestamps involves using wav2vec models that predict characters, as successfully implemented in [whisperX](https://github.com/m-bain/whisperX). However, these approaches have several drawbacks that are not present in approaches based on cross-attention weights such as `whisper_timestamped`. ThesExcerpt of 33,018 characters
Read on GitHub
247
5
2
2
1
1
Simon
1
Ville Saukkonen · Finland
1
Shahrukh Khan · Yoummday GmbH · Germany
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:d57cade3d531f8bd, topic:speech-recognition, topic:asr, topic:whisper