Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine
| Date | Stars |
|---|---|
| 2026-07-24 | 577 |
| 2026-07-25 | 577 |
| 2026-07-28 | 577 |
| 2026-07-30 | 577 |
| 2026-08-06 | 577 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<h1 align="center"> WhisperS2T ⚡ </h1>
<p align="center"><b>An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine!</b></p>
<p align="center">
<a href="https://www.pepy.tech/projects/whisper-s2t">
<img alt="Downloads" src="https://static.pepy.tech/personalized-badge/whisper-s2t?period=total&units=international_system&left_color=grey&right_color=brightgreen&left_text=downloads" />
</a>
<a href="https://pepy.tech/project/whisper-s2t">
<img alt="GitHub Contributors" src="https://img.shields.io/github/contributors/shashikg/WhisperS2T" />
</a>
<a href="https://badge.fury.io/py/whisper-s2t">
<img alt="PyPi Release Version" src="https://badge.fury.io/py/whisper-s2t.svg" />
</a>
<a href="https://github.com/shashikg/WhisperS2T/issues">
<img alt="Issues" src="https://img.shields.io/github/issues/shashikg/WhisperS2T?color=0088ff" />
</a>
</p>
<hr><br>
WhisperS2T is an optimized lightning-fast open-sourced **Speech-to-Text** (ASR) pipeline. It is tailored for the whisper model to provide faster whisper transcription. It's designed to be exceptionally fast than other implementation, boasting a **2.3X speed improvement over [WhisperX](https://github.com/m-bain/whisperX/tree/main) and a 3X speed boost compared to [HuggingFace Pipeline](https://huggingface.co/openai/whisper-large-v2) with FlashAttention 2 ([Insanely Fast Whisper](https://github.com/Vaibhavs10/insanely-fast-whisper))**. Moreover, it includes several heuristics to enhance transcription accuracy.
[**Whisper**](https://github.com/openai/whisper) is a general-purpose speech recognition model developed by OpenAI and not me. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.
## Release Notes
* [Feb 25, 2024]: Added prebuilt docker images and transcript exporter to `txt, json, tsv, srt, vtt`. (Check complete [release note](https://github.com/shashikg/WhisperS2T/releases/tag/v1.3.1))
* [Jan 28, 2024]: Added support for TensorRT-LLM backend.
* [Dec 23, 2023]: Added support for word alignment for CTranslate2 backend (check [benchmark](https://github.com/shashikg/WhisperS2T/releases/tag/v1.2.0)).
* [Dec 19, 2023]: Added support for Whisper-Large-V3 and Distil-Whisper-Large-V2 (check [benchmark](https://github.com/shashikg/WhisperS2T/releases/tag/v1.1.0)).
* [Dec 17, 2023]: Released WhisperS2T!
## Quickstart
Checkout the Google Colab notebooks provided here: [notebooks](notebooks)
## Future Roadmaps
- [x] Ready to use docker container.
- [ ] WhisperS2T-Server: Optimized end-to-end deployment ready server codebase.
- [ ] In depth documentation, use github pages to host it.
- [ ] Explore possibility of integrating Meta's SeamlessM4T model.
- [ ] Add more datasets for WER benchmarking.
## Benchmark and Technical Report
Stay tuned for a technical report comparing WhisperS2T against other whisper pipelines. Meanwhile, check some quick benchmarks on A30 GPU. See `scripts/` directory for the benchmarking scripts that I used.
**NOTE:** I conducted all the benchmarks using the `without_timestamps` parameter set as `True`. Adjusting this parameter to `False` may enhance the Word Error Rate (WER) of the HuggingFace pipeline but at the expense of increased inference time. Notably, the improvements in inference speed were achieved solely through a **superior pipeline design**, without any specific optimization made to the backend inference engines (such as CTranslate2, FlashAttention2, etc.). For instance, WhisperS2T (utilizing FlashAttention2) demonstrates significantly superior inference speed compared to the HuggingFace pipeline (also using FlashAttention2), despite both leveraging the same inference engine—HuggingFace whisper model with FlashAtExcerpt of 10,894 characters
Read on GitHub
79
Mahmoud Ashraf · @TarteelAI
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:6516077cc8a05e2d, topic:speech-recognition, topic:asr, topic:whisper