Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
How to use OpenAIs Whisper to transcribe and diarize audio files
| Date | Stars |
|---|---|
| 2026-07-24 | 377 |
| 2026-07-25 | 377 |
| 2026-07-28 | 377 |
| 2026-07-30 | 377 |
| 2026-08-06 | 377 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Whisper transcription and diarization (speaker-identification)
How to use OpenAIs Whisper to transcribe and diarize audio files
## What is Whisper?
Whisper is an State-of-the-Art speech recognition system from OpenAI that has been trained on 680,000 hours
of multilingual and multitask supervised data collected from the web. This large and diverse
dataset leads to improved robustness to accents, background noise and technical language. In
addition, it enables transcription in multiple languages, as well as translation from those
languages into English. OpenAI released the models and code to serve as a foundation for building useful
applications that leverage speech recognition.
One big downside of Whisper is though, that it can not tell you who is speaking in a conversation.
That's a problem when analyzing conversations. This is where diarization comes in. Diarization is
the process of identifying who is speaking in a conversation.
In this tutorial you will learn how to identify the speakers, and then match them with the transcriptions of Whisper.
We will use `pyannote-audio` to accomplish this. Let's get started!
### Preparing the audio
First, we need to prepare the audio file. We will use the first 20 minutes of Lex Fridmans podcast with Yann download.
To download the video and extract the audio, we will use `yt-dlp` package.
```bash
!pip install -U yt-dlp
```
We will also need [ffmpeg](https://www.wikihow.com/Install-FFmpeg-on-Windows) installed
```bash
!wget -O - -q https://github.com/yt-dlp/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz | xz -qdc| tar -x
```
Now we can do the actual download and audio extraction via the command line.
```bash
!yt-dlp -xv --ffmpeg-location ffmpeg-master-latest-linux64-gpl/bin --audio-format wav -o download.wav -- https://youtu.be/SGzMElJ11Cc
```
Now we have the `download.wav` file in our working directory. Let's cut the first 20 minutes of the audio. We can use the pydub package for this with just a few lines of code.
```bash
!pip install pydub
```
```python
from pydub import AudioSegment
t1 = 0 * 1000 # works in milliseconds
t2 = 20 * 60 * 1000
newAudio = AudioSegment.from_wav("download.wav")
a = newAudio[t1:t2]
a.export("audio.wav", format="wav")
```
`audio.wav` is now the first 20 minutes of the audio file.
### Pyannote's Diarization
`pyannote.audio` is an open-source toolkit written in Python for speaker diarization. Based on PyTorch
machine learning framework, it provides a set of trainable end-to-end neural building blocks that
can be combined and jointly optimized to build speaker diarization pipelines. `pyannote.audio` also
comes with pretrained models and pipelines covering a wide range of domains for voice activity
detection, speaker segmentation, overlapped speech detection, speaker embedding reaching
state-of-the-art performance for most of them.
Installing Pyannote and running it on the video audio to generate the diarizations.
```bash
!pip install pyannote.audio
```
```python
from pyannote.audio import Pipeline
pipeline = Pipeline.from_pretrained('pyannote/speaker-diarization')
```
```python
DEMO_FILE = {'uri': 'blabal', 'audio': 'audio.wav'}
dz = pipeline(DEMO_FILE)
with open("diarization.txt", "w") as text_file:
text_file.write(str(dz))
```
Lets print this out to see what it looks like.
```pyhton
print(*list(dz.itertracks(yield_label = True))[:10], sep="\n")
```
The output:
```
(<Segment(2.03344, 36.8128)>, 0, 'SPEAKER_00')
(<Segment(38.1122, 51.3759)>, 0, 'SPEAKER_00')
(<Segment(51.8653, 90.2053)>, 1, 'SPEAKER_01')
(<Segment(91.2853, 92.9391)>, 1, 'SPEAKER_01')
(<Segment(94.8628, 116.497)>, 0, 'SPEAKER_00')
(<Segment(116.497, 124.124)>, 1, 'SPEAKER_01')
(<Segment(124.192, 151.597)>, 1, 'SPEAKER_01')
(<Segment(152.018, 179.12)>, 1, 'SPEAKER_01')
(<Segment(180.318, 194.037)>, 1, 'SPEAKER_01')
(<Segment(195.016, 207.385)>, 0, 'SPEAKER_00')
```
This looks pretty good already, but let's clean the daExcerpt of 11,446 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:ab30dc276358608f, topic:whisper, readme:speech recognition, name:transcription