Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
speech to text with self-supervised learning based on wav2vec 2.0 framework
| Date | Stars |
|---|---|
| 2026-07-24 | 380 |
| 2026-07-25 | 380 |
| 2026-07-28 | 380 |
| 2026-07-30 | 380 |
| 2026-07-31 | 380 |
| 2026-08-06 | 380 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
## Self-supervised speech recognition with limited amount of labeled data **Note**: As I no longer maintain the repo, If you encounter any problems, please take a look at [similar reported issues](https://github.com/pytorch/fairseq/search?q=wav2vec+2.0&type=issues) from fairseq repo. This is a wrapper version of [wav2vec 2.0 framework](https://github.com/pytorch/fairseq/tree/master/examples/wav2vec), which attempts to build an accurate speech recognition models with small amount of transcribed data (eg. 1 hour) Transfer learning is still the main technique: - Transfer from self-supervised models (pretrain on unlabeled data) - Transfer from multilingual models (pretrain on multilingual data) ## Required resources #### 1. Labeled data, which is pairs of (audio, transcript) The more you have, the better the model is. Prepare at least 1 hour if you have a large amount of unlabeled data. Otherwise, at least 50 hours is recommended. #### 2. Text data for building language models. This should includes both well-written text and conversational text, which can easily collected from news/forums websties. At least 1 GB of text is recommended. #### 3. Unlabeled data (audios without transcriptions) of your own language. This is optional but very crucial. A good amount of unlabeled audios (eg. 500 hours) will significantly reduce the amount of labeled data needed, and also boost up the model performance. Youtube/Podcast is a great place to collect the data for your own language ## Install instruction Please follow this [instruction](https://github.com/mailong25/self-supervised-speech-recognition/blob/master/Dependencies.md) ## Steps to build an accurate speech recognition model for your language ### 1. Train a self-supervised model on unlabeled data (Pretrain) #### 1.1 Prepare unlabeled audios Collect unlabel audios and put them all together in a single directory. Audio format requirements:\ Format: wav, PCM 16 bit, single channel\ Sampling_rate: 16000\ Length: 5 to 30 seconds\ Content: silence should be removed from the audio. Also, each audio should contain only one person speaking.\ Please look at examples/unlabel_audio directory for reference. #### 1.2 Download an initial model Instead of training from scratch, we download and use english wav2vec model for weight initialization. This pratice can be apply to all languages. ``` wget https://dl.fbaipublicfiles.com/fairseq/wav2vec/wav2vec_small.pt ``` #### 1.3 Run Pre-training ``` python3 pretrain.py --fairseq_path path/to/libs/fairseq --audio_path path/to/audio_directory --init_model path/to/wav2vec_small.pt ``` Where: - fairseq_path: path to installed fairseq library, after install [instruction](https://github.com/mailong25/self-supervised-speech-recognition/blob/master/Dependencies.md) - audio_path: path to unlabel audio directory - init_model: downloaded model from step 1.2 Logs and checkpoints will be stored at outputs directory\ Log_file path: outputs/date_time/exp_id/hydra_train.log. You should check the loss value to decide when to stop the training process.\ Best_checkpoint path: outputs/date_time/exp_id/checkpoints/checkpoint_best.pt\ In my casse, it took ~ 4 days for the model to converge, train on 100 hours of data using 2 NVIDIA Tesla V100. ### 2. Finetune the self-supervised model on the labeled data #### 2.1 Prepare labeled data -- Transcript file ---\ One trainng sample per line with format "audio_absolute_path \tab transcript"\ Example of a transcript file: ``` /path/to/1.wav AND IT WAS A MATTER OF COURSE THAT IN THE MIDDLE AGES WHEN THE CRAFTSMEN /path/to/2.wav AND WAS IN FACT THE KIND OF LETTER USED IN THE MANY SPLENDID MISSALS PSALTERS PRODUCED BY PRINTING IN THE FIFTEENTH CENTURY /path/to/3.wav JOHN OF SPIRES AND HIS BROTHER VINDELIN FOLLOWED BY NICHOLAS JENSON BEGAN TO PRINT IN THAT CITY /path/to/4.wav BEING THIN TOUGH AND OPAQUE ``` Some notes on transcript file: - One sample per line - Upper case - All numbers should be transformed into verb
Excerpt of 8,372 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:346cfff420cdec91, topic:speech-recognition, topic:speech-to-text, name:speech recognition