Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, information extraction (i.e., entity, relation and event extraction), knowledge graph, text generation, network embedding
| Date | Stars |
|---|---|
| 2026-07-24 | 577 |
| 2026-07-25 | 577 |
| 2026-07-28 | 577 |
| 2026-07-30 | 577 |
| 2026-07-31 | 578 |
| 2026-08-06 | 578 |
Today
— stars today
This week
+1 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.17%/day
# NLP-Projects
Natural Language Processing projects, which includes concepts and scripts about:
- [0_Word2vec](https://github.com/gaoisbest/NLP-Projects/blob/master/0_Word2vec/README.md)
- `gensim`, `fastText` and `tensorflow` implementations. See [Chinese notes](http://url.cn/5PKmy7W), [中文解读](http://url.cn/5PKmy7W)
- [1_Sentence2vec](https://github.com/gaoisbest/NLP-Projects/blob/master/1_Sentence2vec/README.md)
- `doc2vec`, `word2vec averaging` and `Smooth Inverse Frequency` implementations
- [2_Machine_reading_comprehension](https://github.com/gaoisbest/NLP-Projects/blob/master/2_Machine_reading_comprehension/README.md)
- [3_Dialog_system](https://github.com/gaoisbest/NLP-Projects/blob/master/3_Dialog_system/README.md)
- Categories and components of dialog system
- [4_Text_classification](https://github.com/gaoisbest/NLP-Projects/blob/master/4_Text_classification/README.md)
- `tensorflow LSTM` (See [Chinese notes 1](http://url.cn/5cLDOQI), [中文解读 1](http://url.cn/5cLDOQI) and [Chinese notes 2](http://url.cn/5w5VbaI), [中文解读 2](http://url.cn/5w5VbaI))
- `fastText` implementation
- [5_Pretraining_LM](https://github.com/gaoisbest/NLP-Projects/blob/master/5_Pretraining_LM/README.md)
- Principle of ELMo, ULMFit, GPT, BERT, XLNet
- [6_Sequence_labeling](https://github.com/gaoisbest/NLP-Projects/blob/master/6_Sequence_labeling/README.md)
- [Chinese_word_segmentation](https://github.com/gaoisbest/NLP-Projects/blob/master/6_Sequence_labeling/Chinese_word_segmentation/README.md)
- `HMM Viterbi` implementations. See [Chinese notes](http://url.cn/5x4KR8u), [中文解读](http://url.cn/5x4KR8u)
- [Named_Entity_Recognition](https://github.com/gaoisbest/NLP-Projects/tree/master/6_Sequence_labeling/Named_Entity_Recognition)
- Brands NER via bi-directional LSTM + CRF, `tensorflow` implementation. See [Chinese notes](http://url.cn/5fcC754), [中文解读](http://url.cn/5fcC754)
- [7_Information_retrieval](https://github.com/gaoisbest/NLP-Projects/blob/master/7_Information_retrieval/README.md)
- [8_Information_extraction](https://github.com/gaoisbest/NLP-Projects/blob/master/8_Information_extraction/README.md)
- [9_Knowledge_graph](https://github.com/gaoisbest/NLP-Projects/blob/master/9_Knowledge_graph/README.md)
- [10_Text_generation](https://github.com/gaoisbest/NLP-Projects/blob/master/10_Text_generation/README.md)
- [11_Network_embedding](https://github.com/gaoisbest/NLP-Projects/blob/master/11_Network_embedding/README.md)
# Concepts
### 1. Attention
- Attention == **weighted averages**
- The attention [review 1](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html) and [review 2](https://zhuanlan.zhihu.com/p/31547842) summarize attention mechanism into several types:
- Additive vs Multiplicative attention
- Self attention
- Soft vs Hard attention
- Global vs Local attention
### 2. CNNs, RNNs and Transformer
- **Parallelization** [1]
- RNNs
- Why not good ?
- **Last step's output is input of current step**
- Solutions
- **Simple Recurrent Units (SRU)**
- Perform parallelization on each hidden state neuron independently
- **Sliced RNNs**
- Separate sequences into windows, use RNNs in each window, use another RNNs above windows
- Same as CNNs
- CNNs
- Why good ?
- For different windows in one filter
- For different filters
- **Long-range dependency** [1]
- CNNs
- Why not good ?
- Single convolution can only caputure window-range dependency
- [Solutions](http://web.stanford.edu/class/cs224n/slides/cs224n-2019-lecture11-convnets.pdf)
- Dilated CNNs
- Deep CNNs
- `N * [Convolution + skip-connection]`
- For example, window size=3 and sliding step=1, second convolution can cover 5 words (i.e., 1-2-3, 2-3-4, 3-4-5)
- Transformer > RNNs > CNNs
- **Position** [1]
- CNNs
- Why not good ?
- Convolution preserves **relative-ordeExcerpt of 6,634 characters
Read on GitHub
575
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:99bf9463a8c5ccf6, topic:text-classification, topic:information-extraction, readme:natural language processing
matched fp:99bf9463a8c5ccf6, topic:knowledge-graph, desc:knowledge graph, readme:knowledge graph