Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A Tensorflow implementation of QANet for machine reading comprehension
| Date | Stars |
|---|---|
| 2026-07-24 | 984 |
| 2026-07-25 | 984 |
| 2026-07-28 | 984 |
| 2026-07-30 | 984 |
| 2026-08-08 | 985 |
| 2026-08-30 | 985 |
| 2026-09-01 | 985 |
| 2026-09-02 | 985 |
| 2026-09-20 | 985 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# QANet
A Tensorflow implementation of Google's [QANet](https://openreview.net/pdf?id=B14TlG-RW) (previously Fast Reading Comprehension (FRC)) from [ICLR2018](https://openreview.net/forum?id=B14TlG-RW). (Note: This is not an official implementation from the authors of the paper)
I wrote a blog post about implementing QANet. Check out [here](https://medium.com/@minsangkim/implementing-question-answering-networks-with-cnns-5ae5f08e312b) for more information!
Training and preprocessing pipeline have been adopted from [R-Net by HKUST-KnowComp](https://github.com/HKUST-KnowComp/R-Net). Demo mode is working. After training, just use `python config.py --mode demo` to run an interactive demo server.
Due to a memory issue, a single head dot-product attention is used as opposed to a 8 heads multi-head attention like in the original paper. The hidden size is also reduced to 96 from 128 due to usage of a GTX1080 compared to a P100 used in the paper. (8GB of GPU memory is insufficient. If you have a 12GB memory GPU please share your training results with us.)
Currently, the best model reaches EM/F1 = 70.8/80.1 in 60k steps (6~8 hours). Detailed results are listed below.
## Dataset
The dataset used for this task is [Stanford Question Answering Dataset](https://rajpurkar.github.io/SQuAD-explorer/).
Pretrained [GloVe embeddings](https://nlp.stanford.edu/projects/glove/) obtained from common crawl with 840B tokens used for words.
## Requirements
* Python>=2.7
* NumPy
* tqdm
* TensorFlow>=1.5
* spacy==2.0.9
* bottle (only for demo)
## Usage
To download and preprocess the data, run
```bash
# download SQuAD and Glove
sh download.sh
# preprocess the data
python config.py --mode prepro
```
Just like [R-Net by HKUST-KnowComp](https://github.com/HKUST-KnowComp/R-Net), hyper parameters are stored in config.py. To debug/train/test/demo, run
```bash
python config.py --mode debug/train/test/demo
```
To evaluate the model with the official code, run
```bash
python evaluate-v1.1.py ~/data/squad/dev-v1.1.json train/{model_name}/answer/answer.json
```
The default directory for the tensorboard log file is `train/{model_name}/event`
### Run in Docker container (optional)
To build the Docker image (requires nvidia-docker), run
```
nvidia-docker build -t tensorflow/qanet .
```
Set volume mount paths and port mappings (for demo mode)
```
export QANETPATH={/path/to/cloned/QANet}
export CONTAINERWORKDIR=/home/QANet
export HOSTPORT=8080
export CONTAINERPORT=8080
```
bash into the container
```
nvidia-docker run -v $QANETPATH:$CONTAINERWORKDIR -p $HOSTPORT:$CONTAINERPORT -it --rm tensorflow/qanet bash
```
Once inside the container, follow the commands provided above starting with downloading the SQuAD and Glove datasets.
### Pretrained Model
Pretrained model weights are temporarily not available.
## Detailed Implementaion
* The model adopts character level convolution - max pooling - highway network for input representations similar to [this paper by Yoon Kim](https://arxiv.org/pdf/1508.06615.pdf).
* The encoder consists of positional encoding - depthwise separable convolution - self attention - feed forward structure with layer norm in between.
* Despite the original paper using 200, we observe that using a smaller character dimension leads to better generalization.
* For regularization, a dropout of 0.1 is used every 2 sub-layers and 2 blocks.
* Stochastic depth dropout is used to drop the residual connection with respect to increasing depth of the network as this model heavily relies on residual connections.
* Query-to-Context attention is used along with Context-to-Query attention, which seems to improve the performance more than what the paper reported. This may be due to the lack of diversity in self attention due to 1 head (as opposed to 8 heads) which may have repetitive information that the query-to-context attention contains.
* Learning Excerpt of 5,851 characters
Read on GitHub
Min · Radiant AI · United Kingdom
29
2
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:97365971b6605407, topic:tensorflow
matched fp:97365971b6605407, topic:nlp