Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Deep Modular Co-Attention Networks for Visual Question Answering
| Date | Stars |
|---|---|
| 2026-07-24 | 459 |
| 2026-07-25 | 459 |
| 2026-07-28 | 459 |
| 2026-07-30 | 459 |
| 2026-08-06 | 459 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Deep Modular Co-Attention Networks (MCAN) This repository corresponds to the **PyTorch** implementation of the MCAN for VQA, which won the champion in [VQA Challgen 2019](https://visualqa.org/roe.html). With an ensemble of 27 models, we achieved an overall accuracy **75.23%** and **75.26%** on *test-std* and *test-challenge* splits, respectively. See our [slides](https://drive.google.com/open?id=1geJTEVMMGs9Y3S2_pZxNzgPMccfYpM_r) for details. By using the commonly used bottom-up-attention visual features, a single MCAN model delivers **70.70%** (small model) and **70.93%** (large model) overall accuracy on the *test-dev* split of *VQA-v2* dataset respectively, which significantly outperform existing state-of-the-arts. Please check our [paper](http://openaccess.thecvf.com/content_CVPR_2019/html/Yu_Deep_Modular_Co-Attention_Networks_for_Visual_Question_Answering_CVPR_2019_paper.html) for details.  ## Updates July 10, 2019 - **Pytorch implementation of MCAN along with several state-of-the-art models on VQA/GQA/CLEVR are maintained in our another [OpenVQA](https://github.com/MILVLG/openvqa) project.** June 13, 2019 - Pure PyTorch implementation of MCAN model with deep encoder-decoder strategy. - Self-contained documentation from scratch . - Model zoo consists of pre-trained MCAN-small and MCAN-large models on the VQA-v2 dataset. - Multi-GPUs training and gradient accumulation. ## Table of Contents 0. [Prerequisites](#Prerequisites) 0. [Training](#Training) 0. [Validation and Testing](#Validation-and-Testing) 0. [Pretrained models](#Pretrained-models) 0. [Citation](#Citation) ## Prerequisites #### Software and Hardware Requirements You may need a machine with at least **1 GPU (>= 8GB)**, **20GB memory** and **50GB free disk space**. We strongly recommend to use a SSD drive to guarantee high-speed I/O. You should first install some necessary packages. 1. Install [Python](https://www.python.org/downloads/) >= 3.5 2. Install [Cuda](https://developer.nvidia.com/cuda-toolkit) >= 9.0 and [cuDNN](https://developer.nvidia.com/cudnn) 3. Install [PyTorch](http://pytorch.org/) >= 0.4.1 with CUDA (**Pytorch 1.x is also supported**). 4. Install [SpaCy](https://spacy.io/) and initialize the [GloVe](https://github.com/explosion/spacy-models/releases/download/en_vectors_web_lg-2.1.0/en_vectors_web_lg-2.1.0.tar.gz) as follows: ```bash $ pip install -r requirements.txt $ wget https://github.com/explosion/spacy-models/releases/download/en_vectors_web_lg-2.1.0/en_vectors_web_lg-2.1.0.tar.gz -O en_vectors_web_lg-2.1.0.tar.gz $ pip install en_vectors_web_lg-2.1.0.tar.gz ``` #### Setup The image features are extracted using the [bottom-up-attention](https://github.com/peteanderson80/bottom-up-attention) strategy, with each image being represented as an dynamic number (from 10 to 100) of 2048-D features. We store the features for each image in a `.npz` file. You can prepare the visual features by yourself or download the extracted features from [OneDrive](https://awma1-my.sharepoint.com/:f:/g/personal/yuz_l0_tn/EsfBlbmK1QZFhCOFpr4c5HUBzUV0aH2h1McnPG1jWAxytQ?e=2BZl8O) or [BaiduYun](https://pan.baidu.com/s/1C7jIWgM3hFPv-YXJexItgw#list/path=%2F). The downloaded files contains three files: **train2014.tar.gz, val2014.tar.gz, and test2015.tar.gz**, corresponding to the features of the train/val/test images for *VQA-v2*, respectively. You should place them as follows: ```angular2html |-- datasets |-- coco_extract | |-- train2014.tar.gz | |-- val2014.tar.gz | |-- test2015.tar.gz ``` Besides, we use the VQA samples from the [visual genome dataset](http://visualgenome.org/) to expand the training samples. Similar to existing strategies, we preprocessed the samples by two rules: 1. Select the QA pairs with the corresponding images appear in the MSCOCO train and *val* splits. 2. Select the QA pairs with the answer appear in the processed answer list (occurs more than 8 times in whole
Excerpt of 10,596 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:92779c3cd7d667fd, topic:visual-question-answering, desc:visual question answering, readme:visual question answering