Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
PyTorch implementation of "Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning"
| Date | Stars |
|---|---|
| 2026-07-24 | 347 |
| 2026-07-25 | 347 |
| 2026-07-28 | 347 |
| 2026-07-30 | 347 |
| 2026-08-06 | 347 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Transparency-by-Design networks (TbD-nets)
[](https://mybinder.org/v2/gh/davidmascharka/tbd-nets/binder?filepath=full-vqa-example.ipynb)
This repository contains code for replicating the experiments and visualizations from the paper
[Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual
Reasoning](https://arxiv.org/abs/1803.05268)
>[David Mascharka](https://davidmascharka.com), [Philip Tran](https://github.com/ptran516), [Ryan Soklaski](https://github.com/rsokl), Arjun Majumdar <br>
The paper describes Transparency-by-Design networks (TbD-nets), which are built around a visual
attention mechanism. This repository contains the model architecture put forward in the paper and
code that will allow you to
- [Produce the visualizations from the paper](#recreating-our-visualizations)
- [Ask a natural-language question about an image you provide](#full-vqa)
- [Train a model from scratch on the CLEVR dataset](#training-a-model)
- [Predict answers on the CLEVR test set](#testing-a-model)
A visualization of the output produced by our TbD-net model can be seen below.
<div align="center">
<img src="img/tbd-net.png" width="80%">
</div>
If you find this code useful in your research, please cite
``` tex
@InProceedings{Mascharka_2018_CVPR,
author = {Mascharka, David and Tran, Philip and Soklaski, Ryan and Majumdar, Arjun},
title = {Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}
```
# Full VQA
To ask a natural-language question and provide an image to the model and get an answer and reasoning
chain back, see the [full VQA example notebook](full-vqa-example.ipynb). This will define all the
machinery you need to perform the full VQA task and will allow you to download the necessary models.
[Try it with Binder!](https://mybinder.org/v2/gh/davidmascharka/tbd-nets/binder?filepath=full-vqa-example.ipynb)
# Recreating Our Visualizations
You can [use Binder to use our model without any setup!](https://mybinder.org/v2/gh/davidmascharka/tbd-nets/binder?filepath=visualize-output.ipynb)
To reproduce our work on your local machine, you'll need to clone this repository and [set up
PyTorch](#pytorch). We also recommend using [CUDA and cuDNN](#cudacudnn) if you have a GPU
available.
You can then open up the [visualize-output notebook](visualize-output.ipynb). That will walk you
through running our model and generates all the figures we use in our paper. It will also download
one of our pretrained models. From there, you can play around with the images we provide without
having to download any outside data or models. If you would like to experiment with our other
models, see the [downloading models section](#downloading-models).
# Training a Model
To train a model from scratch, there are a few requirements to take care of. We assume you have
already [set up PyTorch](#pytorch) and [CUDA/cuDNN](#cudacudnn) if you plan on using a GPU (which is
highly recommended).
### 1. Getting data
The CLEVR dataset is available at [its project page](http://cs.stanford.edu/people/jcjohns/clevr/).
The first step for training is to download that data.
You will also need to extract features and process the question files to produce programs before
training a model. The [instructions
here](https://github.com/facebookresearch/clevr-iep/blob/master/TRAINING.md#preprocessing-clevr)
provide a method for this. We recommend cloning that repository and following those instructions.
NOTE: to extract 28x28 features, you will need to add the `--model_stage 2` option to the
`extract_features.py` command. Following the conventions on that page, the commanExcerpt of 11,262 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:8aae0a0617c60257, topic:deep-learning, topic:pytorch
matched fp:8aae0a0617c60257, topic:visual-question-answering