Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.
| Date | Stars |
|---|---|
| 2026-07-24 | 911 |
| 2026-07-25 | 911 |
| 2026-07-28 | 911 |
| 2026-07-30 | 911 |
| 2026-08-07 | 912 |
| 2026-08-15 | 911 |
| 2026-08-18 | 912 |
| 2026-08-25 | 913 |
| 2026-09-15 | 913 |
| 2026-09-20 | 913 |
Today
— stars today
This week
— stars this week
This month
+1 stars this month
Momentum
0.0
growth rate 0.00%/day
[ICCV 2021- Oral] PyTorch Implementation of `Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers <https://arxiv.org/abs/2103.15679>`_
=================================================================================================================================================================
|youtube|
.. |youtube| image:: https://img.shields.io/static/v1?label=ICCV2021&message=12MinuteVideo&color=red
:target: https://www.youtube.com/watch?v=bQTL34Dln-M
Notebooks for LXMERT + DETR:
----------------------------
|DETR_LXMERT|
.. |DETR_LXMERT| image:: https://colab.research.google.com/assets/colab-badge.svg
:target: https://colab.research.google.com/github/hila-chefer/Transformer-MM-Explainability/blob/main/Transformer_MM_Explainability.ipynb
Notebook for CLIP:
----------------------------
|CLIP|
.. |CLIP| image:: https://colab.research.google.com/assets/colab-badge.svg
:target: https://colab.research.google.com/github/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP_explainability.ipynb
**Demo**: You can check out a demo on `Huggingface spaces <https://huggingface.co/spaces/PaulHilders/CLIPGroundingExplainability>`_ or scan the following QR code.
.. image:: https://user-images.githubusercontent.com/19412343/176676771-d26f2146-9901-49e7-99be-b030f3d790de.png
:width: 100
Notebook for ViT:
----------------------------
|ViT|
.. |ViT| image:: https://colab.research.google.com/assets/colab-badge.svg
:target: https://colab.research.google.com/github/hila-chefer/Transformer-MM-Explainability/blob/main/Transformer_MM_explainability_ViT.ipynb
.. sectnum::
Using Colab
----------------
* Please notice that the notebook assumes that you are using a GPU. To switch runtime go to Runtime -> change runtime type and select GPU.
* Installing all the requirements may take some time. After installation, please restart the runtime.
Running Examples
----------------
Notice that we have two `jupyter` notebooks to run the examples presented in the paper.
* `The notebook for LXMERT <./LXMERT.ipynb>`_ contains both the examples from the paper and examples with images from the internet and free form questions.
To use your own input, simply change the `URL` variable to your image and the `question` variable to your free form question.
.. image:: LXMERT.PNG
.. image:: LXMERT-web.PNG
* `The notebook for DETR <./DETR.ipynb>`_ contains the examples from the paper.
To use your own input, simply change the `URL` variable to your image.
.. image:: DETR.PNG
Reproduction of results
-----------------------
^^^^^^^^^^
VisualBERT
^^^^^^^^^^
Run the `run.py` script as follows:
.. code-block:: bash
CUDA_VISIBLE_DEVICES=0 PYTHONPATH=`pwd` python VisualBERT/run.py --method=<method_name> --is-text-pert=<true/false> --is-positive-pert=<true/false> --num-samples=10000 config=projects/visual_bert/configs/vqa2/defaults.yaml model=visual_bert dataset=vqa2 run_type=val checkpoint.resume_zoo=visual_bert.finetuned.vqa2.from_coco_train env.data_dir=/path/to/data_dir training.num_workers=0 training.batch_size=1 training.trainer=mmf_pert training.seed=1234
.. note::
If the datasets aren't already in `env.data_dir`, then the script will download the data automatically to the path in `env.data_dir`.
^^^^^^
LXMERT
^^^^^^
#. Download `valid.json <https://nlp.cs.unc.edu/data/lxmert_data/vqa/valid.json>`_:
.. code-block:: bash
pushd data/vqa
wget https://nlp.cs.unc.edu/data/lxmert_data/vqa/valid.json
popd
#. Download the `COCO_val2014` set to your local machine.
.. note::
If you already downloaded `COCO_val2014` for the `VisualBERT`_ tests, you can simply use the same path you used for `VisualBERT`_.
#. Run the `perturbation.py` script as follows:
.. code-block:: bash
CUDA_VISIBLE_DEVICES=0 PYTHONPATH=`pwd` python lxmert/lxmert/perturbation.py --COCO_path /patExcerpt of 6,750 characters
Read on GitHub
70
4
2
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:21e526534e728433, topic:interpretability, topic:explainable-ai
matched fp:21e526534e728433, topic:transformer
matched fp:21e526534e728433, topic:clip