Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Probing the representations of Vision Transformers.
| Date | Stars |
|---|---|
| 2026-07-24 | 341 |
| 2026-07-25 | 341 |
| 2026-07-28 | 341 |
| 2026-07-30 | 341 |
| 2026-08-06 | 341 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Probing ViTs [](https://github.com/tensorflow/tensorflow/releases/tag/v2.8.0) [](https://huggingface.co/spaces) _By [Aritra Roy Gosthipaty](https://github.com/ariG23498) and [Sayak Paul](https://github.com/sayakpaul) (equal contribution)_ In this repository, we provide tools to probe into the representations learned by different families of Vision Transformers (supervised pre-training with ImageNet-21k, ImageNet-1k, distillation, self-supervised pre-training): * Original ViT [1] * DeiT [2] * DINO [3] We hope these tools will prove to be useful for the community. Please follow along with [this post on keras.io](https://keras.io/examples/vision/probing_vits/) for a better navigation through the repository. **Updates** * June 3, 2022: The project got the [Google OSS Expert Prize](https://www.kaggle.com/discussions/general/328914). * May 10, 2022: The project got a mention from Yannic Kilcher in [ML News](https://youtu.be/pwSnC8jlh50?t=712). Thanks, Yannic! * May 4, 2022: We're glad to receive the [#TFCommunitySpotlight award](https://twitter.com/TensorFlow/status/1521558632768409600?s=20&t=hXgrZOfT_26AuTC_RyCZ_g) for this project. ## Self-attention visualization | Original Image | Attention Maps | Attention Maps Overlayed | | :--: | :--: | :--: | |  |  |  | https://user-images.githubusercontent.com/36856589/162609884-8e51156e-d461-421d-9f8a-4d4e48967bd6.mp4 <small><a href=https://www.pexels.com/video/a-computer-generated-walking-dinosaur-4096297/>Original Video Source</a></small> https://user-images.githubusercontent.com/36856589/162609907-4e432dc4-a731-40f4-9a20-94e0c8f648bc.mp4 <small><a href=https://www.pexels.com/video/a-dog-running-in-a-grass-field-4166343/>Original Video Source</a></small> ## Supervised salient representations In the [DINO](https://ai.facebook.com/blog/dino-paws-computer-vision-with-self-supervised-transformers-and-10x-more-efficient-training/) blog post, the authors show a video with the following caption: > The original video is shown on the left. In the middle is a segmentation example generated by a supervised model, and on the right is one generated by DINO. A screenshot of the video is as follows: <img width="764" alt="image" src="https://user-images.githubusercontent.com/36856589/162615199-b5133e51-460e-4864-a83e-5b8007339ff7.png"><br> We obtain the attention maps generated with the supervised pre-trained model and find that they are not that salient w.r.t the DINO model. We observe a similar behaviour in our experiments as well. The figure below shows the attention heatmaps extracted with a ViT-B16 model pre-trained (supervised) using ImageNet-1k: | Dinosaur | Dog | | :--: | :--: | |  |  | We used this [Colab Notebook](https://github.com/sayakpaul/probing-vits/blob/main/notebooks/vitb16-attention-maps-video.ipynb) to conduct this experiment. ## Hugging Face Spaces You can now probe into the ViTs with your own input images. | Attention Heat Maps | Attention Rollout | | :--: | :--: | | [](https://huggingface.co/spaces/probing-vits/attention-heat-maps) | [](https://huggingface.co/spaces/probing-vits/attention-rollout) | ## Visualizing mean attention distances <div align="center"> <img src="./assets/vit_base_i21k_patch16_224.png" width=450/> </div> ## Methods **We don't propose any novel methods of probing the representations of neural networks. Instead we take the existing works and implement them in Tensor
Excerpt of 8,434 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:82c5e6d9bd988024, topic:tensorflow, readme:pre-training