Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[TNNLS] A Comprehensive Survey of Awesome Visual Transformer Literatures.
| Date | Stars |
|---|---|
| 2026-07-24 | 279 |
| 2026-07-25 | 279 |
| 2026-07-28 | 279 |
| 2026-07-30 | 279 |
| 2026-08-06 | 279 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# <p align=center> [A Survey of Visual Transformers](https://ieeexplore.ieee.org/abstract/document/10088164)</p> ##### <p align=center> [Yang Liu](https://scholar.google.com/citations?user=ock4qjYAAAAJ&hl=zh-CN), [Yao Zhang](https://scholar.google.com/citations?user=vxfJSJIAAAAJ&hl=zh-CN), [Yixin Wang](https://scholar.google.com/citations?user=ykYrXtAAAAAJ&hl=zh-CN), [Feng Hou](https://scholar.google.com/citations?user=gp-OCDoAAAAJ&hl=zh-CN), [Jin Yuan](https://scholar.google.com/citations?hl=zh-CN&user=S1JGPCMAAAAJ), [Jiang Tian](https://scholar.google.com/citations?user=CC_HnVQAAAAJ&hl=zh-CN), [Yang Zhang](https://scholar.google.com/citations?user=fwg2QysAAAAJ&hl=zh-CN), [Zhongchao Shi](https://scholar.google.com/citations?hl=zh-CN&user=GASgQxEAAAAJ), [JianPing Fan](https://scholar.google.com/citations?user=-YsOqQcAAAAJ&hl=zh-CN), [Zhiqiang He](https://ieeexplore.ieee.org/author/37085386255)</p>  There is a comprehensive list of awesome visual Transformers literatures corresponding to the original order of our survey ([A Survey of Visual Transformers](https://ieeexplore.ieee.org/abstract/document/10088164)) published in IEEE Transactions on Neural Networks and Learning Systems (TNNLS). We will regularly update the latest representaive literatures and their released source code on this page. If you find some overlooked literatures, please make an issue or contact at [email protected]. # Content - [Original Transformer](#original-transformer) - [Transformer for Classification](#transformer-for-classification) - [Transformer for Detection](#transformer-for-detection) - [Transformer for Segmentation](#transformer-for-segmentation) - [Transformer for 3D Visual Recognition](#transformer-for-3d-visual-recognition) - [Transformer for Multi-Sensory Data Stream](#transformer-for-multi-sensory-data-stream) - [Other Awesome Transformer Attention Model Lists](#more-awesome-transformer-attention-model-lists) # Original Transformer **Attention Is All You Need.** [12th Jun. 2017] [NeurIPS, 2017].<br> *Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin.*<br> [[PDF](https://arxiv.org/abs/1706.03762)] [[Github](https://github.com/tensorflow/tensor2tensor)] # Transformer for Classification ### 1. Original Visual Transformer **Stand-Alone Self-Attention in Vision Models.** [13th Jun. 2019] [NeurIPS, 2019].<br> *Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, Jonathon Shlens.*<br> [[PDF](https://arxiv.org/abs/1906.05909)] [[Github](https://github.com/google-research/google-research)] **On the Relationship between Self-Attention and Convolutional Layers.** [10th Jan. 2020] [ICLR, 2020].<br> *Jean-Baptiste Cordonnier, Andreas Loukas, Martin Jaggi.*<br> [[PDF](https://arxiv.org/abs/1911.03584)] [[Github](https://github.com/epfml/attention-cnn)] **An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.** [10th Mar. 2021] [ICLR, 2021].<br> *Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby.*<br> [[PDF](https://arxiv.org/abs/2010.11929)] [[Github](https://github.com/google-research/vision_transformer)] ### 2. Transformer Enhanced CNN **Visual Transformers: Token-based Image Representation and Processing for Computer Vision.** [5th Jun 2020].<br> *Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, Peter Vajda.*<br> [[PDF](https://arxiv.org/abs/2006.03677)] **Bottleneck Transformers for Visual Recognition.** [2nd Aug. 2021] [CVPR, 2021].<br> *Aravind Srinivas, Tsung-Yi Lin, Niki Parmar, Jonathon Shlens, Pieter Abbeel, Ashish Vaswani.*<br> [[PDF](https://arxiv.org/abs/2101.11605)] [[Github](https://github.com/rwightman/pytorch-image-models)]
Excerpt of 38,318 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:c2fcd460b7814de1, topic:transformer