Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Implementing Vi(sion)T(transformer)
| Date | Stars |
|---|---|
| 2026-07-24 | 460 |
| 2026-07-25 | 460 |
| 2026-07-28 | 460 |
| 2026-07-30 | 460 |
| 2026-08-06 | 460 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Implementing Vi(sual)T(transformer) in PyTorch
Hi guys, happy new year! Today we are going to implement the famous **Vi**(sual)**T**(transformer) proposed in [AN IMAGE IS WORTH 16X16 WORDS:
TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE](https://arxiv.org/pdf/2010.11929.pdf).
Code is here, an interactive version of this article can be downloaded from [here](https://github.com/FrancescoSaverioZuppichini/ViT).
ViT will be soon available on my **new computer vision library called [glasses](https://github.com/FrancescoSaverioZuppichini/glasses)**
This is a technical tutorial, not your normal medium post where you find out about the top 5 secret pandas functions to make you rich.
So, before beginning, I highly recommend you to:
- have a look at the amazing [The Illustrated Transformer
](https://jalammar.github.io/illustrated-transformer/) website
- watch [Yannic Kilcher video about ViT](https://www.youtube.com/watch?v=TrdevFK_am4&t=1000s)
- read [Einops](https://github.com/arogozhnikov/einops/) doc
So, ViT uses a normal transformer (the one proposed in [Attention is All You Need](https://arxiv.org/abs/1706.03762)) that works on images. But, how?
The following picture shows ViT's architecture
The input image is decomposed into 16x16 flatten patches (the image is not in scale). Then they are embedded using a normal fully connected layer, a special `cls` token is added in front of them and the `positional encoding` is summed. The resulting tensor is passed first into a standard Transformer and then to a classification head. That's it.
The article is structure into the following sections:
- Data
- Patches Embeddings
- CLS Token
- Position Embedding
- Transformer
- Attention
- Residuals
- MLP
- TransformerEncoder
- Head
- ViT
We are going to implement the model block by block with a bottom-up approach. We can start by importing all the required packages
```python
import torch
import torch.nn.functional as F
import matplotlib.pyplot as plt
from torch import nn
from torch import Tensor
from PIL import Image
from torchvision.transforms import Compose, Resize, ToTensor
from einops import rearrange, reduce, repeat
from einops.layers.torch import Rearrange, Reduce
from torchsummary import summary
```
Nothing fancy here, just PyTorch + stuff
## Data
First of all, we need a picture, a cute cat works just fine :)
```python
img = Image.open('./cat.jpg')
fig = plt.figure()
plt.imshow(img)
```
Then, we need to preprocess it
```python
# resize to imagenet size
transform = Compose([Resize((224, 224)), ToTensor()])
x = transform(img)
x = x.unsqueeze(0) # add batch dim
x.shape
```
torch.Size([1, 3, 224, 224])
## Patches Embeddings
The first step is to break-down the image in multiple patches and flatten them.
Quoting from the paper:
This can be easily done using einops.
```python
patch_size = 16 # 16 pixels
pathes = rearrange(x, 'b c (h s1) (w s2) -> b (h w) (s1 s2 c)', s1=patch_size, s2=patch_size)
```
Now, we need to project them using a normal linear layer
We can create a `PatchEmbedding` class to keep our code nice and clean
```python
class PatchEmbedding(nn.Module):
def __init__(self, in_channels: int = 3, patch_size: int = 16, emb_size: int = 768):
self.patch_size = patch_size
super().__init__()
self.projection = nn.Sequential(
# break-down the image in s1 x s2 patches and flat them
Rearrange('b c (h s1) (w s2) -> b (h w) (s1 s2 c)', s1=patExcerpt of 31,881 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:780d44e74f295656, topic:computer-vision, readme:computer vision
matched fp:780d44e74f295656, topic:deep-learning