Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The simplest, fastest repository for training/finetuning small-sized VLMs.
| Date | Stars |
|---|---|
| 2026-07-31 | 4975 |
| 2026-08-01 | 4976 |
| 2026-08-02 | 4976 |
| 2026-08-05 | 4977 |
| 2026-08-06 | 4976 |
Today
-1 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# nanoVLM  <a target="_blank" href="https://colab.research.google.com/github/huggingface/nanoVLM/blob/main/nanoVLM.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> --- > [!TIP] > We have written a [tutorial on nanoVLM](https://huggingface.co/blog/nanovlm) which will guide you through the repository and help you get started in no time. --- > [!NOTE] > We have pushed some more breaking changes on September 9, 2025. These are all the updates to use image splitting and train on multiple nodes. This was used for the ablations of the FineVision release. Some things in the codebase regarding support scripts (eg. the notebook, or memory evals) are propably not working anymore. Similarly to the older trained versions of nanoVLM (similarly to Note below). If you find something that doesn't work anymore please let us know in the Issues or submit a PR! --- > [!NOTE] > We have pushed some breaking changes to the repository on June 4, 2025. To enable us to do smarter packing, we refactored the way image and text embeddings are combined. To keep everything as smooth as possible, we have trained a new nanoVLM-450M with this new pipeline, while leaving the old nanoVLM-222M compatible with the old pipeline If you clone this repository now or pull the updated to your local machine, the default will be the new 450M Model. If you would like a simpler understanding and a simpler codebase, you can use the v0.1 release. This works out of the box with the old 222M model. --- nanoVLM is the simplest repository for training/finetuning a small sized Vision-Language Model with a lightweight implementation in pure PyTorch. The code itself is very readable and approachable, the model consists of a Vision Backbone (`models/vision_transformer.py` ~150 lines), Language Decoder (`models/language_model.py` ~250 lines), Modality Projection (`models/modality_projection.py` ~50 lines) and the VLM itself (`models/vision_language_model.py` ~100 lines) and a simple training loop (`train.py` ~200 lines). Similar to Andrej Karpathy's nanoGPT, we wanted to equip the community with a very simple implementation and training script for Vision Language Models. We do not claim this to be a new SOTA model, rather an educational effort that packs quite a bit of punch if you have the right hardware! You should be able to tweak and play around with the code in no time. ## What can nanoVLM do? The model definition and training logic of this repository fits in ~750 lines, with some more boilerplate logging and parameter loading. Using the [`SigLIP-B/16-224-85M`](https://huggingface.co/google/siglip-base-patch16-224) and [`HuggingFaceTB/SmolLM2-135M`](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) as backbones results in a **222M** nanoVLM. Training this for ~6h on a single H100 GPU on ~1.7M samples of [the cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron) results in an accuracy of 35.3% on MMStar.  It is therefore a simple yet powerful platform to get started with VLMs. Perfect to tinker around with different setups and settings, to explore the capabilities and efficiencies of small VLMs! ## Quick Start You can either clone the repository, setup an environment and start with the scripts, or directly [open in Colab](https://colab.research.google.com/github/huggingface/nanoVLM/blob/main/nanoVLM.ipynb). You can also use the [interactive notebook](./nanoVLM.ipynb) to get started! ## Environment Setup We really like `uv` and recommend using it as your package manager. But feel free to use whichever you prefer. Let's first clone the repository: ```bash git clone https://github.com/huggingface/nanoVLM.git cd nanoVLM ``` If you want to use `uv`: ```bash uv init --bare --python 3.12 uv sync --python 3.12 source .venv/bin/activate uv add torch numpy torchvision pillow datasets huggingface-hub transformers wandb # Optional:
Excerpt of 12,769 characters
Read on GitHub
162
Andrés Marafioti · Hugging Face · Switzerland
114
Aritra Roy Gosthipaty · @huggingface · India
20
Geronimo
15
Kashif Rasul · Germany
12
Brian Li · Advanced Machine Intelligence
6
Lucain · @huggingface
5
5
Kadir Nar · @Vyvo-Labs · Turkey
3
3
Adonai Vera · Voxel51
2
@voxel51
2
2
2
2
1
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:bab3a222f79cc746, llm:description: 'The simplest, fastest repository for training/finetuning small-sized VLMs.' Language: Python. Repo name: nanoVLM (visual-language model).
matched fp:bab3a222f79cc746, llm:description: 'The simplest, fastest repository for training/finetuning small-sized VLMs.' Language: Python. Repo name: nanoVLM (visual-language model).
matched fp:bab3a222f79cc746, llm:description: 'The simplest, fastest repository for training/finetuning small-sized VLMs.' Language: Python. Repo name: nanoVLM (visual-language model).