Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Code for the Molmo2 Vision-Language Model
| Date | Stars |
|---|---|
| 2026-07-31 | 697 |
| 2026-08-04 | 702 |
| 2026-08-06 | 702 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center">
<img src="assets/Molmo2-logo.svg" alt="Molmo2 Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
<br>
<br>
<h1>Molmo 2: State-of-the-art video understanding, pointing, and tracking</h1>
</div>
<p align="center">
<a href="https://github.com/allenai/molmo2/LICENSE">
<img alt="GitHub License" src="https://img.shields.io/github/license/allenai/OLMo">
</a>
<a href="https://allenai.org/blog/molmo2">
<img alt="Blog Post" src="https://img.shields.io/badge/Molmo2-blog-F0529C">
</a>
<a href="https://arxiv.org/abs/2601.10611">
<img alt="Paper URL" src="https://img.shields.io/badge/arxiv-2601.10611-blue">
</a>
<a href="https://huggingface.co/collections/allenai/molmo2">
<img alt="Model Checkpoints" src="https://img.shields.io/badge/%F0%9F%A4%97%20HF-Models-yellow">
</a>
<a href="https://huggingface.co/collections/allenai/molmo2-data">
<img alt="Molmo2 Datasets" src="https://img.shields.io/badge/%F0%9F%A4%97%20HF-Datasets-yellow">
</a>
</p>
This repository is for training and using Ai2's open vision language models, Molmo2 and MolmoPoint.
Molmo2 is state-of-the-art among open-source models and demonstrates exceptional new capabilities in point-driven grounding in single image, multi-image, and video tasks as shown below.
MolmoPoint is an extension with a new architecture for pointing.
This README is mostly concerned with Molmo2, see [MolmoPoint](MOLMO_POINT_README.md)
for how to train MolmoPoint.
<div align="center">
<img src="assets/molmo2_capabilities.png" alt="Molmo2 Capabilites" width="1200" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
</div>
See our [blog post](https://allenai.org/blog/molmo2) or our [paper](https://arxiv.org/abs/2601.10611) for more details about Molmo2.
Huggingface models can be found [here](https://huggingface.co/collections/allenai/molmo2).
## Table of Contents
- [Setup](#setup)
- [Installation](#installation)
- [Docker](#docker)
- [Downloading Data](#downloading-data)
- [Downloading Pretrained Models](#downloading-pretrained-models)
- [Visualizing Data](#visualizing-data)
- [Environment](#environment)
- [Training and Evaluations](#training-and-evaluations)
- [Checkpoints](#checkpoints)
- [Pre-Training](#pre-training)
- [SFT Training](#sft-training)
- [Long-Context SFT Training](#long-context-sft-training)
- [Evaluation](#evaluation)
- [Context Parallel](#context-parallel)
- [Transformers and vLLM](#transformers-and-vllm)
- [Convert Checkpoint to Hugging Face Format](#convert-checkpoint-to-hugging-face-format)
- [Transformers Inference](#transformers-inference)
- [Image Inference Example](#image-inference-example)
- [Video Inference Example](#video-inference-example)
- [MolmoPoint Transformers Inference](#molmopoint-transformers-inference)
- [Image Inference Example](#image-inference-example)
- [Video Inference Example](#video-inference-example)
- [Fast Inference with vLLM](#fast-inference-with-vllm)
- [Install Vision Process Package](#install-vision-process-package)
- [Install vLLM (\>= 0.15.0)](#install-vllm--0150)
- [Run vLLM inference (Gradio Demo)](#run-vllm-inference-gradio-demo)
- [Code](#code)
- [Data Pipeline](#data-pipeline)
- [Message Trees](#message-trees)
- [Packing](#packing)
# Setup
## Installation
We recommend using python >= 3.11
First install [PyTorch](https://pytorch.org) according to the instructions specific to your operating system.
To install dependencies, run:
```bash
git clone https://github.com/allenai/molmo2.git
cd molmo2
pip install torchcodec
pip install -e .[all]
```
It's recommended to install torchcodec separately since it has some complex dependencies that
can break if installed in combination with the others as done using `install -e .[all]`
### Docker
We provide a container with the dependencies (but not the code) pre-installed, pull it with:
`docker pull ghcr.io/allenai/molmExcerpt of 27,419 characters
Read on GitHub
12
2
2
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:c4b09e964c0c6a6a, desc:vision-language, desc:vision language model