Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official Repo For Pixel-LLM Codebase: Sa2VA (T-PAMI-26), SAMTok (CVPR-26), VRT (Arxiv-25), SaSaSa2VA (1-st solution for LSVOS)
| Date | Stars |
|---|---|
| 2026-07-24 | 1650 |
| 2026-07-25 | 1650 |
| 2026-07-28 | 1650 |
| 2026-07-30 | 1650 |
| 2026-07-31 | 1650 |
| 2026-08-06 | 1650 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Pixel LLMs: Pixel-Level Grounded Understanding for Multimodal LLMs
**Pixel LLMs** is a family of projects that bring pixel-level, dense grounded understanding to multimodal LLMs. It is anchored by **Sa2VA** — a unified model that marries SAM-2 with LLaVA for dense grounded understanding of images and videos — together with a growing set of research projects built on top of it.
## Projects
### 🧠 [Sa2VA](./projects/sa2va/README.md) — Marrying SAM2 with LLaVA (IEEE TPAMI 2026)
*Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, Ming-Hsuan Yang*
The core unified model: SAM-2 + MLLM for referring segmentation, grounded conversation, visual prompting, and image/video chat. Supports InternVL2.5/3 and Qwen2.5-VL/Qwen3-VL backbones.
📂 [`projects/sa2va`](./projects/sa2va/README.md) · [📜 arXiv](https://arxiv.org/abs/2501.04001) · [🏠 Page](https://lxtgh.github.io/project/sa2va) · [🤗 Models](https://huggingface.co/collections/ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093)
### 🔍 [VRT](./projects/vrt_sa2va/README.md) — Visual Reasoning Tracer
*Haobo Yuan, Yueyi Sun, Yanwei Li, Tao Zhang, Xueqing Deng, Henghui Ding, Lu Qi, Anran Wang, Xiangtai Li, Ming-Hsuan Yang*
Object-level grounded reasoning built on Sa2VA. Ships **VRT-Bench** (evaluation) and **VRT-80k** (training data).
📂 [`projects/vrt_sa2va`](./projects/vrt_sa2va/README.md) · [📜 arXiv](https://arxiv.org/pdf/2512.05091) · [🏠 Page](https://harboryuan.github.io/visual-reasoning-tracer/) · [🤗 Data](https://huggingface.co/datasets/HarborYuan/VRT-Eval)
### 🧩 [SAMTok](./projects/samtok/README.md) — Representing Any Mask with Two Words (CVPR 2026)
*Yikang Zhou, Tao Zhang, Dengxian Gong, Yuanzheng Wu, Ye Tian, Haochen Wang, Haobo Yuan, Jiacong Wang, Lu Qi, Hao Fei, Anran Wang, Zhuochen Wang, Yujing Wang, Cheng Chen, Shunping Ji, Xiangtai Li*
A unified mask-token interface that lets any MLLM generate and understand masks.
📂 [`projects/samtok`](./projects/samtok/README.md) · [📜 arXiv](https://arxiv.org/abs/2601.16093) · [🏠 Page](https://zhouyiks.github.io/projects/SAMTok/) · [🤗 Models](https://huggingface.co/collections/zhouyik/samtok)
### Extensions
- **[SaSaSa2VA](./projects/sasasa2va/README.md)** — a segmentation-augmented extension of Sa2VA that won **1st place** in the ICCV 2025 LSVOS Challenge RVOS Track 🏅.
- **[Pixel-SAIL](./projects/pixel_sail/README.md)** — single-transformer pixel-level grounding.
## Environment
We manage dependencies with [`uv`](https://docs.astral.sh/uv/). Install it once:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
The environment is defined under [`projects/sa2va`](./projects/sa2va) (`pyproject.toml` + `uv.lock`) and shared across the projects. The quickest way to set it up — with the virtualenv placed in `/tmp` and symlinked back into the project — is the helper script at the repo root:
```bash
bash setup_env.sh # projects/sa2va, --extra=latest
# bash setup_env.sh sa2va legacy # InternVL2.5 or earlier
```
Or do it manually:
```bash
cd projects/sa2va
uv sync --extra=latest # or --extra=legacy
```
Then run training / evaluation from the repository root with the environment activated (`source projects/sa2va/.venv/bin/activate`). See each project's README for project-specific steps.
For tokens / API keys (HuggingFace, OpenRouter), copy the template and fill it in — `setup_env.sh` loads it automatically:
```bash
cp .env.example .env # then edit .env
```
Why `uv`? It treats the environment as code: dependencies are declared in `pyproject.toml` and every transitive package is version-locked in `uv.lock`. The result is a single source of truth that is fully reproducible across machines, trivial to maintain, and recreated exactly with one `uv sync` — no manual `pip install` drift.
## Citation
If you find this repository useful, please consider citing the relevant papers:
```bibtex
@article{sa2va,
Excerpt of 5,079 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:49840b50b1b80156, topic:large-language-models
matched fp:49840b50b1b80156, topic:computer-vision
