Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ICLR 2026] VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning
| Date | Stars |
|---|---|
| 2026-07-31 | 351 |
| 2026-08-01 | 351 |
| 2026-08-02 | 351 |
| 2026-08-05 | 350 |
| 2026-08-06 | 350 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning > Current VLMs are primarily used for visual captioning or visual QA tasks. In this project, we take a step further by demonstrating the potential of a single VLM to solve diverse vision tasks. We hope this work will advance the frontier of VLM research and expand the boundaries of what these models can achieve. Paper: [📖 VisionReasoner](https://arxiv.org/pdf/2505.12081) [📖 Seg-Zero](https://arxiv.org/pdf/2503.06520) HuggingFace Daily: [🤗 VisionReasoner](https://huggingface.co/papers/2505.12081) Model: [🤗 VisionReasoner-7B](https://huggingface.co/Ricky06662/VisionReasoner-7B) [🤗 TaskRouter-1.5B](https://huggingface.co/Ricky06662/TaskRouter-1.5B) Relative Link: [Seg-Zero![[code]](https://img.shields.io/github/stars/JIA-Lab-research/Seg-Zero)](https://github.com/JIA-Lab-research/Seg-Zero) Overview of VisionReasoner: <div align=center> <img width="98%" src="assets/overview.png"/> </div> VisionReasoner demonstrates following features: 1. **VisionReasoner** is a unified framework for visual perception tasks. Through carefully crafted rewards and training strategy, VisionReasoner has strong multi-task capability, addressing diverse visual perception tasks within a shared model. 2. We select several representative tasks to evaluate models unified visual ability, including detection tasks (e.g., [COCO](https://cocodataset.org/#home), [RefCOCOg](https://github.com/lichengunc/refer)), segmentation tasks (e.g., [ReasonSeg](https://github.com/JIA-Lab-research/LISA)), counting tasks (e.g., [CountBench](https://teaching-clip-to-count.github.io/)) and VQA tasks (e.g. [DocVQA](https://www.docvqa.org/)). 3. Experimental results show that VisionReasoner achieves superior performance across ten diverse visual perception tasks within a single unified framework, outperforming baseline models by a significant margin. 4. We have reformulated dozens of visual task types categoried in [Papers With Code](https://paperswithcode.com/datasets?mod=images&page=1). Please refer to [task categorization](task_categorization.md) for details. These task types are categoried as four fundamental task types: detection, segmentation, counting and VQA. More supported task types and more fundamental task types can be added in this framework, such as 3D or medical image processing. ## News [May 17th, 2025] 🔥 [📖 Paper](https://arxiv.org/pdf/2505.12081) is coming! [May 17th, 2025] 🔥 VisionReasoner is coming! VisionReasoner is based on our previous [Seg-Zero](https://github.com/JIA-Lab-research/Seg-Zero). ## Contents - [Model](#model) - [Installation](#installation) - [Inference](#inference) - [Hybrid Mode](#hybrid-mode) - [Image Generation](#image-generation) - [Evaluation](#evaluation) - [Training](#training) - [Citation](#citation) - [Acknowledgement](#acknowledgement) ## Model <div align=center> <img width="98%" src="assets/pipeline.png"/> </div> VisionReasoner model incorporates a reasoning module, which processing image and locates targeted objects, and a segmentation module that produces segmentation masks if needed. Besides, we also train a task router that convert diverse vision tasks into given four fundamental task types. <!-- ## Examples <div align=center> <img width="98%" src="assets/examples.png"/> </div> --> ## Installation > [!NOTE] > If you train VisionReasoner using codes in [Seg-Zero](https://github.com/JIA-Lab-research/Seg-Zero), you can directly use the environment of the training codes. ```bash git clone https://github.com/JIA-Lab-research/VisionReasoner.git cd VisionReasoner conda create -n visionreasoner_test python=3.12 conda activate visionreasoner_test pip3 install torch torchvision pip install -r requirements.txt ``` ## Inference Download model using the following scripts: ```bash mkdir pretrained_models cd pretrained_models git lfs install git clone https://huggingface.co/Ricky06662/VisionRea
Excerpt of 13,562 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:ec7ff016dc14696f, llm:Repository topics and description: 'counting-objects, multimodal, multimodal-large-language-models, object-detection, reasoning-language-models, reinforcement-learning, segmentation, visual-perception' and paper title 'VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning' (ICLR 2026).
matched fp:ec7ff016dc14696f, llm:Repository topics and description: 'counting-objects, multimodal, multimodal-large-language-models, object-detection, reasoning-language-models, reinforcement-learning, segmentation, visual-perception' and paper title 'VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning' (ICLR 2026).
matched fp:ec7ff016dc14696f, llm:Repository topics and description: 'counting-objects, multimodal, multimodal-large-language-models, object-detection, reasoning-language-models, reinforcement-learning, segmentation, visual-perception' and paper title 'VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning' (ICLR 2026).
matched fp:ec7ff016dc14696f, llm:Repository topics and description: 'counting-objects, multimodal, multimodal-large-language-models, object-detection, reasoning-language-models, reinforcement-learning, segmentation, visual-perception' and paper title 'VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning' (ICLR 2026).