Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
World model reinforcement learning for multi-turn VLM agents. RL for vision framework (NeurIPS 2025).
| Date | Stars |
|---|---|
| 2026-08-03 | 491 |
| 2026-08-12 | 493 |
| 2026-08-29 | 494 |
| 2026-08-30 | 493 |
| 2026-09-02 | 494 |
| 2026-09-04 | 495 |
| 2026-09-05 | 496 |
| 2026-09-06 | 497 |
| 2026-09-07 | 498 |
| 2026-09-10 | 498 |
| 2026-09-14 | 499 |
| 2026-09-15 | 501 |
| 2026-09-16 | 503 |
| 2026-09-20 | 504 |
| 2026-09-21 | 505 |
| 2026-09-22 | 504 |
| 2026-09-24 | 505 |
| 2026-09-25 | 506 |
| 2026-09-29 | 508 |
| 2026-10-08 | 509 |
| 2026-10-09 | 510 |
| 2026-10-13 | 512 |
| 2026-10-15 | 514 |
| 2026-10-16 | 514 |
| 2026-10-17 | 515 |
| 2026-10-18 | 515 |
| 2026-10-21 | 516 |
| 2026-10-22 | 515 |
| 2026-10-23 | 516 |
| 2026-10-24 | 517 |
| 2026-10-27 | 518 |
| 2026-10-28 | 518 |
| 2026-10-29 | 518 |
| 2026-11-04 | 518 |
Today
— stars today
This week
— stars this week
This month
+10 stars this month
Momentum
0.0
growth rate 0.00%/day
<h1 align="center">VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents</h1>
<!-- <p align="center" style="font-size: 30px;">
<b>Training VLM agents with multi-turn reinforcement learning</b>
</p>
<p align="center" style="font-size: 10px;">
<b>NeurIPS 2025</b>
</p> -->
<h3 align="center"><b>Training VLM agents with multi-turn reinforcement learning</b></h3>
<h4 align="center"><b>🔥 NeurIPS 2025 🔥</b></h4>
<p align="center" style="font-size: 16px;">
Kangrui Wang*, Pingyue Zhang*, Zihan Wang*, Yaning Gao*, Linjie Li*, Qineng Wang, Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang, Lijuan Wang, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Yejin Choi, <a href="https://limanling.github.io/">Manling Li</a>
</p>
<p align="center" style="font-size: 12px;"><i>(* equal contribution)</i></p>
<p align="center">
<a href="https://arxiv.org/abs/2510.16907"><img src="https://img.shields.io/badge/📜_Paper-B31B1B?style=for-the-badge&logo=arXiv&logoColor=white" alt="Paper"></a>
<a href="https://vagen.readthedocs.io/en/latest"><img src="https://img.shields.io/badge/📚_Documentation-4285F4?style=for-the-badge&logoColor=white" alt="Documentation"></a>
<a href="https://mll-lab.notion.site/vagen"><img src="https://img.shields.io/badge/📝_Blog-FF5722?style=for-the-badge&logoColor=white" alt="Blog"></a>
<a href="https://wandb.ai/ragen-V/vagen-final/reports/VAGEN-Experiment-Log--VmlldzoxMzM2NzczNA?accessToken=c9539vj7s3yxh8qu4rykmgi1kz47935mu9pvkind70m2tt6bdin6tx263ec7yqei"><img src="https://img.shields.io/badge/📊_Experiment_Log-FB8C00?style=for-the-badge&logoColor=white" alt="Experiment Log"></a>
<a href="https://vagen-ai.github.io/"><img src="https://img.shields.io/badge/🌐_Website-00C851?style=for-the-badge&logoColor=white" alt="Website"></a>
</p>
**VAGEN** is a **reinforcement learning (RL) framework that trains multi-turn VLM agents** (vision-language model agents) to build an internal **world model** through explicit visual state reasoning. In addition to task success, VAGEN can reinforce **StateEstimation** ("what is the current state?") and **TransitionModeling** ("what comes next?") with an LLM-as-Judge world-modeling reward. The reusable harness, environment, model, rollout, and evaluation boundaries keep those signals independent from the selected training algorithm.
<div style="width:100%; overflow-x:auto;">
<table style="width:100%;">
<tr>
<td align="center" style="width:20%;"><br>
<img src="https://github.com/user-attachments/assets/6d72800a-9b4d-45ec-b528-ac81efb93966" style="width:72%;"/><br>
<img src="https://github.com/user-attachments/assets/6f283f99-fa15-4e26-9f99-6649a7d72374" style="width:72%;"/><br>
<b>FrozenLake</b>
</td>
<td align="center" style="width:20%;"><br>
<img src="https://github.com/user-attachments/assets/b364e6c9-4c2c-46d0-afca-ee42f271c59c" style="width:75%;"/><br>
<img src="https://github.com/user-attachments/assets/65662eb0-9440-4555-9436-8b9272791ac4" style="width:75%;"/><br>
<b>Navigation</b>
</td>
<td align="center" style="width:20%;"><br>
<img src="https://github.com/user-attachments/assets/145352b5-3a9e-4248-bb94-d3fa46e6c493" style="width:80%;"/><br>
<img src="https://github.com/user-attachments/assets/676de052-37d6-4c99-a7eb-200a58d11ed4" style="width:80%;"/><br>
<b>Sokoban</b>
</td>
<td align="center" style="width:20%;"><br>
<img src="https://github.com/user-attachments/assets/c597f17d-5c62-4319-bdaa-b7fa8e4564e1" style="width:80%;"/><br>
<img src="https://github.com/user-attachments/assets/f61ea55c-ea79-4ead-9345-45be06d24e81" style="width:80%;"/><br>
<b>ManiSkill</b>
</td>
<td align="center" style="width:20%;"><br>
<img src="https://github.com/user-attachments/assets/8646da5f-69be-4283-a078-969f9b8f3f3b" style="width:92%;"/><br>
<img src="https://github.com/user-attachments/assets/691b896a-ce30-4acc-ac49-af2Excerpt of 19,951 characters
Read on GitHub
Kangrui Wang
406
16
2
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:6bdba159857f1eee, llm:Repository topics and description: 'agent, embodied-ai, multi-turn-reinforcement-learning, reinforcement-learning, rl-for-vision, vision-language-model, vlm-agent, world-models' and README: 'VAGEN is a reinforcement learning (RL) framework that trains multi-turn VLM agents ... build an internal world model through explicit visual state reasoning.'
matched fp:6bdba159857f1eee, llm:Repository topics and description: 'agent, embodied-ai, multi-turn-reinforcement-learning, reinforcement-learning, rl-for-vision, vision-language-model, vlm-agent, world-models' and README: 'VAGEN is a reinforcement learning (RL) framework that trains multi-turn VLM agents ... build an internal world model through explicit visual state reasoning.'
matched fp:6bdba159857f1eee, llm:Repository topics and description: 'agent, embodied-ai, multi-turn-reinforcement-learning, reinforcement-learning, rl-for-vision, vision-language-model, vlm-agent, world-models' and README: 'VAGEN is a reinforcement learning (RL) framework that trains multi-turn VLM agents ... build an internal world model through explicit visual state reasoning.'
matched fp:6bdba159857f1eee, llm:Repository topics and description: 'agent, embodied-ai, multi-turn-reinforcement-learning, reinforcement-learning, rl-for-vision, vision-language-model, vlm-agent, world-models' and README: 'VAGEN is a reinforcement learning (RL) framework that trains multi-turn VLM agents ... build an internal world model through explicit visual state reasoning.'