Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Multimodal Retrieval-augmented Generation Framework Built by Tongyi Lab, Alibaba Group.
| Date | Stars |
|---|---|
| 2026-07-31 | 972 |
| 2026-08-04 | 974 |
| 2026-08-06 | 974 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# <div align="center">✨Moving Towards Next-Generation RAG via Multi-Modal Agentic Reinforcement Learning</div>
<div align="center">
<p><strong>A Multi-Turn Multi-Modal Agent Training Framework</strong></p>
<a href="https://arxiv.org/pdf/2602.12735v1" target="_blank"><img src=https://img.shields.io/badge/arXiv-paper_VimRAG-red></a>
<a href="https://arxiv.org/pdf/2505.22019" target="_blank"><img src=https://img.shields.io/badge/arXiv-paper_VRAG-red></a>
<br>
<a href="https://huggingface.co/collections/Alibaba-NLP/vrag" target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-VRAG_Collection-blue></a>
<a href="https://huggingface.co/Qiuchen-Wang/Qwen2.5-VL-7B-VRAG" target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-VRAG_Model-blue></a>
<br>
<a href="https://huggingface.co/datasets/Qiuchen-Wang/ViDoSeek" target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-ViDoSeek_Benchmark-blue></a>
<a href="https://huggingface.co/datasets/Alibaba-NLP/xvbench" target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-XVBench_Benchmark-blue></a>
<br>
<a href="https://www.modelscope.cn/collections/iic/VRAG" target="_blank"><img src=https://img.shields.io/badge/ModelScope-VRAG_Collection-624aff?logo=modelscope></a>
<a href="https://www.modelscope.cn/models/iic/Qwen2.5-VL-7B-VRAG" target="_blank"><img src=https://img.shields.io/badge/ModelScope-VRAG_Model-624aff?logo=modelscope></a>
<a href="https://www.modelscope.cn/datasets/iic/ViDoSeek" target="_blank"><img src=https://img.shields.io/badge/ModelScope-ViDoSeek_Benchmark-624aff?logo=modelscope></a>
<!-- <a href="https://huggingface.co/datasets/Alibaba-NLP/xvbench" target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-XVBench_Benchmark-blue></a> -->
</div>
---
<div align="center">
<p align="center">
<img src="assets/compare.png" width="90%" height="100%" />
</p>
</div>
---
## 📑 Table of Contents
- [✨Moving Towards Next-Generation RAG via Multi-Modal Agentic Reinforcement Learning](#moving-towards-next-generation-rag-via-multi-modal-agentic-reinforcement-learning)
- [📑 Table of Contents](#-table-of-contents)
- [🔥 News](#-news)
- [🚀 Overview \& New Feature](#-overview--new-feature)
- [⚙️ Dependencies](#️-dependencies)
- [🚀 Quick Start](#-quick-start)
- [One-Command Launch](#one-command-launch)
- [🔍 Build Your Own Retriever](#-build-your-own-retriever)
- [Step 1: Prepare Corpus](#step-1-prepare-corpus)
- [Step 2: Build Index](#step-2-build-index)
- [Step 3: Start Search Engine API](#step-3-start-search-engine-api)
- [💻 Run Demo](#-run-demo)
- [VimRAG Demo (Recommended)](#vimrag-demo-recommended)
- [VRAG Demo (Local Model)](#vrag-demo-local-model)
- [⚙️ Model Training](#️-model-training)
- [VRAG-RL](#vrag-rl)
- [VimRAG](#vimrag)
- [📁 Project Structure](#-project-structure)
- [🙏 Acknowledgments](#-acknowledgments)
- [📝 Citation](#-citation)
- [🔗 Related Projects](#-related-projects)
---
## 🔥 News
- ⏳ The project is still under ongoing development, and the training code of VimRAG will be available after being reviewed by the company.
- 🎉 We have released the report of the **VimRAG**.
- 🎉 We have released the retriever based on FAISS, enabling retrieval with [GVE embedding](https://huggingface.co/Alibaba-NLP/GVE-7B) and [Qwen3-VL-Embedding](https://github.com/QwenLM/Qwen3-VL-Embedding).
- 🎉 We have released the demo of **VRAG-RL**, allowing you to customize your own VRAG.
- 🎉 Our framework integrates SOTA visual embedding models, enabling you to create your own retriever.
---
## 🚀 Overview & New Feature
- We introduce **VimRAG**, a novel framework tailored for multimodal Retrieval-Augmented Reasoning across text, images, and videos.
- We propose the Multimodal Memory Graph and Graph-Guided Policy Optimization, modeling the reasoning process as a dynamic directed acyclic grapExcerpt of 13,590 characters
Read on GitHub
34
2
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:31734ab630a5cb8c, desc:retrieval-augmented generation, desc:retrieval augmented