Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[EMNLP 2025] ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents
| Date | Stars |
|---|---|
| 2026-07-31 | 669 |
| 2026-08-02 | 669 |
| 2026-08-06 | 669 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<!-- # <div align="center">ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents<div> -->
<!-- # <div align="center">ViDoRAG<div>
<div align="center">
<strong>
<h3>Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents</h3>
</strong>
</div> -->
# ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents
<div align="center">
<a href='https://huggingface.co/datasets/autumncc/ViDoSeek'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Datasets-green'></a>
<a href='https://arxiv.org/abs/2502.18017'><img src='https://img.shields.io/badge/Paper-arXiv-red'></a>
</div>
<!-- <p align="center">•
<a href="#-Overview"> 🚀Overview </a> •
<a href="#-visrag-pipeline">✨ VisRAG Pipeline</a> •
<a href="#%EF%B8%8F-setup">⚙️ Setup</a> •
<a href="#%EF%B8%8F-training">⚡️ Training</a>
</p> -->
<div align="center">
<p align="center">
<img src="assets/dataset.jpg" width="65%" height="50%" />
</p>
</div>
## 🔥 News
- 🎉 We have released the evaluation code, allowing you to customize your own evaluation pipeline.
- 🎉 Our framework integrates various embedding models, enabling you to create your own retriever.
- 🎉 We have released the ViDoSeek dataset, which is suitable for Retrieval-augmented Generation in the large visually rich document collection.
## 🚀Overview
- We introduce **ViDoSeek**, a benchmark specifically designed for visually rich document retrieval-reason-answer, fully suited for evaluation of RAG within large document corpus.
- We propose **ViDoRAG**, a novel RAG framework that utilizes a multi-agent, actor-critic paradigm for iterative reasoning, enhancing the noise robustness of generation models.
- We introduce a GMM-based multi-modal hybrid retrieval strategy to effectively integrate visual and textual pipelines.
- Extensive experiments demonstrate the effectiveness of our method. ViDoRAG significantly outperforms strong baselines, achieving over 10% improvement, thus establishing a new state-of-the-art on ViDoSeek.
## 🔍ViDoSeek Dataset
We release our ViDoSeek dataset which designed for visually rich document retrieval-reason-answer. In ViDoSeek, each query has a unique answer and specific reference pages.
The provided JSON structure includes a unique identifier (uid) to distinguish queries, the query content (query), a reference answer (reference_answer), and metadata (meta_info) containing the original file name (file_name), reference page numbers (reference_page), data source type (source_type), and query type (query_type):
```json
{
"uid": "04d8bb0db929110f204723c56e5386c1d8d21587_2",
"query": "What is the temperature of Steam explosion of Pretreatment for Switchgrass and Sugarcane bagasse preparation?",
"reference_answer": "195-205 Centigrade",
"meta_info": {
"file_name": "Pretreatment_of_Switchgrass.pdf",
"reference_page": [10, 11],
"source_type": "Text",
"query_type": "Multi-Hop"
}
}
```
You can use Git LFS to download annotation files and original documents from Hugging Face or ModelScope. The format of the files can refer to `./data/ExampleDataset`.
### Download the dataset
```bash
bash scripts/ViDoSeek_down.sh
```
Then, you can use the following script to convert the original file into images:
```python
python ./scripts/pdf2images.py
```
Optionally, you can use OCR models or Vision-Language Models (VLMs) to recognize text within images:
```python
## triditional OCR models
python ./scripts/ocr_triditional.py
## VLMs as ocr models (Optional)
python ./scripts/ocr_vlms.py
```
## 💻 Running ViDoRAG
ViDoRAG is a novel multi-agent RAG framework tailored for complex reasoning across visual documents. ViDoRAG employs a Gaussian Mixture Model (GMM)-based hybrid strategy to effectively handle multi-modal retrieval.
<div align="center">
<p align="center">
<img src="assets/pipeline.jpg" width="80%" height="50%" />
</p>
</diExcerpt of 8,140 characters
Read on GitHub
5
2
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a4b2f0cc0c00ea7a, desc:retrieval-augmented generation, desc:retrieval augmented, desc:document retrieval