Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
✨✨ [ICLR 2026] R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning
| Date | Stars |
|---|---|
| 2026-07-31 | 292 |
| 2026-08-04 | 292 |
| 2026-08-05 | 292 |
| 2026-08-06 | 292 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center" width="90%">
<img src="docs/logo.png" width="80%" height="80%">
</p>
<font size=7><div align='center' >
[[📖 arXiv Paper](https://arxiv.org/abs/2505.02835)]
[[📊 R1-Reward Model](https://huggingface.co/yifanzhang114/R1-Reward)]
[[📝 R1-Reward Data](https://huggingface.co/datasets/yifanzhang114/R1-Reward-RL)]
</div></font>
## Training Multimodal Reward Model Through Stable Reinforcement Learning
**R1-Reward** is a comprehensive project focused on enhancing multimodal reward modeling through reinforcement learning. We introduce the **StableReinforce** algorithm and release the **R1-Reward model**, achieving significant performance improvements on established benchmarks.
* **R1-Reward Model:** A state-of-the-art (SOTA) multimodal reward model demonstrating substantial gains (Voting@15):
* **13.5%** improvement on VL Reward-Bench.
* **3.5%** improvement on MM-RLHF Reward-Bench.
* **14.6%** improvement on Multimodal Reward Bench.
* **StableReinforce Algorithm:** A novel reinforcement learning method that enhances the Reinforce++ approach by improving training loss stability, advantage estimation, and reward function design.
* **Open-Source Resources:** We provide the R1-Reward model, the R1-Reward RL training dataset, and inference code for IXC-Reward，MM-RLHF Reward and R1-Reward on the three benchmarks in Figure 1.
<p align="center" width="100%">
<img src="docs/teaser.png" width="100%" height="100%">
</p>
## Table of Contents
1. [Installation](#1-installation)
2. [Data Preparation](#2-data-preparation)
3. [Training with StableReinforce](#3-training-with-stablereinforce)
4. [Evaluation](#4-evaluation)
5. [Usage Example: How to Use R1-Reward](#5-usage-example-how-to-use-r1-reward)
6. [Citation](#6-citation)
7. [Related Projects](#7-related-projects)
## 1. Installation
### **1.1. Clone the Repository:**
```bash
git clone https://github.com/yfzhang114/r1_reward
cd r1_reward
```
### **1.2. Create Conda Environment and Install Dependencies:**
```bash
conda create -n r1_reward python=3.10 -y
conda activate r1_reward
pip install -e .[vllm]
pip install flash_attn --no-build-isolation
```
---
## 2. Data Preparation
**2.1. Download Data:**
Obtain the [📊 R1-Reward-RL Data](https://huggingface.co/datasets/yifanzhang114/R1-Reward-RL).
**2.2. Process Data and Images:**
Process the dataset. For each sample, ensure the corresponding image is saved locally. Update the dataset files (e.g., `.jsonl` files) to include the correct local path to the image for each sample. The image path should be referenced within the `content` list as shown below:
*Example Image Path Entry:*
```json
{
"type": "image",
"image": "/path/to/your/images/edouard-vuillard_the-cake-shop.jpg" // <-- Update this path
}
```
**2.3. Training Data Format:**
The training data should follow this JSON structure, where `message` contains the prompt and evaluation task, and `answer` indicates the preferred response index (`"1"` or `"2"`):
```json
{
"message": "[{\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"You are a highly skilled and impartial evaluator... <evaluation prompt text> ...#### [Response 2]:\\nThe artwork is a print...\"}, {\"type\": \"image\", \"image\": \"/mmu_mllm_hdd/yfzhang/data/RLAIF-V-image/edouard-vuillard_the-cake-shop.jpg\"}]}]", // Note: Full prompt text truncated for brevity
"question": "Give a comprehensive description of this artwork, from its visual elements to its conceptual meaning.",
"answer": ["<answer>2</answer>"] // Indicates Response 2 is preferred
}
```
*(Please refer to the dataset for the full `message` text format)*
## 3. Training with StableReinforce
Follow these steps to train your reward model using the StableReinforce algorithm:
**3.1. Configure Paths:**
In your training script or environment settings, define the following paths:
* `DATASET`: Specify the path to your training dataset.
* `SAVE_PATH`: Specify the directory where the trainExcerpt of 13,944 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:fcbc5f2a85bb7247, llm:Repository title and description: 'R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning' (ICLR 2026). Language: Python. Focus on training multimodal reward models and reinforcement learning.
matched fp:fcbc5f2a85bb7247, llm:Repository title and description: 'R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning' (ICLR 2026). Language: Python. Focus on training multimodal reward models and reinforcement learning.
matched fp:fcbc5f2a85bb7247, llm:Repository title and description: 'R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning' (ICLR 2026). Language: Python. Focus on training multimodal reward models and reinforcement learning.