Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"
| Date | Stars |
|---|---|
| 2026-07-31 | 398 |
| 2026-08-06 | 398 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
 # Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs [Xin Lai](https://scholar.google.com/citations?user=tqNDPA4AAAAJ&hl), [Zhuotao Tian](https://scholar.google.com/citations?user=mEjhz-IAAAAJ&hl), [Yukang Chen](https://scholar.google.com/citations?user=6p0ygKUAAAAJ&hl), [Senqiao Yang](https://scholar.google.com/citations?user=NcJc-RwAAAAJ&hl), [Xiangru Peng](xxxx), [Jiaya Jia](https://scholar.google.com/citations?user=XPAkzTEAAAAJ&hl=en) [](https://huggingface.co/collections/xinlai/step-dpo-6682e12dfbbb2917c8161df7) [](https://huggingface.co/datasets/xinlai/Math-Step-DPO-10K) [](https://arxiv.org/pdf/2406.18629) [](http://103.170.5.190:7870/) [](licenses/LICENSE) [](licenses/DATA_LICENSE) [](licenses/WEIGHT_LICENSE) This repo provides the implementation of **Step-DPO**, a simple, effective, and data-efficient method for boosting the long-chain reasoning ability of LLMs, with **a data construction pipeline** that yields a **high-quality dataset** containing 10K step-wise preference pairs. Notably, **Step-DPO** boosts the performance of **Qwen2-7B-Instruct** from **53.0%** to **58.6%** on MATH, and **85.5%** to **87.9%** on GSM8K, with as few as **10K data** and **hundreds of training steps**! Moreover, **Step-DPO**, when applied to **Qwen2-72B-Instruct**, achieves scores of **70.8%** and **94.0%** on the test sets of **MATH** and **GSM8K**, respectively, **surpassing a series of closed-source models** without bells and wistles, including GPT-4-1106, Claude-3-Opus, and Gemini-1.5-Pro.  ## TABLE OF CONTENTS 1. [News](#news) 2. [Datasets](#datasets) 3. [Models](#models) 4. [Installation](#installation) 5. [Training](#training) 6. [Evaluation](#evaluation) 7. [Data Construction Pipeline](#data-construction-pipeline) 8. [Deployment](#deployment) 9. [Examples](#examples) 10. [Acknowledgement](#acknowledgement) 11. [Citation](#citation) ## News - [x] [2024.7.7] We release the scripts for [Data Construction Pipeline](#data-construction-pipeline)! You can construct dataset on your own with these scripts! - [x] [2024.7.1] We release the demo of the model [Qwen2-7B-Instruct-Step-DPO](https://huggingface.co/xinlai/Qwen2-7B-Instruct-Step-DPO). Welcome to try it on [Demo](http://103.170.5.190:7870/)! - [x] [2024.6.28] We release the pre-print of [Step-DPO](https://arxiv.org/pdf/2406.18629) and this GitHub repo, including training/evaluation scripts, pre-trained models and data. ## Datasets We build a 10K math preference datasets for Step-DPO, which can be downloaded from the following link. | Dataset | Size | Link | | ------------------------ | ------ | ------------------------------------------------------------ | | xinlai/Math-Step-DPO-10K | 10,795 | 🤗 [Hugging Face](https://huggingface.co/datasets/xinlai/Math-Step-DPO-10K) | ## Models It is notable that the model **Qwen2-72B-Instruct + Step-DPO** could achieve **70.8%** and **94.0%** on MATH and GSM8K test sets. Step-DPO also brings considerable improvement over various models as follows. Welcome to download and use. | Models | Size | MATH | GSM8K | Odyssey-MATH | Link | | :------------------------------ | :--: | :----: | :---: | :---: | :----------------------------------------------------------: | | Qwen2-7B-Instruct | 7B | 53.0
Excerpt of 10,710 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4e71c685ed715ea0, topic:dpo, name:dpo, desc:dpo
matched fp:4e71c685ed715ea0, topic:llm