Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official implementation of paper: SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
| Date | Stars |
|---|---|
| 2026-07-31 | 330 |
| 2026-08-06 | 330 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center">
# *SFT Memorizes, RL Generalizes*: <br>A Comparative Study of Foundation Model Post-training
<p>
<img src="assets/teaser.png" alt="Cambrian" width="500" height="auto">
</p>
<a href="https://arxiv.org/abs/2501.17161v1" target="_blank">
<img alt="arXiv" src="https://img.shields.io/badge/arXiv-SFT vs RL-red?logo=arxiv" height="25" />
</a>
<a href="https://tianzhechu.com/SFTvsRL/" target="_blank">
<img alt="Website" src="https://img.shields.io/badge/🌎_Website-tianzhechu.com/SFTvsRL-blue.svg" height="25" />
</a>
<a href="https://huggingface.co/collections/tianzhechu/sftvsrl-models-and-data-6797ba6de522c7de7fcb80ba" target="_blank">
<img alt="HF Model: Cambrian-1" src="https://img.shields.io/badge/%F0%9F%A4%97%20_Model-Checkpoints&Data-ffc107?color=ffc107&logoColor=white" height="25" />
</a>
<div style="font-family: charter; text-align: center; margin: 0 auto;">
<a href="https://tianzhechu.com/" class="author-link" target="_blank">Tianzhe Chu*</a>  
<a href="https://yx-s-z.github.io/" class="author-link" target="_blank">Yuexiang Zhai*</a>  
<a href="https://jihanyang.github.io/" class="author-link" target="_blank">Jihan Yang</a>  
<a href="https://tsb0601.github.io/petertongsb/" class="author-link" target="_blank">Shengbang Tong</a>  
<br>
<a href="https://www.sainingxie.com/" class="author-link" target="_blank">Saining Xie</a>  
<a href="https://webdocs.cs.ualberta.ca/~dale/" class="author-link" target="_blank">Dale Schuurmans</a>  
<a href="https://cs.stanford.edu/~quocle/" class="author-link" target="_blank">Quoc V. Le</a>  
<a href="https://people.eecs.berkeley.edu/~svlevine/" class="author-link" target="_blank">Sergey Levine</a>  
<a href="https://people.eecs.berkeley.edu/~yima/" class="author-link" target="_blank">Yi Ma</a>  
</div>
<br>
</div>
> *Misc: We prompt DALL-E 3 via "Conceptual figure of 'SFT Memorizes, RL Generalizes', with trendlines and style of Hong Kong" but somehow skycrapters dominate the picture...*
## Release
- [02/24/25] Support API Evaluator. Use our environments to evaluate your API-based models~
- [02/8/25] We add [SFT scripts](sft/) and text-only [SFT data](https://huggingface.co/datasets/tianzhechu/SFTvsRL_Data). Still updating~
- [01/28/25] Excited to shout out our paper *SFT Memorizes, RL Generalizes*! We release the environments, training scripts, evaluation scripts, SFT data, and initial checkpoints.
## Installation
### Prepare
Our codebase is tested on H800 servers with <code>python 3.13.0</code> <code>torch 2.5.1+cu124</code>.
1. Clone this repository and navigate to into the codebase
```Shell
git clone https://github.com/LeslieTrue/SFTvsRL.git
cd SFTvsRL
```
2. Install Packages
```Shell
conda create -n SFTvsRL python==3.13 -y
conda activate SFTvsRL
pip install -r requirements.txt
cd gym
pip install -e . # install gym environment
cd ..
```
### Download Initial Checkpoints (Optional)
We instantiate RL experiments on top of SFT initialized checkpoints to guarantee model's basic instruction following capabilities. We provide all 4 initial checkpoints for \{GeneralPoints, V-IRL\}X\{Language (-L), Vision-Language (-VL)\}.
```Shell
huggingface-cli download tianzhechu/GP-L-Init --local-dir YOUR_LOCAL_DIR
huggingface-cli download tianzhechu/GP-VL-Init --local-dir YOUR_LOCAL_DIR
huggingface-cli download tianzhechu/VIRL-L-Init --local-dir YOUR_LOCAL_DIR
huggingface-cli download tianzhechu/VIRL-VL-Init --local-dir YOUR_LOCAL_DIR
```
It's optional to download these checkpoints via huggingface CLI. You may directly specify <code>repo_name</code> as <code>CKPT_NAME</code> in shell scripts.
## Getting Started
1. Install packages and prepare the initial checkpoints (optional).
- Check [here](https://huggingface.cExcerpt of 7,185 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:7fdf51b6577457af, llm:Repository description: 'Official implementation of paper: SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training' (SFT vs RL for post-training of foundation models). Language: Python.
matched fp:7fdf51b6577457af, llm:Repository description: 'Official implementation of paper: SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training' (SFT vs RL for post-training of foundation models). Language: Python.
matched fp:7fdf51b6577457af, llm:Repository description: 'Official implementation of paper: SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training' (SFT vs RL for post-training of foundation models). Language: Python.